Jiez19971225‘s Blog

做自己爱做的事

0%

实现一个jsonParser

发表于 更新于 分类于   |   字数统计: 3.3k(字)   |   阅读时长: 18(分)

想要学习编译原理这门课已经很久了，自己之前也有尝试看龙书或者听公开课，可是这样学到的公式性的知识太多，每次都没有坚持学习下去。

放假这几天学习了极客时间上的《编译原理之美 》这门专栏，把讲编译器前端部分的看完了。这个老师讲的是真的好，比较浅显的语言来讲解教科书上用大段公式说明的知识，同时结合了大量的实践内容，帮助我快速了解了编译原理的核心要点的同时，同时教授了我动手的能力。

之前面试 bilibili 的时候，面试官就问过我如何用代码实现验证一个 json 的有效性，当时回答很糟糕，完全没有使用编译技术这方面的意识。

其实编译技术就很好处理这种问题，走一边词法分析和语法分析的流程，在解析 token 或者 构建 ast 的过程中如果出错了，那么这个 json 就是非法的了。

既是为了练手，也是填上之前面试的这个坑，下面就简单实现一个 jsonParser。

写在前面

写一个 jsonParse 其实很简单，但是自己还是花了不少时间来做这件事，也走了一些弯路，还抄了不少参考资料才写好的 😓。

资料列表：

首先要解析 json，我们必须要了解 json 的语法规则。接触了 json 这么久，我还是第一次看 json 的语法规则定义，有不少新发现的。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
json
  element

value
  object
  array
  string
  number
  "true"
  "false"
  "null"

object
  '{' ws '}'
  '{' members '}'

members
  member
  member ',' members

member
  ws string ws ':' element

array
  '[' ws ']'
  '[' elements ']'

elements
  element
  element ',' elements

element
  ws value ws

string
  '"' characters '"'

characters
  ""
  character characters

character
  '0020' . '10FFFF' - '"' - '\'
  '\' escape

escape
  '"'
  '\'
  '/'
  'b'
  'f'
  'n'
  'r'
  't'
  'u' hex hex hex hex

hex
  digit
  'A' . 'F'
  'a' . 'f'

number
  integer fraction exponent

integer
  digit
  onenine digits
  '-' digit
  '-' onenine digits

digits
  digit
  digit digits

digit
  '0'
  onenine

onenine
  '1' . '9'

fraction
  ""
  '.' digits

exponent
  ""
  'E' sign digits
  'e' sign digits

sign
  ""
  '+'
  '-'

ws
  ""
  '0020' ws
  '000A' ws
  '000D' ws
  '0009' ws

通过看规范，我们知道 json 支持 \u0020-\uffff unicode 的，支持部分转义字符，支持数字的科学计数法表示，这些在我们写 parser 时都是要考虑进去的。

规则定义属性名是 string，也就是可以是空串，或者包含转义字符。

1
2
3
4
{
  "": "",
  "\\n": "1"
}

这样的 json 是合法的，至少我之前是想不到的。

具体实现

lexer 的写法参考了专栏的示例，用了 DFA 的方式，状态转移图直接参考 json 官网的弹珠图就好了。

举个典型的，数字的状态转移。

number的弹珠图

我们定义如下状态

NumberMinus,
NumberZeroStart,
NumberNormal,
NumberDot,
NumberAfterDot,
NumberE,
NumberESign,
NumberAfterE,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
case DFAState.NumberMinus:
  if (ch === CharacterCodes._0) {
    this.state = DFAState.NumberZeroStart;
    this.token.value += this.getCh();
  } else if (this.isDigit(ch)) {
    this.state = DFAState.NumberNormal;
    this.token.value += this.getCh();
  } else {
    throw new ErrorObject(LexErrorType.InvalidNumber);
  }
  break;
case DFAState.NumberZeroStart:
  if (ch === CharacterCodes.dot) {
    this.state = DFAState.NumberAfterDot;
    this.token.value += this.getCh();
  } else {
    this.initState(ch);
  }
  break;
case DFAState.NumberNormal:
  if (this.isDigit(ch)) {
    this.token.value += this.getCh();
  } else if (ch === CharacterCodes.dot) {
    this.state = DFAState.NumberDot;
    this.token.value += this.getCh();
  } else if (ch === CharacterCodes.e || ch === CharacterCodes.E) {
    this.state = DFAState.NumberE;
    this.token.value += this.getCh();
  } else {
    this.initState(ch);
  }
  break;
case DFAState.NumberDot:
  if (this.isDigit(ch)) {
    this.token.value += this.getCh();
    this.state = DFAState.NumberAfterDot;
  } else {
    throw new ErrorObject(LexErrorType.InvalidNumber);
  }
  break;
case DFAState.NumberE:
  if (this.isDigit(ch)) {
    this.token.value += this.getCh();
    this.state = DFAState.NumberAfterE;
  } else if (
    ch === CharacterCodes.plus ||
    ch === CharacterCodes.minus
  ) {
    this.state = DFAState.NumberESign;
  } else {
    throw new ErrorObject(LexErrorType.InvalidNumber);
  }
  break;
case DFAState.NumberESign:
  if (this.isDigit(ch)) {
    this.token.value += this.getCh();
    this.state = DFAState.NumberAfterE;
  } else {
    throw new ErrorObject(LexErrorType.InvalidNumber);
  }
  break;
case DFAState.NumberAfterDot:
  if (ch === CharacterCodes.e || ch === CharacterCodes.E) {
    this.state = DFAState.NumberE;
    this.token.value += this.getCh();
  } else if (this.isDigit(ch)) {
    this.token.value += this.getCh();
  } else {
    this.initState(ch);
  }
  break;
case DFAState.NumberAfterE:
  if (this.isDigit(ch)) {
    this.token.value += this.getCh();
  } else {
    this.initState(ch);
  }
  break;


initState(ch: any) {
  // 省略。。。。
  this.token = new Token();
  if (ch === CharacterCodes.minus) {
    this.state = DFAState.NumberMinus;
    this.token.type = TokenType.NumberType;
    this.token.value += this.getCh();
  } else if (ch === CharacterCodes._0) {
    this.state = DFAState.NumberZeroStart;
    this.token.type = TokenType.NumberType;
    this.token.value += this.getCh();
  } else if (this.isDigit(ch)) {
    this.state = DFAState.NumberNormal;
    this.token.type = TokenType.NumberType;
    this.token.value += this.getCh();
  } // 省略。。。
}

只是将图中状态的转移用代码的形式表现出来，做这类工作的时候，分析的过程比编码要重要的多。

parser 也很简单，用递归下降的方式很容易实现。

完整代码如下，只是实现了基本的功能，没做什么错误处理

lexer

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
import { CharacterCodes } from "./CharacterCodes";
import { Token, TokenType } from "./token";

enum DFAState {
  Initial,
  OpenBracket,
  CloseBracket,
  OpenBraces,
  CloseBraces,
  Colon,
  Comma,
  NumberMinus,
  NumberZeroStart,
  NumberNormal,
  NumberDot,
  NumberAfterDot,
  NumberE,
  NumberESign,
  NumberAfterE,
  StringNormal,
  StringSlash,
  StringSlashU,
  StringInHex,
  StringEnd,
  NullType,
  TrueType,
  FalseType,
}

const nullSteps = ["n", "nu", "nul", "null"];
const trueSteps = ["t", "tr", "tru", "true"];
const falseSteps = ["f", "fa", "fal", "fals", "false"];

enum LexErrorType {
  EOF,
  InvalidKeyword,
  InvalidNumber,
  InvalidStringCharacter,
  UnexeceptedCharacter,
}

const wschar = [
  CharacterCodes.space,
  CharacterCodes.lineFeed,
  CharacterCodes.tab,
  CharacterCodes.formFeed,
  CharacterCodes.backspace,
  CharacterCodes.carriageReturn,
];

export class ErrorObject {
  type!: LexErrorType;
  constructor(type: LexErrorType) {
    this.type = type;
  }
}

export class Lexer {
  script: string;
  pos: number = 0;
  state: DFAState = DFAState.Initial;
  token: Token = new Token();
  tokenList: Token[] = [];
  hexTemp = "";

  isDigit(ch: number): boolean {
    return ch > CharacterCodes._0 && ch < CharacterCodes._9;
  }
  isHex(ch: number): boolean {
    return (
      this.isDigit(ch) ||
      (ch >= CharacterCodes.a && ch <= CharacterCodes.f) ||
      (ch >= CharacterCodes.A && ch <= CharacterCodes.F)
    );
  }

  constructor(script: string) {
    this.script = script;
  }

  read() {
    if (this.pos < this.script.length) {
      return this.script.charCodeAt(this.pos++);
    } else {
      throw LexErrorType.EOF;
    }
  }

  getCh() {
    return this.script.substring(this.pos - 1, this.pos);
  }

  tokenize(): Token[] {
    let ch;
    try {
      while ((ch = this.read())) {
        try {
          switch (this.state) {
            case DFAState.Initial:
            case DFAState.OpenBracket:
            case DFAState.CloseBracket:
            case DFAState.OpenBraces:
            case DFAState.CloseBraces:
            case DFAState.Colon:
            case DFAState.Comma:
            case DFAState.StringEnd:
              this.initState(ch);
              break;
            case DFAState.NumberMinus:
              if (ch === CharacterCodes._0) {
                this.state = DFAState.NumberZeroStart;
                this.token.value += this.getCh();
              } else if (this.isDigit(ch)) {
                this.state = DFAState.NumberNormal;
                this.token.value += this.getCh();
              } else {
                throw new ErrorObject(LexErrorType.InvalidNumber);
              }
              break;
            case DFAState.NumberZeroStart:
              if (ch === CharacterCodes.dot) {
                this.state = DFAState.NumberAfterDot;
                this.token.value += this.getCh();
              } else {
                this.initState(ch);
              }
              break;
            case DFAState.NumberNormal:
              if (this.isDigit(ch)) {
                this.token.value += this.getCh();
              } else if (ch === CharacterCodes.dot) {
                this.state = DFAState.NumberDot;
                this.token.value += this.getCh();
              } else if (ch === CharacterCodes.e || ch === CharacterCodes.E) {
                this.state = DFAState.NumberE;
                this.token.value += this.getCh();
              } else {
                this.initState(ch);
              }
              break;
            case DFAState.NumberDot:
              if (this.isDigit(ch)) {
                this.token.value += this.getCh();
                this.state = DFAState.NumberAfterDot;
              } else {
                throw new ErrorObject(LexErrorType.InvalidNumber);
              }
              break;
            case DFAState.NumberE:
              if (this.isDigit(ch)) {
                this.token.value += this.getCh();
                this.state = DFAState.NumberAfterE;
              } else if (
                ch === CharacterCodes.plus ||
                ch === CharacterCodes.minus
              ) {
                this.state = DFAState.NumberESign;
              } else {
                throw new ErrorObject(LexErrorType.InvalidNumber);
              }
              break;
            case DFAState.NumberESign:
              if (this.isDigit(ch)) {
                this.token.value += this.getCh();
                this.state = DFAState.NumberAfterE;
              } else {
                throw new ErrorObject(LexErrorType.InvalidNumber);
              }
              break;
            case DFAState.NumberAfterDot:
              if (ch === CharacterCodes.e || ch === CharacterCodes.E) {
                this.state = DFAState.NumberE;
                this.token.value += this.getCh();
              } else if (this.isDigit(ch)) {
                this.token.value += this.getCh();
              } else {
                this.initState(ch);
              }
              break;
            case DFAState.NumberAfterE:
              if (this.isDigit(ch)) {
                this.token.value += this.getCh();
              } else {
                this.initState(ch);
              }
              break;
            case DFAState.StringNormal:
              if (ch >= 0 && ch <= 0x1f) {
                throw new ErrorObject(LexErrorType.InvalidStringCharacter);
              } else if (ch === CharacterCodes.doubleQuote) {
                this.token.value += this.getCh();
                this.state = DFAState.StringEnd;
              } else if (ch === CharacterCodes.backslash) {
                this.token.value += this.getCh();
                this.state = DFAState.StringSlash;
              } else {
                this.token.value += this.getCh();
              }
              break;
            case DFAState.StringSlash:
              if (
                [
                  CharacterCodes.n,
                  CharacterCodes.t,
                  CharacterCodes.f,
                  CharacterCodes.b,
                  CharacterCodes.r,
                  CharacterCodes.doubleQuote,
                  CharacterCodes.slash,
                  CharacterCodes.backslash,
                ].includes(ch)
              ) {
                this.token.value += this.getCh();
                this.state = DFAState.StringNormal;
              } else if (ch === CharacterCodes.u) {
                this.token.value += this.getCh();
                this.hexTemp = "";
                this.state = DFAState.StringSlashU;
              } else {
                throw new ErrorObject(LexErrorType.InvalidStringCharacter);
              }
              break;
            case DFAState.StringSlashU:
              const character = this.getCh();
              if (this.isHex(ch)) {
                this.token.value += character;
                this.hexTemp += character;
                if (this.hexTemp.length === 4) {
                  this.state = DFAState.StringNormal;
                }
              } else {
                throw new ErrorObject(LexErrorType.InvalidStringCharacter);
              }
              break;
            case DFAState.NullType:
              {
                let character = this.getCh();
                if (nullSteps.includes(this.token.value + character)) {
                  this.token.value += character;
                } else {
                  if (this.token.value === "null") {
                    this.initState(ch);
                  } else {
                    throw new ErrorObject(LexErrorType.InvalidKeyword);
                  }
                }
              }
              break;
            case DFAState.TrueType:
              {
                let character = this.getCh();
                if (trueSteps.includes(this.token.value + character)) {
                  this.token.value += character;
                } else {
                  if (this.token.value === "true") {
                    this.initState(ch);
                  } else {
                    throw new ErrorObject(LexErrorType.InvalidKeyword);
                  }
                }
              }
              break;
            case DFAState.FalseType:
              {
                let character = this.getCh();
                if (falseSteps.includes(this.token.value + character)) {
                  this.token.value += character;
                } else {
                  if (this.token.value === "false") {
                    this.initState(ch);
                  } else {
                    throw new ErrorObject(LexErrorType.InvalidKeyword);
                  }
                }
              }
              break;
          }
        } catch (err) {
          if (err instanceof ErrorObject) {
            this.handleError(err);
            break;
          }
        }
      }
    } catch (err) {
      switch (err) {
        case LexErrorType.EOF:
          console.log("解析完毕");
          break;
      }
    }
    this.tokenList.push(this.token);
    return this.tokenList;
  }

  initState(ch: any) {
    if (this.token.value) {
      this.tokenList.push(this.token);
    }
    this.token = new Token();
    if (ch === CharacterCodes.doubleQuote) {
      this.state = DFAState.StringNormal;
      this.token.type = TokenType.StringType;
      this.token.value += this.getCh();
    } else if (ch === CharacterCodes.minus) {
      this.state = DFAState.NumberMinus;
      this.token.type = TokenType.NumberType;
      this.token.value += this.getCh();
    } else if (ch === CharacterCodes._0) {
      this.state = DFAState.NumberZeroStart;
      this.token.type = TokenType.NumberType;
      this.token.value += this.getCh();
    } else if (this.isDigit(ch)) {
      this.state = DFAState.NumberNormal;
      this.token.type = TokenType.NumberType;
      this.token.value += this.getCh();
    } else if (ch === CharacterCodes.openBracket) {
      this.state = DFAState.OpenBracket;
      this.token.type = TokenType.OpenBracket;
      this.token.value += this.getCh();
    } else if (ch === CharacterCodes.closeBracket) {
      this.state = DFAState.CloseBracket;
      this.token.type = TokenType.CloseBracket;
      this.token.value += this.getCh();
    } else if (ch === CharacterCodes.openBrace) {
      this.state = DFAState.OpenBraces;
      this.token.type = TokenType.OpenBraces;
      this.token.value += this.getCh();
    } else if (ch === CharacterCodes.closeBrace) {
      this.state = DFAState.CloseBraces;
      this.token.type = TokenType.CloseBraces;
      this.token.value += this.getCh();
    } else if (ch === CharacterCodes.colon) {
      this.state = DFAState.Colon;
      this.token.type = TokenType.Colon;
      this.token.value += this.getCh();
    } else if (ch === CharacterCodes.comma) {
      this.state = DFAState.Comma;
      this.token.type = TokenType.Comma;
      this.token.value += this.getCh();
    } else if (ch === CharacterCodes.n) {
      this.state = DFAState.NullType;
      this.token.type = TokenType.NullType;
      this.token.value += this.getCh();
    } else if (ch === CharacterCodes.t) {
      this.state = DFAState.TrueType;
      this.token.type = TokenType.TrueType;
      this.token.value += this.getCh();
    } else if (ch === CharacterCodes.f) {
      this.state = DFAState.FalseType;
      this.token.type = TokenType.FalseType;
      this.token.value += this.getCh();
    } else if (wschar.includes(ch)) {
      // 跳过空白字符
    } else {
      throw new ErrorObject(LexErrorType.UnexeceptedCharacter);
    }
  }

  handleError(err: ErrorObject) {
    console.log(`在位置${this.pos - 1}遇到错误`);
    console.log(`错误字符 ${this.getCh()}`);

    switch (err.type) {
      case LexErrorType.InvalidKeyword:
        console.log("非法关键字");
      case LexErrorType.UnexeceptedCharacter:
        console.log("不被接受的字符: " + this.getCh());
    }
  }
}

parser

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
import { Lexer } from "./lexer";
import { Token, TokenType } from "./token";

enum AstNodeType {
  JsonObject,
  JsonObjectProperty,
  JsonArray,
  JsonString,
  JsonNumber,
  JsonTrueType,
  JsonFalseType,
  JsonNullType,
}

enum ParseErrorType {
  Error,
}

class ParseErrorObject {
  type!: ParseErrorType;
  constructor(type: ParseErrorType) {
    this.type = type;
  }
}

class AstNode {
  type: AstNodeType | null = null;
  children: AstNode[] = [];
  value: any = undefined;
  parent: AstNode | null = null;
}

class Parser {
  tokenList: Token[];
  pos = 0;
  peek() {
    return this.tokenList[this.pos];
  }
  read() {
    return this.tokenList[this.pos++];
  }

  constructor(tokenList: any[]) {
    this.tokenList = tokenList;
  }

  parseProgram() {
    try {
      let parseResult = this.parseValue() as AstNode;
      return parseResult.value;
    } catch (err) {
      console.log("错误位置", this.pos, this.read());
    }
  }

  parseValue() {
    let parseResult;
    parseResult = this.parseNumber();
    if (parseResult !== false) {
      return parseResult;
    }
    parseResult = this.parseString();
    if (parseResult !== false) {
      return parseResult;
    }
    parseResult = this.parseTrueType();
    if (parseResult !== false) {
      return parseResult;
    }
    parseResult = this.parseFalseType();
    if (parseResult !== false) {
      return parseResult;
    }
    parseResult = this.parseNullType();
    if (parseResult !== false) {
      return parseResult;
    }
    parseResult = this.parseArray();
    if (parseResult !== false) {
      return parseResult;
    }
    parseResult = this.parseObject();
    if (parseResult !== false) {
      return parseResult;
    }
    throw new ParseErrorObject(ParseErrorType.Error);
  }

  parseObject() {
    const astNode = new AstNode();
    astNode.type = AstNodeType.JsonObject;
    astNode.value = {};
    if (this.peek().type !== TokenType.OpenBraces) {
      return false;
    }
    this.read();
    while (true) {
      if (this.peek().type === TokenType.CloseBraces) {
        this.read();
        break;
      }
      const properyAstNode = this.parseObjectProperty() as AstNode;
      properyAstNode.parent = astNode;
      astNode.children.push(properyAstNode);
      astNode.value[properyAstNode.value[0]] = properyAstNode.value[1];
      if (this.peek().type === TokenType.Comma) {
        this.read();
      } else {
        if (this.peek().type === TokenType.CloseBraces) {
          this.read();
          break;
        } else {
          throw new ParseErrorObject(ParseErrorType.Error);
        }
      }
    }
    return astNode;
  }

  parseArray() {
    const astNode = new AstNode();
    astNode.type = AstNodeType.JsonArray;
    astNode.value = [];
    if (this.peek().type !== TokenType.OpenBracket) {
      return false;
    }
    this.read();

    while (true) {
      if (this.peek().type === TokenType.CloseBracket) {
        this.read();
        break;
      }
      const valueAstNode = this.parseValue() as AstNode;
      valueAstNode.parent = astNode;
      astNode.children.push(valueAstNode);
      astNode.value.push(valueAstNode.value);
      if (this.peek().type === TokenType.Comma) {
        this.read();
      } else {
        if (this.peek().type === TokenType.CloseBracket) {
          this.read();
          break;
        } else {
          throw new ParseErrorObject(ParseErrorType.Error);
        }
      }
    }
    return astNode;
  }

  parseObjectProperty() {
    if (this.peek().type !== TokenType.StringType) {
      return false;
    }
    const propertyNameToken = this.read();
    if (this.peek().type !== TokenType.Colon) {
      throw new ParseErrorObject(ParseErrorType.Error);
    }
    this.read();
    const valueAstNode = this.parseValue() as AstNode;
    const astNode = new AstNode();
    astNode.type = AstNodeType.JsonObjectProperty;
    valueAstNode.parent = astNode;
    astNode.children.push(valueAstNode);
    astNode.value = [JSON.parse(propertyNameToken.value), valueAstNode.value];
    return astNode;
  }

  parseNumber() {
    if (this.peek().type !== TokenType.NumberType) {
      return false;
    }
    const astNode = new AstNode();
    astNode.type = AstNodeType.JsonNumber;
    const token = this.read();
    astNode.value = JSON.parse(token.value);
    return astNode;
  }

  parseString() {
    if (this.peek().type !== TokenType.StringType) {
      return false;
    }
    const astNode = new AstNode();
    astNode.type = AstNodeType.JsonString;
    const token = this.read();
    astNode.value = JSON.parse(token.value);
    return astNode;
  }

  parseTrueType() {
    if (this.peek().type !== TokenType.TrueType) {
      return false;
    }
    const astNode = new AstNode();
    astNode.type = AstNodeType.JsonTrueType;
    const token = this.read();
    astNode.value = JSON.parse(token.value);
    return astNode;
  }

  parseFalseType() {
    if (this.peek().type !== TokenType.FalseType) {
      return false;
    }
    const astNode = new AstNode();
    astNode.type = AstNodeType.JsonFalseType;
    const token = this.read();
    astNode.value = JSON.parse(token.value);
    return astNode;
  }

  parseNullType() {
    if (this.peek().type !== TokenType.NullType) {
      return false;
    }
    const astNode = new AstNode();
    astNode.type = AstNodeType.JsonNullType;
    const token = this.read();
    astNode.value = JSON.parse(token.value);
    return astNode;
  }
}

const lexer = new Lexer(
  `{"name":"\\n\\uaaaatest project",\n\n"number":1.5E5,  "array":[1,2,true],      "description":"test project","private":true,"workspaces":{"shell":"shell"}}`
);

const tokenList = lexer.tokenize();

tokenList.forEach((token, index) => {
  console.log(
    index,
    token.value.padEnd(20, " "),
    TokenType[token.type as TokenType]
  );
});

const parser = new Parser(tokenList);

console.log(parser.parseProgram());