77 Commits

Author SHA1 Message Date
Huáng Jùnliàng
a8fea4037d
Faster identifier tokenizing (#13262)
* add benchmark

* perf: faster identifier tokenizing

- Mover iterator identifier parsing to the Flow plugin
- If the character is an identifier start, pass it to readWord1
2021-05-06 18:47:19 -04:00
Huáng Jùnliàng
a387973821
Refactor private name tokenizing (#13256)
* add benchmark

* refactor: create tt.privateName token for private names

* add backward compat privateName = hash + name to Babel 7

* perf: get private name SV from token value

* chore: tweak benchmark file

* chore: update test fixtures

* convert tt.privateName to PrivateIdentifier

* perf: avoid most isPrivateName call

* Update packages/babel-parser/src/parser/expression.js

Co-authored-by: Justin Ridgewell <justin@ridgewell.name>

* perf: use inlinable codePointAtPos

* make prettier happy

Co-authored-by: Justin Ridgewell <justin@ridgewell.name>
2021-05-06 09:46:09 -04:00
Huáng Jùnliàng
278193b6f7
fix: update chunkStart on missing unicode escape (#13261) 2021-05-06 08:31:11 -04:00
Sosuke Suzuki
0ee98139a6 Introduce parser error codes (#13033) 2021-04-28 18:21:31 +02:00
Huáng Jùnliàng
a0c1a9a9e4
Disallow non octal decimal escape before use strict (#12366) 2020-12-15 12:24:32 +01:00
Huáng Jùnliàng
ea2892fefc
add declare to class properties type annotations (#12257)
* add declare to class properties type annotations

* chore: use preset-flow
2020-10-27 10:05:01 -04:00
iamfotx
59d97d9bca
[parser] Better error message for missing number exponent (#12072)
* fix(babel-parser) better error message for missing number exponent after 'e'

* fix(babel-parser) requested changes

* msg

Co-authored-by: Brian Ng <bng412@gmail.com>
2020-10-14 21:17:38 +02:00
Huáng Jùnliàng
df9ee2c7cd
Simplify tokenizer update context (#11944)
* replace lineBreak.test by hasPrecedingLingBreak

* refactor: simplify updateContext
2020-08-11 13:43:03 -04:00
Brian Ng
c0f6f0394d
Support ConditionalExpressions in dry-error-messages rule (#11917)
* Support ConditionalExpressions in dry-error-messages rule

* tests
2020-08-05 16:28:35 -04:00
Huáng Jùnliàng
a1eabb84ea
rescan gt/lt token after TsAsExpression is parsed (#11912)
* refactor: move inType checks to flow plugin

* polish: replace hardcoded char codes

* fix: rescan greater/less token after asExpression is parsed
2020-08-04 17:00:21 -04:00
Huáng Jùnliàng
0e985fb287 feat: enable numericSeparator parsing support (#11863) 2020-07-29 18:32:12 -04:00
Huáng Jùnliàng
059e9124ff
Add decimal parsing support (#11640)
* docs: add DecimalLiteral to AST spec

* add decimal support

* fix: throw invalid decimal on start

* add DecimalLiteral type definitions

* update parser typings

* add generator support

* add syntax-decimal plugin

* Add syntax-decimal to babel-standalone

* add syntax-decimal to missing plugin helpers

* fix incorrect test macro
2020-07-29 16:43:15 -04:00
Huáng Jùnliàng
2bf38fb914
fix: disallow \8, \9 in strict mode string (#11852) 2020-07-21 09:44:56 -04:00
Huáng Jùnliàng
3680f019d7
fix: allow 09.1_1 and 09e1_1 in sloppy mode (#11854)
* fix: allow 09.1_1 and 09e1_1 in sloppy mode

* polish: avoid extra input source scanning

* chore: move comment [skip ci]
2020-07-21 09:44:28 -04:00
Kai Cataldo
75c2300c28
Add @babel/eslint-plugin-development-internal (#11376)
* Add @babel/eslint-plugin-internal

* Add dry-error-messages rule

* Address feedback

* Enable new rule

* fix author field

* Fix errors

* Add readme

* Add example configuration

* Handle directories

* run make bootstrap

* More updates!

* Fix errors

* Update tests

* Fix CI race condition
2020-06-22 19:43:29 -04:00
Huáng Jùnliàng
71d3527ef5
Properly parse export default from when exportDefaultFrom is not enabled (#11676)
Co-authored-by: Nicolò Ribaudo <nicolo.ribaudo@gmail.com>
2020-06-05 09:08:21 -05:00
Huáng Jùnliàng
b5c4a46a69
refactor: split locationParser into ParserErrors and error message (#11653) 2020-05-30 15:05:42 -04:00
Nicolò Ribaudo
31b361b736
Use ?. where it represents the intended semantics (#11512) 2020-05-09 23:31:50 +02:00
Nicolò Ribaudo
90a9103e55
Update Flow to 0.123.0 (#11500) 2020-04-30 15:26:03 +02:00
Huáng Jùnliàng
2e4f18ac92
Add some parser missing plugins errors (#11478)
* fix: do not throw invalid hash in tokenizer

* refactor: bigint has been enabled by default

* polish: add numeric separator missing plugin error

* fix: forward expectPlugin declaration
2020-04-25 10:26:39 +02:00
Rick Button
3ce7c2e394
Added support for record and tuple syntax. (#10865)
* Added support for record and tuple syntax.

This commit adds support for both the hash #{} and bar {||}
syntaxes to babel-parser, as well as adds the supporting
babel-plugin-syntax-record-and-tuple plugin to enable support
for the syntax. Does not include any transform for records and
tuples.

* typo

* added check to ensure recordAndTuple in babel-parser

* switched to syntaxType option instead of explicit entries for each record and tuple type

* switched to using recordAndTupleSyntaxType for generator options instead of adding to node

* added tests for generator option recordAndTupleSyntaxType

* added test for record and tuple bar syntax with flow typings

* added tests for invalid/missing recordAndTuple syntaxType parser option

* fixed flowcheck errors

* fix merge with class privates in tokenizer

* Update packages/babel-parser/src/types.js

Co-Authored-By: Nicolò Ribaudo <nicolo.ribaudo@gmail.com>

* improved recordAndTuple generator error message, added tests for invalid,missing options

* updated error messages for invalid parser syntaxType option

* updated error message

* added better error messages for when the recordAndTuple syntaxType is doesn't match the syntax used

* updated record and tuple support to use new error message templates

* added recordAndTuple to missing plugin helpers

* Fix linting

Co-authored-by: Nicolò Ribaudo <nicolo.ribaudo@gmail.com>
2020-03-16 23:57:44 +01:00
Nicolò Ribaudo
2bce1e5e20
Parse BigInts by default (#11117) 2020-03-16 22:59:45 +01:00
Kai Cataldo
2057d2b159
fix: non-directive "use strict" should not enable parsing in strict mode (#11188)
* fix: non-directive "use strict" should not enable parsing in strict mode

* Remove dead code

* Add stricter type annotations

* set oldStrict explicitly to null

* Fix error

* label callback argument

* update comment

* Address feedback

* Remove this.state.octalPosition

* Add additional tests

* Revert "Remove this.state.octalPosition"

This reverts commit bcc78c9530f8c840f85e86053b75efce662f34d1.

* Remove containsOctal

* Report multiple octal literals in single token

* Fix comments

* remove Array.prototype.flat()
2020-03-05 17:34:27 -05:00
Huáng Jùnliàng
21c9141872
Refactor: add parser message template (#11192)
* refactor: add raiseWithData method

* refactor: error message template

* fix missing plugin error structure

* fix flow errors

* refactor: use error message template in eslint plugin

* refacotr: use error message template in flow plugin

* refactor: use error message template in typescript plugin

* refactor: use error message template in jsx plugin

* address review comments

* Update packages/babel-parser/src/parser/location.js

Co-Authored-By: Nicolò Ribaudo <nicolo.ribaudo@gmail.com>

Co-authored-by: Nicolò Ribaudo <nicolo.ribaudo@gmail.com>
2020-03-03 14:06:05 -05:00
vages
2603c2e227
fix(babel-parser): chain off optionally chained keys named cla… (#11198)
* fix(babel-parser): chain off optionally chained keys named class and function

Fixes #11197

* Extract override as variable to satisfy typechecker

Flow did not trust that curContext.override would be defined
2020-03-03 18:27:41 +01:00
Nicolò Ribaudo
8ab27c8ffe Wrap type-only class fields in flow comments (#11096)
* Wrap type-only class fields in flow comments

* Fix
2020-02-09 11:45:42 +01:00
Huáng Jùnliàng
0b3dea8f54
Turn on no-fallthrough rule (#11093)
* chore: turn on no-fallthrough

* chore: fix no-fallthrough errors
2020-02-07 22:05:19 +01:00
Chunpeng Huo
9bc04baeb5 fix(babel-parser): avoid state.clone() to clone the whole token store (#11029)
* fix(babel-parser): avoid state.clone() to clone the whole token store

Fixed the performance issue on large input when turned on option {tokens: true} and typescript plugin which uses quite a few state.clone().

* test(babel-parser): turn on 2 typescript tests with tokens:true

The output.json is generated by old master to make sure no regression.

* fix(babel-parser): avoid duplicated tokens trapped by mainly typescript/flow plugins

* test(babel-parser): update output.json to latest master result

* chore(babel-parser): improve performance by storing tokensLength in state
2020-01-19 22:38:33 -05:00
Nicolò Ribaudo
771c730fda
[parser] Disallow duplicate and undeclared private names (#10456)
* [parser] Add private names tracking to Scope

- Disallow duplicate private names
- Disallow undeclared private names

* Update tests

* Test all possible duplications

* Test undeclared private names

* Better error message for top-level private names

* Fix flow

* Update test262 whitelist

* Update fixtures

* Update flow whitelist

* Remove old output.json

* Move ClassScopeHandler to a separate class

* Make the code readable
2020-01-10 02:22:05 +01:00
Huáng Jùnliàng
467667af88 When reading a new string, U+2028/2029 should correctly set th… (#10944)
* test: add test on template

* fix: update lineStart when string contains U+2028/U+2029
2020-01-03 23:07:54 +01:00
Huáng Jùnliàng
2f3f77926d
refactor: remove unused invalidTemplateEscapePosition tokenizer state (#10935)
* refactor: remove unused invalidTemplateEscapePosition tokenizer state

* fix flow error
2019-12-30 16:11:56 -05:00
Nicolò Ribaudo
87feda7c2a
@babel/parser error recovery (#10363)
* Add error recovery support to @babel/parser

* Update @babel/parser tests to always recover from errors

* Update this.raise usage in @babel/parser:

- expression.js
- lval.js
- statement.js
- estree.js
- flow.js
- jsx/index.js
- tokenizer/index.js

* Update @babel/parser fixtures with recovered errors

* Fix tests out of @babel/parser

* Do not use try/catch for control flow

* Update invalid fixtures

* Do not report invalid lhs in toAssignable

* Do not validate function id multiple times

* Dedupe reserved await errors

* Remove duplicate errors about strict reserved bindings

* Remove duplicated error about yield/await inside params

* Don't error twice for methods in object patterns

* Don't report invalid super() twice

* Remove dup error about reserved param for expr arrows

* Remove double escapes in migrated tests

* Dedupe errors about invalid escapes in identifiers

* Remove duplicated error about decorated constructor

* Remove duplicated error about spread in flow class

* Don't throw for invalid super usage

* Don't fail for object decorators with stage 2

* Fix flow inexact type errors

* Fix flow

* Fix errors about escapes in keywords (ref: #10455)

* Update after rebase

* Fix todo

* Remove duplicated error when using += for defaults

* Remove unnecessary throw

* Nit: use ??
2019-11-05 10:15:00 +01:00
piotr
5800fc97b3 [parser] Exception to 8 and 9 in tagged template (#10567) 2019-10-17 00:01:22 +02:00
Huáng Jùnliàng
0856618ed5 perf: replace lookahead by lookaheadCharCode (#10371)
* perf: replace lookahead by lookaheadCharCode

* fix: flow ignore

* refactor: add nextTokenStart method

* refactor: duplicated isNewLine code

* refactor: remove lookahead usage from babylon core
2019-10-08 19:09:05 +02:00
Gonzalo Rodríguez
490798a26d [parser] Disallow numeric separators in legacy octal like inte… (#10493)
* Disallow numeric separators in lols

* Update whitelist

* Rename test

* Better syntax when checking
2019-09-25 01:32:50 +02:00
Gonzalo Rodríguez
69d00dc5b0 [parser] Invalid NonOctal Decimal (#10467)
* Add test

* Add nonOctalDecimal verification

* Update regex and code style

* Refactor non octal detection

* Add numeric separator test

* Disallow numeric separators in non octals

* Update whitelist

* Better test naming

* Disallow numeric separators in non oct for all numbers

* Specific error above the general error

* Add test for invalid leading zero in num separator

* Add start position to error
2019-09-23 23:07:52 +02:00
Ivan Medina
f339d2d034 [parser] Disallow numeric separator in unicode scape sequences (#10468)
* [parser] Disallow numeric separator in unicode scape sequences (#10460)

* raise error only when numeric separator plugin is set

* Adds argument for checking numeric separator

* Fix condition for readability

* Add test for hex escape sequence and rephrase error message

* Remove exposure for allowNumSeparator in readHexChar method
2019-09-23 22:47:18 +02:00
Nicolò Ribaudo
cb881e6b3f
[parser] Report escapes in kws only if they won't be used as i… (#10455)
We can do it by tokenizing escaped keywords as normal identifiers, so that they won't
be accidentally used as real keywords.
2019-09-18 00:13:51 +02:00
Huáng Jùnliàng
4395c22f3a Miscellaneous perf tweak (#10421)
* refactor: early return on hot path

* refactor: read next2 only when next is dot

* perf: mini refactor
2019-09-10 19:47:52 +02:00
Daniel Tschinder
4da7a01aa9
Avoid unnecessary work during lookahead (#9982)
* Do not call pushComment when doing lookahead

* Do no updateContext when doing lookahead

* Do not clone contexts anymore
2019-05-15 17:02:12 -07:00
Daniel Tschinder
c7587c016e
Don't accept '\08' or '\09' in strict mode (#9769) 2019-04-01 19:41:50 -07:00
Daniel Tschinder
aaefc83a6c
Allow HTML comments on first line (#9760) 2019-03-25 14:22:57 -07:00
Daniel Tschinder
cf4bd8bb8d
Remove input and length from state (#9646) 2019-03-11 00:42:42 -07:00
Daniel Tschinder
d8a5329834
Reorganize token types and use a map for them (#9645) 2019-03-06 14:30:04 -08:00
Daniel Tschinder
828169e611
Fix line continuation with Unicode line terminators (#9403) 2019-01-28 13:35:35 -08:00
Daniel Tschinder
46ba5940c2
Make yield a contextual keyword (#9400) 2019-01-23 14:33:23 -08:00
Daniel Tschinder
178f2d7949 Make let a contextual keyword 2019-01-22 13:12:02 -08:00
Daniel Tschinder
4e5e319d5b perf: remove double check for keywords in readWord
Instead of calling isKeyword we simple check directly if the keyword token is available
2019-01-17 16:16:31 -08:00
Daniel Tschinder
c2e41588aa perf: run flow code in flow plugin 2019-01-17 16:15:05 -08:00
Daniel Tschinder
8df0500f55 perf: Simplify reading of tokens.
This avoids checking isIdentifierStart for every single token
2019-01-17 16:15:01 -08:00