feat(parser): add ParseOptions::enable_ident_hashes#24491
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0562c01f2a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Merging this PR will not alter performance
Comparing Footnotes
|
|
"precompute_ident_hashes" is a misleading name. Should be something like "disable_ident_hashes". |
ParseOptions::precompute_ident_hashesParseOptions::enable_ident_hashes
Merge activity
|
Adds `ParseOptions::enable_ident_hashes` (default `true`) and `Ident::new_unhashed` in `oxc_str`, and turns it off in the parse-only paths of the `oxc-parser` napi crate and the JS/JSON formatters. ### Why `Ident`'s precomputed hash (#19143) is a parse-time investment that pays off in semantic analysis — but parse-only consumers (parse + serialize, formatting) pay it for nothing. Profiling against other native parsers showed identifier hashing at 3.5-4.6% of parse time. Measured on M-series (yuku-style native harness, 50 warmup + 300 runs, median): | fixture | default | option off | |---|---|---| | react.development.js | 0.156 ms | 0.152 ms (−2.3%) | | binder.ts | 0.383 ms | 0.370 ms (−3.5%) | | App.tsx | 1.140 ms | 1.111 ms (−2.5%) | | checker.ts | 7.862 ms | 7.499 ms (−4.6%) | | typescript.js | 25.07 ms | 24.16 ms (−3.6%) | ### Contract Unhashed `Ident`s store hash `0`; `Eq`/`Hash`/`ContentEq` include the stored hash, so unhashed and hashed `Ident`s of the same string do not compare equal. Semantic analysis (or anything relying on `Ident` hashing) must not run on an AST parsed with the option disabled. Default behavior is unchanged. Every `Ident` the parser creates funnels through the `ParserImpl::ident()` helper, so the option applies uniformly. This covers identifiers lexed directly (`parse_identifier_kind`, `parse_private_identifier`) as well as those reconstructed from a `&str`/`Str` — import/export specifier locals, JSX element/member names, and the TS `intrinsic` type name — which would otherwise always be hashed through `Into<Ident>`. On 64-bit platforms `Ident::new_unhashed` compiles to a no-op: an unhashed `Ident` has the same layout and bit representation as `&str`. ### Opt-outs - **napi `oxc-parser`** ties `enable_ident_hashes` to whether semantic errors are requested. Both parse entry points run `SemanticBuilder` only when `showSemanticErrors` is set, so the common parse-and-serialize path skips hashing while the semantic-errors path keeps it. Serialized output is unchanged — the hash is internal to `Ident` and never emitted. - **JS and JSON formatters** turn it off unconditionally: they never run semantic analysis and only read `Ident::as_str` / compare idents against string literals.
ba4f367 to
59da124
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ba4f36721c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fa44588653
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Adds `ParseOptions::enable_ident_hashes` (default `true`) and `Ident::new_unhashed` in `oxc_str`, and turns it off in the parse-only paths of the `oxc-parser` napi crate and the JS/JSON formatters. ### Why `Ident`'s precomputed hash (#19143) is a parse-time investment that pays off in semantic analysis — but parse-only consumers (parse + serialize, formatting) pay it for nothing. Profiling against other native parsers showed identifier hashing at 3.5-4.6% of parse time. Measured on M-series (yuku-style native harness, 50 warmup + 300 runs, median): | fixture | default | option off | |---|---|---| | react.development.js | 0.156 ms | 0.152 ms (−2.3%) | | binder.ts | 0.383 ms | 0.370 ms (−3.5%) | | App.tsx | 1.140 ms | 1.111 ms (−2.5%) | | checker.ts | 7.862 ms | 7.499 ms (−4.6%) | | typescript.js | 25.07 ms | 24.16 ms (−3.6%) | ### Contract Unhashed `Ident`s store hash `0`; `Eq`/`Hash`/`ContentEq` include the stored hash, so unhashed and hashed `Ident`s of the same string do not compare equal. Semantic analysis (or anything relying on `Ident` hashing) must not run on an AST parsed with the option disabled. Default behavior is unchanged. Every `Ident` the parser creates funnels through the `ParserImpl::ident()` helper, so the option applies uniformly. This covers identifiers lexed directly (`parse_identifier_kind`, `parse_private_identifier`) as well as those reconstructed from a `&str`/`Str` — import/export specifier locals, JSX element/member names, the TS `intrinsic` type name, TS import-type option keys (`with`/`assert`), and the `import x = this` recovery path — which would otherwise always be hashed through `Into<Ident>`. Exhaustively checked by parsing the test262/babel/typescript/misc/prettier corpora (96.5k files, 5.6M identifiers) in both option states and walking every `Ident` in the AST. On 64-bit platforms `Ident::new_unhashed` compiles to a no-op: an unhashed `Ident` has the same layout and bit representation as `&str`. ### Opt-outs - **napi `oxc-parser`** ties `enable_ident_hashes` to whether semantic errors are requested. Both parse entry points run `SemanticBuilder` only when `showSemanticErrors` is set, so the common parse-and-serialize path skips hashing while the semantic-errors path keeps it. Serialized output is unchanged — the hash is internal to `Ident` and never emitted. - **JS and JSON formatters** turn it off: they never run semantic analysis and only read `Ident::as_str` / compare idents against string literals. Exception: the JS formatter keeps hashes when built with the `detect_code_removal` feature, whose check runs `SemanticBuilder` on the formatter-parsed AST.
ad4706c to
3d22307
Compare
### 💥 BREAKING CHANGES - 54cc121 ast: [**BREAKING**] Split `MetaProperty` into `ImportMeta` and `NewTarget` (#24557) (camc314) ### 🚀 Features - 4c71560 parser: More friendly error for spread element in dynamic imports (#24705) (sapphi-red) - 7b045cd minfier: Drop last break from last switch case (#24673) (Armano) - 7d3c178 minifier: Remove unreachable recursive functions (#24125) (Dunqing) - 94f99b3 ast: Allow `NONE` to be passed to AST builder methods where `Option<ArenaVec>` is expected (#24629) (overlookmotel) - 77230c5 ast: Accept arrays for `ArenaVec` params of AST builder methods (#24621) (overlookmotel) - f08b152 allocator: Implement `FromIn` for array to `Vec` conversion (#24620) (overlookmotel) - 2338c13 track-memory-allocations: Track heap deallocs, alloc bytes, and peak growth (#24619) (Boshen) - 7aa4739 syntax,transformer: Move JSX entity decoder to `oxc_syntax` (#24617) (camc314) - 2b097c4 str: Export `Str` as `ArenaStr` (#24604) (overlookmotel) - 3acf4c1 minifier: Expand switch optimiation to remove empty cases (#24520) (Armano) - 129b759 parser: Improve diagnostics for unparenthesized LHS on exponential expr (#24569) (camc314) - 4d0c601 minifier: Fold arithmetic over undefined and null operands (#24485) (Dunqing) - 91541dd minifier: Drop empty switch statements (#24527) (Armano) - d05224d ast_visit: Generate VisitJs visitor that skips TypeScript type-space (#24499) (Boshen) - 3d22307 parser: Add `ParseOptions::enable_ident_hashes` (#24491) (Boshen) ### 🐛 Bug Fixes - 64c2241 minifier: Align class heritage removal with assumptions (#24533) (Dunqing) - 48b59f4 parser: Span ambient generator diagnostics (#24711) (camc314) - e750a82 ecmascript: Fix false negative for may_have_side_effects on dynamic property access (#24709) (sapphi-red) - f145d73 minifier: Guard reordered identifier reads (#24698) (Dunqing) - a2ef382 isolated-declarations: Reject `window.Symbol` as global symbol reference (#24689) (camc314) - b1bcf72 minifier: Invalidate facts for redeclared bindings (#24658) (Dunqing) - 921b834 minifier: Don't treat a conditionally-assigned var as write-once (#24650) (Dunqing) - 061af1f minifier: Avoid stale pure function summaries (#24636) (Dunqing) - 40c2f43 allocator: `Vec::from_array_in` do not allocate zero-length array (#24628) (overlookmotel) - 70994ae codegen: Preserve comments before expression operands (#24510) (Dunqing) - 7b4baff parser: Reject new import member access (#23459) (camc314) - 128b385 minifier: Clippy warning with no-debug-assertions (#24547) (camc314) - 8421feb parser: Use first `as` span for imported name (#24537) (leaysgur) - c517aa0 parser: Reject invalid accessor assertions (#24504) (camc314) ### ⚡ Performance - 884d9eb parser: Pre-size cover-grammar assignment target buffers (#24683) (Boshen) - d3f07a0 diagnostics: Box OxcDiagnosticInner to reduce binary size (#24665) (Boshen) - bcc9de0 parser: Defer diagnostic creation until parse exit (#24663) (Boshen) - c35d8ab allocator: Mark `ReplaceWith` panic path cold (#24515) (camc314) - ba65790 semantic, allocator: Branchless `clone_in` for semantic IDs (#24564) (overlookmotel) - 747feec parser: Build AST nodes with the AST builder instead of cloning (#24540) (Boshen) - ba35a0d react_compiler: Use IndexVec for dense id-keyed maps (#24549) (Boshen) - b685062 react_compiler: Keep small hot-path collections inline (#24514) (Marius Schulz) - a149e95 transformer: Outline rare expression exits (#24512) (camc314) - 7808a6e react_compiler: Make aliasing effects cheap to intern and clone (#24506) (Marius Schulz) - 3a36f2a react_compiler: Store AbstractValue reasons as a u16 bitmask (#24480) (Boshen) - 1c96753 react_compiler: Use FxHashMap for the lookup-only aliasing node map (#24490) (Boshen) Co-authored-by: Cameron <[email protected]>
# Oxlint ### 💥 BREAKING CHANGES - 54cc121 ast: [**BREAKING**] Split `MetaProperty` into `ImportMeta` and `NewTarget` (#24557) (camc314) ### 🚀 Features - 7b045cd minfier: Drop last break from last switch case (#24673) (Armano) - dd18383 linter/node: Implement no-top-level-await rule (#24634) (Connor Shea) - 16a65f2 linter/react: Implement function-component-definition rule (#24471) (Cole Ellison) - 7f1f585 linter: Reuse `jest/padding-around-test-blocks` for `vitest/padding-around-test-blocks` (#24519) (Mikhail Baev) - 99978a8 linter/import/consistent-type-specifier-style: Support `prefer-top-level-if-only-type-imports` option (#24502) (camc314) ### 🐛 Bug Fixes - 0184ad6 linter/unicorn/no-useless-undefined: Preserve valid parameter defaults (#24686) (camc314) - 8694167 linter/eslint/prefer-destructuring: Handle typed declarations (#24616) (camc314) - 477cf0f linter/eslint/no-throw-literal: Handle assigned errors (#24561) (Cole Ellison) - ac9200a linter: Detect React components from returned JSX (#24521) (camc314) - c0a6522 linter/eslint/no-useless-computed-key: Allow TS syntax in computed keys (#24524) (Cole Ellison) ### ⚡ Performance - 346eed1 linter/unicorn/prefer-event-target: Only run on `Class` and `NewExpression` nodes (#24685) (Mikhail Baev) - 7be5cf0 oxlint/lsp: Only invoke lint on code actions when document is not opened (#24676) (Sysix) - d3f07a0 diagnostics: Box OxcDiagnosticInner to reduce binary size (#24665) (Boshen) - 90ae040 linter/reporter/stylish: Compute diagnostic Info once per diagnostic (#24525) (connorshea) ### 📚 Documentation - e6f7174 linter/valid-expect: Fix correct example being identical to incorrect one (#24468) (mkan0141) # Oxfmt ### 💥 BREAKING CHANGES - 54cc121 ast: [**BREAKING**] Split `MetaProperty` into `ImportMeta` and `NewTarget` (#24557) (camc314) ### 🚀 Features - 3d22307 parser: Add `ParseOptions::enable_ident_hashes` (#24491) (Boshen) ### 🐛 Bug Fixes - 6fe866a oxfmt: Keep tailwind classes glued to template expr with `preserveWhitespace` (#24609) (leaysgur) - 33e32d8 formatter_css: Use `line_suffix` for EOL line comment (#24580) (leaysgur) - 5f76998 formatter_graphql: Keep same line comments pending across intervening tokens (#24579) (leaysgur) Co-authored-by: Boshen <[email protected]> Co-authored-by: Cameron <[email protected]>
Adds
ParseOptions::enable_ident_hashes(defaulttrue) andIdent::new_unhashedinoxc_str, and turns it off in the parse-only paths of theoxc-parsernapi crate and the JS/JSON formatters.Why
Ident's precomputed hash (#19143) is a parse-time investment that pays off in semantic analysis — but parse-only consumers (parse + serialize, formatting) pay it for nothing. Profiling against other native parsers showed identifier hashing at 3.5-4.6% of parse time.Measured on M-series (yuku-style native harness, 50 warmup + 300 runs, median):
Contract
Unhashed
Idents store hash0;Eq/Hash/ContentEqinclude the stored hash, so unhashed and hashedIdents of the same string do not compare equal. Semantic analysis (or anything relying onIdenthashing) must not run on an AST parsed with the option disabled. Default behavior is unchanged.Every
Identthe parser creates funnels through theParserImpl::ident()helper, so the option applies uniformly. This covers identifiers lexed directly (parse_identifier_kind,parse_private_identifier) as well as those reconstructed from a&str/Str— import/export specifier locals, JSX element/member names, the TSintrinsictype name, TS import-type option keys (with/assert), and theimport x = thisrecovery path — which would otherwise always be hashed throughInto<Ident>. Exhaustively checked by parsing the test262/babel/typescript/misc/prettier corpora (96.5k files, 5.6M identifiers) in both option states and walking everyIdentin the AST.On 64-bit platforms
Ident::new_unhashedcompiles to a no-op: an unhashedIdenthas the same layout and bit representation as&str.Opt-outs
oxc-parsertiesenable_ident_hashesto whether semantic errors are requested. Both parse entry points runSemanticBuilderonly whenshowSemanticErrorsis set, so the common parse-and-serialize path skips hashing while the semantic-errors path keeps it. Serialized output is unchanged — the hash is internal toIdentand never emitted.Ident::as_str/ compare idents against string literals. Exception: the JS formatter keeps hashes when built with thedetect_code_removalfeature, whose check runsSemanticBuilderon the formatter-parsed AST.