fix(track-memory-allocations): exclude arena chunks from Sys allocs#24292
Conversation
How to use the Graphite Merge QueueAdd either label to this PR to merge it via the merge queue:
You must have a Graphite account in order to use the merge queue. Sign up using this link. An organization admin has enabled the Graphite Merge Queue in this repository. Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue. This stack of pull requests is managed by Graphite. Learn more about stacking. |
Merging this PR will not alter performance
Comparing Footnotes
|
Merge activity
|
|
Merging - I think i'm seeing this on #24302 |
…24292) ## Issue The `Allocations` CI job can fail on PRs with a 1-allocation `Sys allocs` difference that local `cargo allocs` cannot reproduce. #22621 fixed one instance of this in the transformer stage. #24112 and #24017 hit the same failure again in the minifier stage: `kitchen-sink.tsx` measures 2309 Sys allocs on macOS/arm64 but 2310 on Linux/x64, so whichever platform regenerates the snapshot last breaks CI for the other. ## Why The tracker counts every call to the system allocator. That includes the chunks bump arenas request. Whether an arena needs one more chunk depends on the total number of bytes in the arena, and byte totals are not the same on every target. For example, hashbrown tables use 16-byte control groups on x86_64 but 8-byte groups on aarch64. When an arena's content size sits close to a chunk boundary, one target crosses it and the other does not. In the observed failure, the minifier's mangler-phase semantic build creates a fresh `Scoping`. Its arena holds symbol names, resolved-reference vectors, and the per-scope `bindings` hash tables, which are slightly larger on x86_64. The traced Linux-only allocation happened under `Scoping::add_resolved_reference -> ArenaVec::push -> Arena::new_chunk`, when the push overflowed the current chunk and requested a new 720880-byte chunk. On aarch64 the same content fit in the existing chunk, so the count stayed one lower. Every other allocation class in the measurement grows on element counts, which are identical across platforms. Logging every allocation size during the minifier stage on macOS/arm64, Linux/arm64, and Linux/x64 showed the three sequences match one-to-one except for arena chunk requests. Any PR that shifts arena content near a chunk boundary can trip this again in any stage, so fixing one stage at a time does not close the class. ## Fix Count chunk allocations in `oxc_allocator` under the tracker-only `track_allocations` feature, and subtract them from the reported Sys allocs in the tracker. The counter is global rather than per-arena because short-lived arenas (like the one backing `Scoping`) are created and dropped inside the measured operation. Arena usage is still measured by the `Arena allocs` / `Arena reallocs` columns; only the platform-sensitive chunk-request count is excluded from `Sys allocs`. Parser and transformer rows are unchanged (the main arena is pre-warmed, and #22621 already removed transformer scoping growth). Semantic, minifier, and formatter rows drop by their per-stage chunk counts. ## Validation `cargo allocs` produces byte-identical snapshots on macOS/arm64, Linux/arm64 (Docker), and Linux/x64 (Docker), and is idempotent across reruns. `cargo test -p oxc_allocator` passes; clippy is clean with `track_allocations` both on and off, and the counter compiles out entirely without the feature. AI usage: AI assistance was used to investigate, implement, and validate this change.
3d39cf4 to
460176a
Compare
### 🚀 Features - 616bfa2 minifier: Remove unreachable code after terminating statements (#24441) (Dunqing) - ddab89a data_structures: Add `likely` and `unlikely` functions (#24368) (overlookmotel) - a3a39f9 react_compiler: Implement enableEmitHookGuards codegen (#24329) (Boshen) - b79eef7 minifier: Apply De Morgan's law to negated comparison chains in jump guards and loop tests (#24279) (Dunqing) - 34ff7b4 minifier: Drop write-only property assignments to unused local bindings by default (#24112) (Dunqing) - 1b829d8 semantic: Record const enums in EnumData (#24268) (Dunqing) - ba0944c semantic: Add `Scoping::set_symbol_span` (#24221) (camc314) ### 🐛 Bug Fixes - 7d33363 minifier: Preserve guaranteed throws from class heritage evaluation (#24349) (Dunqing) - 058a62f semantic: Track ambient contexts in `SemanticBuilder` (#24327) (camc314) - 721eb0b transformer/decorator: Scope accessor class binding (#24330) (camc314) - 1ebdce3 semantic: Allow reserved keywords in ambient declaration types (#24325) (camc314) - 460176a track-memory-allocations: Exclude arena chunks from Sys allocs (#24292) (Dunqing) - af4922b transformer: Clear lowered namespace redeclarations (#24300) (camc314) - ffd2765 semantic: Mark declared computed `MethodDefinition`s as type references (#24296) (camc314) - f17514b isolated-declarations: Emit const readonly fields as types (#24288) (camc314) - 40f769d minifier: Make `__proto__` write tracking execution-order independent (#24280) (Dunqing) - 6371fed transformer: Remove stale enum member bindings (#24272) (camc314) - f05dfab transformer: Correct symbol flags for lowered namespaces (#24271) (Dunqing) - 84eeb55 transformer: Correct symbol flags for lowered enums (#24269) (Dunqing) - c3057da transformer: Preserve generated class binding spans (#24220) (camc314) - 8260096 transformer: Correct span for lowered namespace symbol (#24222) (camc314) - 42d00d3 semantic: Mark declared class heritage as type references (#24237) (camc314) - 588d997 semantic: Mark TS `PropertyDefinition`s computed fields as type references (#24233) (camc314) - 9b95632 semantic: Mark computed method keys in `TSMethodSignature`s as type references (#24232) (camc314) ### ⚡ Performance - 5b26643 transformer_plugins: Dispatch global defines by trailing name (#23666) (Boshen) - dce0f29 react_compiler: Replace all compiled functions in a single AST walk (#24403) (Boshen) - f85f0d8 ast: Delegate inherited enum variants in clone_in and estree derives (#23555) (Boshen) - 3ff0234 allocator: Remove `unwrap` from `ReplaceWith` (#24365) (overlookmotel) - ab22e80 transformer: Fix Rust 1.97 performance regression (#24354) (camc314) - b47585c parser: Use `ReplaceWith` instead of `TakeIn` (#24018) (overlookmotel) - b227a06 minifier: Use `ReplaceWith` instead of `TakeIn` (#24017) (overlookmotel) Co-authored-by: Boshen <[email protected]>

Issue
The
AllocationsCI job can fail on PRs with a 1-allocationSys allocsdifference that localcargo allocscannot reproduce. #22621 fixed one instance of this in the transformer stage. #24112 and #24017 hit the same failure again in the minifier stage:kitchen-sink.tsxmeasures 2309 Sys allocs on macOS/arm64 but 2310 on Linux/x64, so whichever platform regenerates the snapshot last breaks CI for the other.Why
The tracker counts every call to the system allocator. That includes the chunks bump arenas request. Whether an arena needs one more chunk depends on the total number of bytes in the arena, and byte totals are not the same on every target. For example, hashbrown tables use 16-byte control groups on x86_64 but 8-byte groups on aarch64. When an arena's content size sits close to a chunk boundary, one target crosses it and the other does not.
In the observed failure, the minifier's mangler-phase semantic build creates a fresh
Scoping. Its arena holds symbol names, resolved-reference vectors, and the per-scopebindingshash tables, which are slightly larger on x86_64. The traced Linux-only allocation happened underScoping::add_resolved_reference -> ArenaVec::push -> Arena::new_chunk, when the push overflowed the current chunk and requested a new 720880-byte chunk. On aarch64 the same content fit in the existing chunk, so the count stayed one lower.Every other allocation class in the measurement grows on element counts, which are identical across platforms. Logging every allocation size during the minifier stage on macOS/arm64, Linux/arm64, and Linux/x64 showed the three sequences match one-to-one except for arena chunk requests.
Any PR that shifts arena content near a chunk boundary can trip this again in any stage, so fixing one stage at a time does not close the class.
Fix
Count chunk allocations in
oxc_allocatorunder the tracker-onlytrack_allocationsfeature, and subtract them from the reported Sys allocs in the tracker. The counter is global rather than per-arena because short-lived arenas (like the one backingScoping) are created and dropped inside the measured operation.Arena usage is still measured by the
Arena allocs/Arena reallocscolumns; only the platform-sensitive chunk-request count is excluded fromSys allocs.Parser and transformer rows are unchanged (the main arena is pre-warmed, and #22621 already removed transformer scoping growth). Semantic, minifier, and formatter rows drop by their per-stage chunk counts.
Validation
cargo allocsproduces byte-identical snapshots on macOS/arm64, Linux/arm64 (Docker), and Linux/x64 (Docker), and is idempotent across reruns.cargo test -p oxc_allocatorpasses; clippy is clean withtrack_allocationsboth on and off, and the counter compiles out entirely without the feature.AI usage: AI assistance was used to investigate, implement, and validate this change.