Skip to content

fix(track-memory-allocations): exclude arena chunks from Sys allocs#24292

Merged
graphite-app[bot] merged 1 commit into
mainfrom
fix/allocs-exclude-arena-chunks
Jul 8, 2026
Merged

fix(track-memory-allocations): exclude arena chunks from Sys allocs#24292
graphite-app[bot] merged 1 commit into
mainfrom
fix/allocs-exclude-arena-chunks

Conversation

@Dunqing

@Dunqing Dunqing commented Jul 8, 2026

Copy link
Copy Markdown
Member

Issue

The Allocations CI job can fail on PRs with a 1-allocation Sys allocs difference that local cargo allocs cannot reproduce. #22621 fixed one instance of this in the transformer stage. #24112 and #24017 hit the same failure again in the minifier stage: kitchen-sink.tsx measures 2309 Sys allocs on macOS/arm64 but 2310 on Linux/x64, so whichever platform regenerates the snapshot last breaks CI for the other.

Why

The tracker counts every call to the system allocator. That includes the chunks bump arenas request. Whether an arena needs one more chunk depends on the total number of bytes in the arena, and byte totals are not the same on every target. For example, hashbrown tables use 16-byte control groups on x86_64 but 8-byte groups on aarch64. When an arena's content size sits close to a chunk boundary, one target crosses it and the other does not.

In the observed failure, the minifier's mangler-phase semantic build creates a fresh Scoping. Its arena holds symbol names, resolved-reference vectors, and the per-scope bindings hash tables, which are slightly larger on x86_64. The traced Linux-only allocation happened under Scoping::add_resolved_reference -> ArenaVec::push -> Arena::new_chunk, when the push overflowed the current chunk and requested a new 720880-byte chunk. On aarch64 the same content fit in the existing chunk, so the count stayed one lower.

Every other allocation class in the measurement grows on element counts, which are identical across platforms. Logging every allocation size during the minifier stage on macOS/arm64, Linux/arm64, and Linux/x64 showed the three sequences match one-to-one except for arena chunk requests.

Any PR that shifts arena content near a chunk boundary can trip this again in any stage, so fixing one stage at a time does not close the class.

Fix

Count chunk allocations in oxc_allocator under the tracker-only track_allocations feature, and subtract them from the reported Sys allocs in the tracker. The counter is global rather than per-arena because short-lived arenas (like the one backing Scoping) are created and dropped inside the measured operation.

Arena usage is still measured by the Arena allocs / Arena reallocs columns; only the platform-sensitive chunk-request count is excluded from Sys allocs.

Parser and transformer rows are unchanged (the main arena is pre-warmed, and #22621 already removed transformer scoping growth). Semantic, minifier, and formatter rows drop by their per-stage chunk counts.

Validation

cargo allocs produces byte-identical snapshots on macOS/arm64, Linux/arm64 (Docker), and Linux/x64 (Docker), and is idempotent across reruns. cargo test -p oxc_allocator passes; clippy is clean with track_allocations both on and off, and the counter compiles out entirely without the feature.

AI usage: AI assistance was used to investigate, implement, and validate this change.

Dunqing commented Jul 8, 2026

Copy link
Copy Markdown
Member Author

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

  • 0-merge - adds this PR to the back of the merge queue
  • hotfix - for urgent changes, fast-track this PR to the front of the merge queue

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

@Dunqing
Dunqing marked this pull request as ready for review July 8, 2026 09:00
@Dunqing
Dunqing requested a review from overlookmotel as a code owner July 8, 2026 09:00
@github-actions github-actions Bot added the A-allocator Area - Allocator label Jul 8, 2026
@codspeed-hq

codspeed-hq Bot commented Jul 8, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 62 untouched benchmarks
⏩ 9 skipped benchmarks1


Comparing fix/allocs-exclude-arena-chunks (3d39cf4) with main (ec26af2)2

Open in CodSpeed

Footnotes

  1. 9 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on main (453d725) during the generation of this report, so ec26af2 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@camc314 camc314 added the 0-merge Merge with Graphite Merge Queue label Jul 8, 2026

camc314 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Merge activity

@camc314

camc314 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Merging - I think i'm seeing this on #24302

…24292)

## Issue

The `Allocations` CI job can fail on PRs with a 1-allocation `Sys allocs` difference that local `cargo allocs` cannot reproduce. #22621 fixed one instance of this in the transformer stage. #24112 and #24017 hit the same failure again in the minifier stage: `kitchen-sink.tsx` measures 2309 Sys allocs on macOS/arm64 but 2310 on Linux/x64, so whichever platform regenerates the snapshot last breaks CI for the other.

## Why

The tracker counts every call to the system allocator. That includes the chunks bump arenas request. Whether an arena needs one more chunk depends on the total number of bytes in the arena, and byte totals are not the same on every target. For example, hashbrown tables use 16-byte control groups on x86_64 but 8-byte groups on aarch64. When an arena's content size sits close to a chunk boundary, one target crosses it and the other does not.

In the observed failure, the minifier's mangler-phase semantic build creates a fresh `Scoping`. Its arena holds symbol names, resolved-reference vectors, and the per-scope `bindings` hash tables, which are slightly larger on x86_64. The traced Linux-only allocation happened under `Scoping::add_resolved_reference -> ArenaVec::push -> Arena::new_chunk`, when the push overflowed the current chunk and requested a new 720880-byte chunk. On aarch64 the same content fit in the existing chunk, so the count stayed one lower.

Every other allocation class in the measurement grows on element counts, which are identical across platforms. Logging every allocation size during the minifier stage on macOS/arm64, Linux/arm64, and Linux/x64 showed the three sequences match one-to-one except for arena chunk requests.

Any PR that shifts arena content near a chunk boundary can trip this again in any stage, so fixing one stage at a time does not close the class.

## Fix

Count chunk allocations in `oxc_allocator` under the tracker-only `track_allocations` feature, and subtract them from the reported Sys allocs in the tracker. The counter is global rather than per-arena because short-lived arenas (like the one backing `Scoping`) are created and dropped inside the measured operation.

Arena usage is still measured by the `Arena allocs` / `Arena reallocs` columns; only the platform-sensitive chunk-request count is excluded from `Sys allocs`.

Parser and transformer rows are unchanged (the main arena is pre-warmed, and #22621 already removed transformer scoping growth). Semantic, minifier, and formatter rows drop by their per-stage chunk counts.

## Validation

`cargo allocs` produces byte-identical snapshots on macOS/arm64, Linux/arm64 (Docker), and Linux/x64 (Docker), and is idempotent across reruns. `cargo test -p oxc_allocator` passes; clippy is clean with `track_allocations` both on and off, and the counter compiles out entirely without the feature.

AI usage: AI assistance was used to investigate, implement, and validate this change.
@graphite-app
graphite-app Bot force-pushed the fix/allocs-exclude-arena-chunks branch from 3d39cf4 to 460176a Compare July 8, 2026 14:23
@graphite-app
graphite-app Bot merged commit 460176a into main Jul 8, 2026
31 checks passed
@graphite-app graphite-app Bot removed the 0-merge Merge with Graphite Merge Queue label Jul 8, 2026
@graphite-app
graphite-app Bot deleted the fix/allocs-exclude-arena-chunks branch July 8, 2026 14:27
Boshen added a commit that referenced this pull request Jul 14, 2026
### 🚀 Features

- 616bfa2 minifier: Remove unreachable code after terminating statements
(#24441) (Dunqing)
- ddab89a data_structures: Add `likely` and `unlikely` functions
(#24368) (overlookmotel)
- a3a39f9 react_compiler: Implement enableEmitHookGuards codegen
(#24329) (Boshen)
- b79eef7 minifier: Apply De Morgan's law to negated comparison chains
in jump guards and loop tests (#24279) (Dunqing)
- 34ff7b4 minifier: Drop write-only property assignments to unused local
bindings by default (#24112) (Dunqing)
- 1b829d8 semantic: Record const enums in EnumData (#24268) (Dunqing)
- ba0944c semantic: Add `Scoping::set_symbol_span` (#24221) (camc314)

### 🐛 Bug Fixes

- 7d33363 minifier: Preserve guaranteed throws from class heritage
evaluation (#24349) (Dunqing)
- 058a62f semantic: Track ambient contexts in `SemanticBuilder` (#24327)
(camc314)
- 721eb0b transformer/decorator: Scope accessor class binding (#24330)
(camc314)
- 1ebdce3 semantic: Allow reserved keywords in ambient declaration types
(#24325) (camc314)
- 460176a track-memory-allocations: Exclude arena chunks from Sys allocs
(#24292) (Dunqing)
- af4922b transformer: Clear lowered namespace redeclarations (#24300)
(camc314)
- ffd2765 semantic: Mark declared computed `MethodDefinition`s as type
references (#24296) (camc314)
- f17514b isolated-declarations: Emit const readonly fields as types
(#24288) (camc314)
- 40f769d minifier: Make `__proto__` write tracking execution-order
independent (#24280) (Dunqing)
- 6371fed transformer: Remove stale enum member bindings (#24272)
(camc314)
- f05dfab transformer: Correct symbol flags for lowered namespaces
(#24271) (Dunqing)
- 84eeb55 transformer: Correct symbol flags for lowered enums (#24269)
(Dunqing)
- c3057da transformer: Preserve generated class binding spans (#24220)
(camc314)
- 8260096 transformer: Correct span for lowered namespace symbol
(#24222) (camc314)
- 42d00d3 semantic: Mark declared class heritage as type references
(#24237) (camc314)
- 588d997 semantic: Mark TS `PropertyDefinition`s computed fields as
type references (#24233) (camc314)
- 9b95632 semantic: Mark computed method keys in `TSMethodSignature`s as
type references (#24232) (camc314)

### ⚡ Performance

- 5b26643 transformer_plugins: Dispatch global defines by trailing name
(#23666) (Boshen)
- dce0f29 react_compiler: Replace all compiled functions in a single AST
walk (#24403) (Boshen)
- f85f0d8 ast: Delegate inherited enum variants in clone_in and estree
derives (#23555) (Boshen)
- 3ff0234 allocator: Remove `unwrap` from `ReplaceWith` (#24365)
(overlookmotel)
- ab22e80 transformer: Fix Rust 1.97 performance regression (#24354)
(camc314)
- b47585c parser: Use `ReplaceWith` instead of `TakeIn` (#24018)
(overlookmotel)
- b227a06 minifier: Use `ReplaceWith` instead of `TakeIn` (#24017)
(overlookmotel)

Co-authored-by: Boshen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-allocator Area - Allocator

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants