Skip to content

feat(track-memory-allocations): add arena size#24553

Merged
graphite-app[bot] merged 1 commit into
mainfrom
allocs-snap-arena-bytes
Jul 15, 2026
Merged

feat(track-memory-allocations): add arena size#24553
graphite-app[bot] merged 1 commit into
mainfrom
allocs-snap-arena-bytes

Conversation

@Boshen

@Boshen Boshen commented Jul 15, 2026

Copy link
Copy Markdown
Member

First of three PRs splitting the byte-metrics work (followed by chunk-alloc-bytes and heap-alloc-bytes).

Adds an arena size value to each per-file block: the measured Allocator's used bytes (Allocator::used_bytes()) at the end of the stage. It is a point-in-time gauge read after the operation, not a diffed counter — the arena is not reset between stages, so e.g. the transformer's value includes the AST the parser built.

checker.ts
  file size: 2.92 MB
  sys allocs: 1818
  sys reallocs: 10
  arena allocs: 264366
  arena reallocs: 22860
  arena size: 12985208 (12.99 MB)

Structurally, this introduces the counter/gauge split (StageStats wraps the diffed AllocatorStats counters plus point-in-time gauges), and byte values print as the exact number plus a human-readable size.

Platform note: byte totals vary between platforms (type layouts differ, e.g. hashbrown tables are wider on x86_64 than aarch64) while counts are identical everywhere, so snapshots record byte values from CI's platform (Linux x64). Regenerating on another platform diffs byte-value lines only; take those lines from the CI job's output.

🤖 Generated with Claude Code

Boshen commented Jul 15, 2026

Copy link
Copy Markdown
Member Author

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

  • 0-merge - adds this PR to the back of the merge queue
  • hotfix - for urgent changes, fast-track this PR to the front of the merge queue

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

Boshen commented Jul 15, 2026

Copy link
Copy Markdown
Member Author

Merge activity

First of three PRs splitting the byte-metrics work (followed by chunk-alloc-bytes and heap-alloc-bytes).

Adds an `arena size` value to each per-file block: the measured `Allocator`'s used bytes (`Allocator::used_bytes()`) at the end of the stage. It is a point-in-time gauge read after the operation, not a diffed counter — the arena is not reset between stages, so e.g. the transformer's value includes the AST the parser built.

```
checker.ts
  file size: 2.92 MB
  sys allocs: 1818
  sys reallocs: 10
  arena allocs: 264366
  arena reallocs: 22860
  arena size: 12985208 (12.99 MB)
```

Structurally, this introduces the counter/gauge split (`StageStats` wraps the diffed `AllocatorStats` counters plus point-in-time gauges), and byte values print as the exact number plus a human-readable size.

**Platform note**: byte totals vary between platforms (type layouts differ, e.g. hashbrown tables are wider on x86_64 than aarch64) while counts are identical everywhere, so snapshots record byte values from CI's platform (Linux x64). Regenerating on another platform diffs byte-value lines only; take those lines from the CI job's output.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@graphite-app
graphite-app Bot force-pushed the allocs-snap-arena-bytes branch from 6caed9b to 510c435 Compare July 15, 2026 13:21
@graphite-app
graphite-app Bot merged commit 510c435 into main Jul 15, 2026
27 checks passed
@graphite-app graphite-app Bot removed the 0-merge Merge with Graphite Merge Queue label Jul 15, 2026
@graphite-app
graphite-app Bot deleted the allocs-snap-arena-bytes branch July 15, 2026 13:25
graphite-app Bot pushed a commit that referenced this pull request Jul 15, 2026
…ependent (#24565)

The `Allocations` CI job fails whenever the snapshots are regenerated on a different CPU architecture than CI (e.g. https://github.com/oxc-project/oxc/actions/runs/29420345285/job/87368903834): every `arena size` value differs by 32–128 bytes between aarch64 and x86_64. The cause is target-dependent type layout — hashbrown's table control groups are 16 bytes wide on x86_64 (SSE2) but 8 bytes on aarch64 (NEON) — so each live `HashMap` in the arena shifts `used_bytes()` slightly. This is the same root cause that #22621 handled for chunk counts; the `arena size` gauge added in #24553 re-exposed it. Allocation counts are unaffected and stay exact.

Exact byte equality across architectures isn't achievable (the arena genuinely occupies a different number of bytes), so instead the snapshot writer tolerates layout noise:

- Snapshots are now YAML (`allocs_*.yaml`) with the same layout as before: one value per line, file names at column 0 for git hunk headers, exact numbers as the diffed values, and human-readable sizes moved to trailing `#` comments.
- When regenerating, `cargo allocs` parses the committed snapshot (with `saphyr`, already a workspace dependency used by `oxc_coverage`) and keeps the committed `arena size` when the measured value is within `ARENA_SIZE_TOLERANCE` (1024 bytes, ~8× the observed worst-case cross-arch drift). Differences beyond the tolerance — real regressions such as growing an AST node type change sizes by orders of magnitude more — rewrite the value and fail the `git diff --exit-code` check as before.
- The trailing comments are rendered from the recorded value, so a snapshot regenerated on any platform stays byte-identical while within tolerance. No CI workflow changes are needed, and drift can't accumulate silently: each run compares against the committed value, so creeping sub-tolerance changes eventually cross the threshold and surface.

The committed values here are aarch64-measured; x86_64 CI measures 32–128 bytes higher and keeps them. Open PRs that touch the old `.snap` files will need a rebase and a `cargo allocs` rerun.

🤖 Implemented with [Claude Code](https://claude.com/claude-code).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant