Skip to content

perf(sourcemap): use raw tokens during composition#10255

Draft
hyfdev wants to merge 3 commits into
mainfrom
codex/sourcemap-composition-raw-tokens
Draft

perf(sourcemap): use raw tokens during composition#10255
hyfdev wants to merge 3 commits into
mainfrom
codex/sourcemap-composition-raw-tokens

Conversation

@hyfdev

@hyfdev hyfdev commented Jul 13, 2026

Copy link
Copy Markdown
Member

Why

The two PRs below this one establish the composition behavior for explicit unmapped boundaries and coarse same-line mappings. The hot loop only needs numeric token fields, but it still constructs source-view wrappers for every final and traced token.

This PR iterates raw Tokens instead. It keeps strict lookup as the common path and reads the first token on the same line only when strict lookup misses. The behavior and output are unchanged; the tests added by the two lower PRs cover both exceptional paths.

Benchmark

Command:

cargo bench -p bench --bench bench -- sourcemap/collapse_codegen_chain \
  --warm-up-time 2 --measurement-time 5 --sample-size 50

Environment: Apple M3 Pro, Darwin 25.5.0 arm64, rustc 1.97.0.

The repository's 52,004-token, three-map composition benchmark measured:

source-view tokens:  [2.2210 ms 2.2308 ms 2.2416 ms]
raw tokens, run 1:   [2.0215 ms 2.0285 ms 2.0368 ms]  (-8.93% median)
raw tokens, run 2:   [2.0089 ms 2.0179 ms 2.0298 ms]  (-9.28% median)

Both comparisons were statistically significant (p = 0.00). These numbers describe this isolated benchmark, not an end-to-end build speedup.

Validation

  • cargo test -p rolldown_sourcemap
  • just test-rust
  • both plugin regression fixtures from the lower PRs
  • cargo clippy -p rolldown_sourcemap --all-targets -- --deny warnings

hyfdev commented Jul 13, 2026

Copy link
Copy Markdown
Member Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more


How to use the Graphite Merge Queue

Add the label graphite: merge-when-ready to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

@hyfdev
hyfdev changed the base branch from codex/issue-10070-root-fix to graphite-base/10255 July 13, 2026 07:50
@hyfdev
hyfdev force-pushed the graphite-base/10255 branch from 204d8b1 to 2d55e36 Compare July 13, 2026 07:50
@hyfdev
hyfdev force-pushed the codex/sourcemap-composition-raw-tokens branch from 9434116 to 1aa4c44 Compare July 13, 2026 07:50
@hyfdev
hyfdev changed the base branch from graphite-base/10255 to codex/sourcemap-unmapped-boundaries July 13, 2026 07:50
@hyfdev
hyfdev force-pushed the codex/sourcemap-unmapped-boundaries branch from 2d55e36 to 64b4cb3 Compare July 13, 2026 07:57
@hyfdev
hyfdev force-pushed the codex/sourcemap-composition-raw-tokens branch 2 times, most recently from 1b99f9f to e51699c Compare July 13, 2026 09:49
@hyfdev
hyfdev force-pushed the codex/sourcemap-unmapped-boundaries branch from 64b4cb3 to f3b2bcb Compare July 13, 2026 09:49
@hyfdev
hyfdev force-pushed the codex/sourcemap-composition-raw-tokens branch from e51699c to 61eba3b Compare July 15, 2026 05:55
@hyfdev
hyfdev force-pushed the codex/sourcemap-unmapped-boundaries branch from f3b2bcb to c81e977 Compare July 15, 2026 05:55
@hyfdev
hyfdev force-pushed the codex/sourcemap-composition-raw-tokens branch from 61eba3b to f5060d4 Compare July 15, 2026 06:31
@hyfdev
hyfdev force-pushed the codex/sourcemap-unmapped-boundaries branch 2 times, most recently from 3600bc0 to 2e29bfc Compare July 15, 2026 08:37
Base automatically changed from codex/sourcemap-unmapped-boundaries to main July 15, 2026 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant