Skip to content

perf(cache): reuse the child path when its parent canonicalizes to itself#1288

Merged
Boshen merged 4 commits into
mainfrom
perf/canonicalize-ptr-eq-fast-path
Jul 11, 2026
Merged

perf(cache): reuse the child path when its parent canonicalizes to itself#1288
Boshen merged 4 commits into
mainfrom
perf/canonicalize-ptr-eq-fast-path

Conversation

@Boshen

@Boshen Boshen commented Jul 10, 2026

Copy link
Copy Markdown
Member

Summary

Each level of canonicalize_with_visited does one job: append the path's last component to whatever its parent canonicalized into — today via strip_prefix (O(len) component walk) + normalize_with (scratch-buffer copy of the whole path, full-path FxHash, dashmap shard probe).

When no ancestor is a symlink — the common case — the parent canonicalizes to itself, so that append can only rebuild the child's own key, and the shard probe is guaranteed to hand back the entry we already hold: four O(len) operations computing the identity function. This PR detects that case and returns the entry directly, per non-symlink level of every first-time canonicalization.

Example

pnpm project, node_modules/lodash → .pnpm/[email protected]/node_modules/lodash; canonicalize /app/node_modules/lodash/index.js (parent-first recursion):

  • /app/node_modules: parent /app canonicalized to itself → the rebuild could only reproduce /app/node_modulesfast path returns the entry as-is. Most levels of most paths look like this.
  • /app/node_modules/lodash: fast path for the string, then the (unchanged) symlink check lstats it, follows the link, and its canonical becomes the .pnpm entry.
  • /app/node_modules/lodash/index.js: the parent's canonical is now a different entry → ptr_eq fails → the rebuild does real work, appending index.js to the .pnpm parent. The rebuild runs exactly where its output differs from its input.

Why the guard is sound

The cache interns paths — one Arc<CachedPathImpl> per byte string — so Arc::ptr_eq(&parent_canonical.0, &parent.0) ⟺ canonicalizing the parent changed nothing ⟺ no ancestor was rewritten by a symlink. Under that precondition the rebuild returns path's own Arc iff the rebuilt string is byte-identical to path's key. PathBuf::push of one normal component appends MAIN_SEPARATOR + name, except it appends no separator when the base already ends with one (a root). So the fast path is restricted to keys shaped exactly <parent><one native separator><file_name>, checked in O(1):

  • file_name().is_some() — excludes ./.. tails, where the rebuild folds (/a/b/../a);
  • parent_len + 1 + name.len() == path_len — excludes trailing slashes and doubled separators mid-path;
  • path_bytes[parent_len] == MAIN_SEPARATOR — excludes mixed joints on Windows (C:\proj/src, where push would emit \);
  • path_bytes[parent_len − 1] != MAIN_SEPARATOR — excludes root-adjacent doubled separators (//x, C:\\x), where push appends no joint separator and the + 1 would otherwise count the doubled separator itself (caught in review; clean root children like /usr already failed the length check, so no fast-path coverage is lost);
  • wasm always takes the rebuild: component normalization is what trims the trailing NULs uvwasi can leave in directory entries, and reusing the key verbatim would keep them.

Keys failing the guard are exactly the spellings the rebuild would fold rather than reproduce — they take the rebuild path as before. The symlink check on the resulting entry is unchanged; only the string computation is skipped.

Correctness

  • canonicalize_matches_os_for_all_node_modules: 12,441 paths across 7 installed bench-pm combos (npm/pnpm/yarn/bun × flat/isolated/hoisted) match std::fs::canonicalize byte-for-byte.
  • New canonicalize_dirty_cache_keys test covers every guard exclusion (./.. tails and mid-chain, trailing, doubled, and root-adjacent doubled separators on Unix and Windows) against the OS oracle. It compares OsStr bytes, because Path's component-based PartialEq treats //x and /x as equal and would mask separator-level differences; the root-adjacent cases fail on the unfixed fast path and pass with the guard.
  • Full suite: 276 unit + 15 integration + 4 dependencies + 8 resolve_package + 13 doctests, clippy --deny warnings, fmt, on all CI platforms including Windows and big-endian.

Benchmarks

CodSpeed (instrumented instruction count, cold-cache pm workloads that exercise this path):

bench base head efficiency
pm/yarn-isolated 1,052.5 µs 851.1 µs +23.7%
pm/pnpm-isolated 1,050.8 µs 853.7 µs +23.1%
pm/pnpm-hoisted 1,070.9 µs 887.7 µs +20.7%
pm/npm-flat 968.8 µs 817.1 µs +18.6%
pm/bun-isolated 1,035.1 µs 897.3 µs +15.4%
pm/bun-flat 970.6 µs 844.2 µs +15.0%
pm/yarn-flat 944.6 µs 876.6 µs +7.8%

The flagged resolver_real[multi-thread] −6.2% is measurement slop: that bench reuses one resolver across iterations, so its steady state never reaches the changed code (the canonicalized OnceLock returns before the guard), and the same bench was untouched on e201289, which contains the identical fast path. Local wall-clock A/B (interleaved fixed binaries on a quiet machine, change measured first) agreed in direction: −1% to −3% on 7/8 pm combos, yarn-pnp neutral (dominated by .pnp.cjs parsing).

🤖 Generated with Claude Code

…self

When no ancestor of a path is a symlink, canonicalizing the parent yields
the parent's own cache entry, and rebuilding the child key via
strip_prefix + normalize_with (scratch-buffer copy, full-path hash, shard
probe) reproduces the child itself. Detect this with Arc::ptr_eq plus a
byte-length/separator guard and reuse the child entry directly.

Keys shaped other than <parent><MAIN_SEPARATOR><file_name> (.. or . tails,
trailing/doubled separators, forward-slash joints on Windows) fail the
guard and take the rebuild path, which folds them as before.
@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.07%. Comparing base (888c6af) to head (e7af5ca).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1288      +/-   ##
==========================================
+ Coverage   94.05%   94.07%   +0.01%     
==========================================
  Files          21       21              
  Lines        4275     4285      +10     
==========================================
+ Hits         4021     4031      +10     
  Misses        254      254              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed-hq

codspeed-hq Bot commented Jul 10, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 7.74%

⚡ 7 improved benchmarks
❌ 1 regressed benchmark
✅ 13 untouched benchmarks
⏩ 5 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
resolver_memory[multi-thread] 246.4 µs 255.1 µs -3.41%
pm/bun-flat 970.6 µs 843.1 µs +15.13%
pm/yarn-isolated 1,052.5 µs 939.8 µs +11.99%
pm/pnpm-hoisted 1,070.9 µs 968.4 µs +10.59%
pm/bun-isolated 1,035.1 µs 936.2 µs +10.57%
pm/pnpm-isolated 1,050.8 µs 977.5 µs +7.51%
pm/npm-flat 968.8 µs 919.2 µs +5.39%
pm/yarn-flat 944.6 µs 897.8 µs +5.21%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing perf/canonicalize-ptr-eq-fast-path (e7af5ca) with main (888c6af)

Open in CodSpeed

Footnotes

  1. 5 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e201289ea1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/cache/cache_impl.rs
…alize fast path

A root parent already ends with the separator, so rebuilding the child
adds no joint separator and the guard's + 1 counted a doubled separator
after the root instead: //x (or C:\\x) reused the dirty key rather than
folding it to /x the way strip_prefix + normalize_with does. Require the
byte before the joint to be a non-separator, which sends root children
down the rebuild path.

Path's component-based PartialEq treats //x and /x as equal, so the
dirty-key test now compares canonicalization results as OsStr bytes, and
covers root-adjacent doubled separators on both Unix and Windows.
@Boshen

Boshen commented Jul 11, 2026

Copy link
Copy Markdown
Member Author

Note on the CodSpeed report for 7f00a33: the flagged resolver_real[multi-thread] −6.18% is measurement slop, not the diff — that bench reuses one resolver across iterations, so its steady state never reaches the changed code (the canonicalized OnceLock returns before the guard), and the same bench was untouched on e201289, which contains the identical fast path (7f00a33 only adds one byte compare to the guard). The report itself carries the "different runtime environments" warning, which also explains the overall number moving +9.4% → +14.33% between two commits whose diff is three lines. The pm/* cold-cache benches — the ones that actually exercise the change — are consistently +7.8% to +23.7%.

Boshen added 2 commits July 11, 2026 19:13
Component normalization trims the trailing NULs uvwasi can leave in
directory entries; reusing the key verbatim would keep them. Also reword
the fast-path comment around the append-to-the-canonical-parent model.
@Boshen
Boshen merged commit 86af5b0 into main Jul 11, 2026
18 checks passed
@Boshen
Boshen deleted the perf/canonicalize-ptr-eq-fast-path branch July 11, 2026 14:43
@oxc-guard oxc-guard Bot mentioned this pull request Jul 10, 2026
Boshen pushed a commit that referenced this pull request Jul 12, 2026
## 🤖 New release

* `oxc_resolver`: 11.23.0 -> 11.24.0
* `oxc_resolver_napi`: 11.23.0 -> 11.24.0

<details><summary><i><b>Changelog</b></i></summary><p>

## `oxc_resolver`

<blockquote>

##
[11.24.0](v11.23.0...v11.24.0)
- 2026-07-12

### <!-- 0 -->🚀 Features

- expose tsconfig paths resolve method without file existence check
([#1282](#1282)) (by
@sapphi-red)
- add `ResolveError::TsconfigLoadFailed` for tsconfig read and parse
failures
([#1287](#1287)) (by
@shulaoda)

### <!-- 2 -->🚜 Refactor

- replace cfg-if dependency with std cfg_select! macro
([#1290](#1290)) (by
@Boshen)

### <!-- 3 -->📚 Documentation

- normalize README sponsor section
([#1273](#1273)) (by
@Boshen)

### <!-- 4 -->⚡ Performance

- *(cache)* reuse the child path when its parent canonicalizes to itself
([#1288](#1288)) (by
@Boshen)
- *(napi)* shrink release binaries (path remap + build-std without
backtrace)
([#1283](#1283)) (by
@Boshen)

### Contributors

* @sapphi-red
* @Boshen
* @renovate[bot]
* @shulaoda
</blockquote>



</p></details>

---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).

Co-authored-by: oxc-guard[bot] <276638029+oxc-guard[bot]@users.noreply.github.com>
@oxc-guard oxc-guard Bot mentioned this pull request Jul 12, 2026
Boshen pushed a commit that referenced this pull request Jul 12, 2026
## 🤖 New release

* `oxc_resolver`: 11.24.0 -> 11.24.1
* `oxc_resolver_napi`: 11.24.0 -> 11.24.1

<details><summary><i><b>Changelog</b></i></summary><p>

## `oxc_resolver`

<blockquote>

##
[11.24.0](v11.23.0...v11.24.0)
- 2026-07-12

### <!-- 0 -->🚀 Features

- expose tsconfig paths resolve method without file existence check
([#1282](#1282)) (by
@sapphi-red)
- add `ResolveError::TsconfigLoadFailed` for tsconfig read and parse
failures
([#1287](#1287)) (by
@shulaoda)

### <!-- 2 -->🚜 Refactor

- replace cfg-if dependency with std cfg_select! macro
([#1290](#1290)) (by
@Boshen)

### <!-- 3 -->📚 Documentation

- normalize README sponsor section
([#1273](#1273)) (by
@Boshen)

### <!-- 4 -->⚡ Performance

- *(cache)* reuse the child path when its parent canonicalizes to itself
([#1288](#1288)) (by
@Boshen)
- *(napi)* shrink release binaries (path remap + build-std without
backtrace)
([#1283](#1283)) (by
@Boshen)

### Contributors

* @sapphi-red
* @Boshen
* @renovate[bot]
* @shulaoda
</blockquote>



</p></details>

---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).

Co-authored-by: oxc-guard[bot] <276638029+oxc-guard[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant