Skip to content

turbo-tasks-backend: improve print_cache_item_size instrumentation#91742

Merged
sokra merged 4 commits intocanaryfrom
sokra/fix-print-cache-item-size
Mar 25, 2026
Merged

turbo-tasks-backend: improve print_cache_item_size instrumentation#91742
sokra merged 4 commits intocanaryfrom
sokra/fix-print-cache-item-size

Conversation

@sokra
Copy link
Copy Markdown
Member

@sokra sokra commented Mar 20, 2026

What?

Fixes a compilation hang that occurred when the print_cache_item_size debug feature was enabled,
and cleans up the surrounding instrumentation code.

Root cause of the hang (double lock / deadlock):

During persist_snapshot, the storage iterates over all modified tasks. For each task it calls:

let inner = self.shard.storage.map.get(&task_id).unwrap();
//          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
//          acquires a READ lock on the DashMap shard for task_id
let item = (self.shard.process)(task_id, &inner, &mut self.buffer);

The process closure receives &TaskStorage while that read lock is still held. Inside the
closure, the old code called:

self.get_task_name(task_id, turbo_tasks)

get_task_name creates an ExecuteContext, then calls ctx.task(task_id, TaskDataCategory::Data),
which calls storage.access_mut(task_id), which calls self.map.entry(key). That entry() call
tries to acquire a write lock on the same DashMap shard — which is already read-locked on the
same thread. parking_lot::RwLock is not reentrant, so the thread blocks forever waiting for its
own read lock to be released.

Fix: the &TaskStorage reference (inner) is already in scope inside the closure. Call
inner.get_persistent_task_type() directly instead of going through get_task_name, which avoids
any additional lock acquisition.

Additional changes in this PR:

  1. Make compressed-size reporting opt-in — split print_cache_item_size into two Cargo features.
    The base feature now shows uncompressed sizes only (no lz4 overhead). A new
    print_cache_item_size_with_compressed feature re-enables compressed-size reporting for when
    you need it.

  2. Extract helpers to eliminate duplication — the output block repeated #[cfg]-gated
    FormatSizes { … } struct literals six times and duplicated the task_name computation twice.
    Extract:

    • FormatSizes struct + impl Display (with #[cfg] isolated inside)
    • TaskCacheStats::task_name(storage) — single source for the grouping key
    • TaskCacheStats::sort_key() — encapsulates the #[cfg]-switched primary sort field
    • TaskCacheStats::format_total/data/avg_data/meta/avg_meta() — each returning FormatSizes

Why?

The compilation was hanging and unusable. The lock ordering bug was non-obvious because the re-entrant
acquisition happened across an abstraction boundary (get_task_name hiding the access_mut call).

How?

Avoid the redundant lock by using the &TaskStorage already provided to the closure.
Pure refactor for everything else — no behaviour change for any given feature combination.

sokra and others added 3 commits March 20, 2026 20:37
…with_compressed feature

The lz4 compression stats add overhead. Split print_cache_item_size into two features:
the base feature shows uncompressed sizes, while the new
print_cache_item_size_with_compressed feature adds compressed size reporting.
Enable print_cache_item_size by default for easier debugging.
Avoids the overhead of get_task_name by using the already-available inner
reference. Also fixes the print_cache_item_size guard to trigger on
encode_data || encode_meta instead of only encode_meta.
Extract FormatSizes struct and helper methods (task_name, sort_key,
format_total/data/meta) onto TaskCacheStats to eliminate duplicated
cfg-gated patterns and inline struct literals at the call site.

Co-Authored-By: Claude <[email protected]>
@nextjs-bot nextjs-bot added created-by: Turbopack team PRs by the Turbopack team. Turbopack Related to Turbopack with Next.js. labels Mar 20, 2026
@nextjs-bot
Copy link
Copy Markdown
Collaborator

nextjs-bot commented Mar 20, 2026

Failing test suites

Commit: da9382b | About building and testing Next.js

pnpm test-start-turbo test/e2e/app-dir/use-params/use-params.test.ts (turbopack) (job)

  • css-data-url-global-pages > should apply styles from data url correctly (DD)
Expand output

● css-data-url-global-pages › should apply styles from data url correctly

next build failed with code/signal 1

   97 |             if (code || signal)
   98 |               reject(
>  99 |                 new Error(
      |                 ^
  100 |                   `next build failed with code/signal ${code || signal}`
  101 |                 )
  102 |               )

  at ChildProcess.<anonymous> (lib/next-modes/next-start.ts:99:17)

pnpm test-start-turbo test/e2e/app-dir/next-config-ts/tsconfig-extends/next-config-ts-tsconfig-extends-cjs.test.ts (turbopack) (job)

  • next-config-ts-tsconfig-extends-cjs > should support tsconfig extends (CJS) (DD)
Expand output

● next-config-ts-tsconfig-extends-cjs › should support tsconfig extends (CJS)

next build failed with code/signal 1

   97 |             if (code || signal)
   98 |               reject(
>  99 |                 new Error(
      |                 ^
  100 |                   `next build failed with code/signal ${code || signal}`
  101 |                 )
  102 |               )

  at ChildProcess.<anonymous> (lib/next-modes/next-start.ts:99:17)

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq bot commented Mar 20, 2026

Merging this PR will improve performance by 4.64%

⚡ 8 improved benchmarks
✅ 9 untouched benchmarks
⏩ 3 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation build[framer-motion-all] 2.5 s 2.4 s +3.62%
Simulation build[date-fns-single] 1.2 s 1.2 s +3.55%
Simulation build[joy] 1.8 s 1.7 s +4.1%
Simulation build[lucide-react-all] 6.8 s 6.5 s +4.64%
Simulation build[mui] 2.6 s 2.5 s +3.88%
Simulation build[framer-motion-single] 1.9 s 1.8 s +3.82%
Simulation build[shiki] 5.1 s 4.8 s +4.55%
Simulation build[date-fns-all] 1.9 s 1.8 s +4.12%

Comparing sokra/fix-print-cache-item-size (da9382b) with canary (aa3ba7e)

Open in CodSpeed

Footnotes

  1. 3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@nextjs-bot
Copy link
Copy Markdown
Collaborator

nextjs-bot commented Mar 20, 2026

Stats from current PR

✅ No significant changes detected

📊 All Metrics
📖 Metrics Glossary

Dev Server Metrics:

  • Listen = TCP port starts accepting connections
  • First Request = HTTP server returns successful response
  • Cold = Fresh build (no cache)
  • Warm = With cached build artifacts

Build Metrics:

  • Fresh = Clean build (no .next directory)
  • Cached = With existing .next directory

Change Thresholds:

  • Time: Changes < 50ms AND < 10%, OR < 2% are insignificant
  • Size: Changes < 1KB AND < 1% are insignificant
  • All other changes are flagged to catch regressions

⚡ Dev Server

Metric Canary PR Change Trend
Cold (Listen) 455ms 455ms █▁▃▁▁
Cold (Ready in log) 440ms 439ms █▁▃▁▁
Cold (First Request) 1.106s 1.088s █▂▃▂▂
Warm (Listen) 456ms 457ms █▁▃▁▁
Warm (Ready in log) 443ms 444ms █▁▃▁▁
Warm (First Request) 342ms 344ms █▁▂▁▁
📦 Dev Server (Webpack) (Legacy)

📦 Dev Server (Webpack)

Metric Canary PR Change Trend
Cold (Listen) 455ms 456ms ▁▁▁█▁
Cold (Ready in log) 434ms 436ms ▁▁▁█▁
Cold (First Request) 1.923s 1.919s ▁▁▂█▃
Warm (Listen) 456ms 455ms ▁▁▁█▁
Warm (Ready in log) 435ms 436ms ▁▁▁█▁
Warm (First Request) 1.943s 1.942s ▁▁▂█▂

⚡ Production Builds

Metric Canary PR Change Trend
Fresh Build 3.836s 3.722s █▁▃▂▁
Cached Build 3.803s 3.775s █▁▃▂▁
📦 Production Builds (Webpack) (Legacy)

📦 Production Builds (Webpack)

Metric Canary PR Change Trend
Fresh Build 14.235s 14.237s ▁▁▁█▁
Cached Build 14.354s 14.325s ▁▁▁█▁
node_modules Size 484 MB 484 MB █████
📦 Bundle Sizes

Bundle Sizes

⚡ Turbopack

Client

Main Bundles
Canary PR Change
0-d~0o3ehhgmw.js gzip 151 B N/A -
0~lwfcrlb4v_9.css gzip 115 B 115 B
00h0nz7r436~l.js gzip 13.3 kB N/A -
00ivb_iunbucu.js gzip 13 kB N/A -
01at~wdgj81ve.js gzip 48.9 kB N/A -
02ku7edzc_wf7.js gzip 450 B N/A -
03~yq9q893hmn.js gzip 39.4 kB 39.4 kB
04z94byop2jge.js gzip 155 B N/A -
05s5bqmu.v.s2.js gzip 70.8 kB N/A -
092lcb3fqrrf9.js gzip 8.52 kB N/A -
0aj~xs1l1g8tg.js gzip 8.53 kB N/A -
0dt88d2.q3puq.js gzip 156 B N/A -
0g5sqjyu0dlae.js gzip 65.7 kB N/A -
0h35gmp9u328z.js gzip 8.54 kB N/A -
0h6fkavebp.iz.js gzip 8.47 kB N/A -
0ino_yf1k3h6k.js gzip 10.4 kB N/A -
0l9ef8qyt48q4.js gzip 161 B N/A -
0lqjp7063d~ts.js gzip 156 B N/A -
0mc16gv2x1bet.js gzip 13.7 kB N/A -
0mkdvwb8sm1cs.js gzip 157 B N/A -
0mncpry_rfn-y.js gzip 7.61 kB N/A -
0moy~uao4dl.m.js gzip 9.19 kB N/A -
0n3yn2kt3mgkf.js gzip 162 B N/A -
0oqyjf21.xfvl.js gzip 169 B N/A -
0q50rtpusjy90.js gzip 2.28 kB N/A -
0smgy2grrrlka.js gzip 8.58 kB N/A -
0svcqoqon1z.7.js gzip 154 B N/A -
0t1dzhdfh0txh.js gzip 215 B 215 B
0vt7pofxnk8in.js gzip 10.1 kB N/A -
0z6jc9z0349ap.js gzip 155 B N/A -
0zid7o0-vupvp.js gzip 225 B N/A -
11mdhajjc6875.js gzip 157 B N/A -
11yo3xfd6b147.js gzip 12.9 kB N/A -
12_8n261pygfq.js gzip 156 B N/A -
13.84hqxl_1p7.js gzip 9.76 kB N/A -
138ltu~qb2ydz.js gzip 158 B N/A -
1554wr-t7p6z-.js gzip 8.55 kB N/A -
15tjst79~qy3_.js gzip 1.46 kB N/A -
15z_v00ne4ud0.js gzip 8.47 kB N/A -
17d_m3p4j9w6r.js gzip 5.62 kB N/A -
17yu~3yiu7d2m.js gzip 8.52 kB N/A -
turbopack-0_..jw30.js gzip 4.15 kB N/A -
turbopack-01..hk.z.js gzip 4.13 kB N/A -
turbopack-01..3tdd.js gzip 4.17 kB N/A -
turbopack-02..cl~l.js gzip 4.15 kB N/A -
turbopack-09..wmtl.js gzip 4.16 kB N/A -
turbopack-0a..hs-0.js gzip 4.16 kB N/A -
turbopack-0f..4bnp.js gzip 4.16 kB N/A -
turbopack-0g..7vlj.js gzip 4.16 kB N/A -
turbopack-0p..6se..js gzip 4.16 kB N/A -
turbopack-0r...~ib.js gzip 4.16 kB N/A -
turbopack-0s..dnao.js gzip 4.16 kB N/A -
turbopack-0x..ivz7.js gzip 4.16 kB N/A -
turbopack-15..u41w.js gzip 4.15 kB N/A -
turbopack-16..~kqj.js gzip 4.15 kB N/A -
0.ig-nh6ddby4.js gzip N/A 153 B -
02ff96.~as-xu.js gzip N/A 157 B -
03t__~.5lvgeu.js gzip N/A 5.62 kB -
04d6ll75jqx3r.js gzip N/A 9.19 kB -
04wavr--oh-1-.js gzip N/A 155 B -
0583exyh-yhc7.js gzip N/A 9.76 kB -
072lv63r8dcz~.js gzip N/A 8.58 kB -
075t9dxgbf0m8.js gzip N/A 13.7 kB -
0873k5691ee0i.js gzip N/A 70.8 kB -
09123-xgm97yo.js gzip N/A 160 B -
0a3drqm~aryl6.js gzip N/A 153 B -
0ar1~bwpezfgw.js gzip N/A 13.3 kB -
0c99mq1ez2bke.js gzip N/A 450 B -
0cq-cmde_ws6u.js gzip N/A 8.47 kB -
0fwf102w10o9~.js gzip N/A 8.52 kB -
0gtmn.q_j1v5r.js gzip N/A 10.4 kB -
0h5~v-tahitcf.js gzip N/A 10.1 kB -
0i85sm0i~_x1y.js gzip N/A 48.8 kB -
0ka9wyuwc0va-.js gzip N/A 154 B -
0m8yz2-y.owhu.js gzip N/A 7.6 kB -
0nclq9z6yzzm5.js gzip N/A 1.46 kB -
0nga-9kug7bgp.js gzip N/A 154 B -
0nzumcogektg7.js gzip N/A 8.55 kB -
0p5sjual.nuis.js gzip N/A 13 kB -
0pet9ixg7-64s.js gzip N/A 149 B -
0pzv1cr_lc8sc.js gzip N/A 154 B -
0s.c-cn5eebrx.js gzip N/A 8.47 kB -
0t4u4gi13w~ge.js gzip N/A 65.7 kB -
0tna7lg6q4zne.js gzip N/A 12.9 kB -
0votdfxr5fb5u.js gzip N/A 2.28 kB -
0ykl9bs_qj.5..js gzip N/A 8.52 kB -
0zfen0tnxp4gh.js gzip N/A 8.55 kB -
10wkq1h9jzkg..js gzip N/A 225 B -
12s.i1.5kfw6p.js gzip N/A 168 B -
13yp0tf_.vo3_.js gzip N/A 152 B -
143nm3aj63v.8.js gzip N/A 152 B -
149ndfh8zfcaz.js gzip N/A 8.53 kB -
17vn26efzi_wl.js gzip N/A 154 B -
turbopack-01..1zy5.js gzip N/A 4.15 kB -
turbopack-04..ylsp.js gzip N/A 4.15 kB -
turbopack-05..2tc..js gzip N/A 4.16 kB -
turbopack-0g..j7lx.js gzip N/A 4.15 kB -
turbopack-0g..sg37.js gzip N/A 4.16 kB -
turbopack-0h..1v_h.js gzip N/A 4.16 kB -
turbopack-0i..qamy.js gzip N/A 4.17 kB -
turbopack-0o..hr_c.js gzip N/A 4.16 kB -
turbopack-0t..0nu2.js gzip N/A 4.16 kB -
turbopack-0v..9.j3.js gzip N/A 4.16 kB -
turbopack-0v..v9_z.js gzip N/A 4.16 kB -
turbopack-0x..yhs..js gzip N/A 4.16 kB -
turbopack-0y..l8a..js gzip N/A 4.13 kB -
turbopack-13..jmn~.js gzip N/A 4.16 kB -
Total 463 kB 463 kB ✅ -43 B

Server

Middleware
Canary PR Change
middleware-b..fest.js gzip 713 B 711 B
Total 713 B 711 B ✅ -2 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 431 B 430 B
Total 431 B 430 B ✅ -1 B

📦 Webpack

Client

Main Bundles
Canary PR Change
5528-HASH.js gzip 5.54 kB N/A -
6280-HASH.js gzip 60.6 kB N/A -
6335.HASH.js gzip 169 B N/A -
912-HASH.js gzip 4.59 kB N/A -
e8aec2e4-HASH.js gzip 62.7 kB N/A -
framework-HASH.js gzip 59.7 kB 59.7 kB
main-app-HASH.js gzip 255 B 254 B
main-HASH.js gzip 39.3 kB 39.2 kB
webpack-HASH.js gzip 1.68 kB 1.68 kB
262-HASH.js gzip N/A 4.59 kB -
2889.HASH.js gzip N/A 169 B -
5602-HASH.js gzip N/A 5.55 kB -
6948ada0-HASH.js gzip N/A 62.7 kB -
9544-HASH.js gzip N/A 61.3 kB -
Total 235 kB 235 kB ⚠️ +696 B
Polyfills
Canary PR Change
polyfills-HASH.js gzip 39.4 kB 39.4 kB
Total 39.4 kB 39.4 kB
Pages
Canary PR Change
_app-HASH.js gzip 194 B 194 B
_error-HASH.js gzip 183 B 180 B 🟢 3 B (-2%)
css-HASH.js gzip 331 B 330 B
dynamic-HASH.js gzip 1.81 kB 1.81 kB
edge-ssr-HASH.js gzip 256 B 256 B
head-HASH.js gzip 351 B 352 B
hooks-HASH.js gzip 384 B 383 B
image-HASH.js gzip 580 B 581 B
index-HASH.js gzip 260 B 260 B
link-HASH.js gzip 2.51 kB 2.51 kB
routerDirect..HASH.js gzip 320 B 319 B
script-HASH.js gzip 386 B 386 B
withRouter-HASH.js gzip 315 B 315 B
1afbb74e6ecf..834.css gzip 106 B 106 B
Total 7.98 kB 7.98 kB ✅ -1 B

Server

Edge SSR
Canary PR Change
edge-ssr.js gzip 125 kB 125 kB
page.js gzip 270 kB 270 kB
Total 395 kB 395 kB ⚠️ +217 B
Middleware
Canary PR Change
middleware-b..fest.js gzip 617 B 614 B
middleware-r..fest.js gzip 156 B 155 B
middleware.js gzip 43.9 kB 43.8 kB
edge-runtime..pack.js gzip 842 B 842 B
Total 45.5 kB 45.4 kB ✅ -110 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 715 B 718 B
Total 715 B 718 B ⚠️ +3 B
Build Cache
Canary PR Change
0.pack gzip 4.34 MB 4.33 MB 🟢 13.1 kB (0%)
index.pack gzip 110 kB 109 kB
index.pack.old gzip 110 kB 109 kB
Total 4.56 MB 4.55 MB ✅ -13.8 kB

🔄 Shared (bundler-independent)

Runtimes
Canary PR Change
app-page-exp...dev.js gzip 334 kB 334 kB
app-page-exp..prod.js gzip 182 kB 182 kB
app-page-tur...dev.js gzip 334 kB 334 kB
app-page-tur..prod.js gzip 181 kB 181 kB
app-page-tur...dev.js gzip 330 kB 330 kB
app-page-tur..prod.js gzip 179 kB 179 kB
app-page.run...dev.js gzip 331 kB 331 kB
app-page.run..prod.js gzip 180 kB 180 kB
app-route-ex...dev.js gzip 76.2 kB 76.2 kB
app-route-ex..prod.js gzip 51.8 kB 51.8 kB
app-route-tu...dev.js gzip 76.2 kB 76.2 kB
app-route-tu..prod.js gzip 51.9 kB 51.9 kB
app-route-tu...dev.js gzip 75.8 kB 75.8 kB
app-route-tu..prod.js gzip 51.6 kB 51.6 kB
app-route.ru...dev.js gzip 75.8 kB 75.8 kB
app-route.ru..prod.js gzip 51.6 kB 51.6 kB
dist_client_...dev.js gzip 324 B 324 B
dist_client_...dev.js gzip 326 B 326 B
dist_client_...dev.js gzip 318 B 318 B
dist_client_...dev.js gzip 317 B 317 B
pages-api-tu...dev.js gzip 43.4 kB 43.4 kB
pages-api-tu..prod.js gzip 33 kB 33 kB
pages-api.ru...dev.js gzip 43.4 kB 43.4 kB
pages-api.ru..prod.js gzip 33 kB 33 kB
pages-turbo....dev.js gzip 52.8 kB 52.8 kB
pages-turbo...prod.js gzip 38.6 kB 38.6 kB
pages.runtim...dev.js gzip 52.8 kB 52.8 kB
pages.runtim..prod.js gzip 38.6 kB 38.6 kB
server.runti..prod.js gzip 62.5 kB 62.5 kB
Total 2.96 MB 2.96 MB
📎 Tarball URL
https://vercel-packages.vercel.app/next/commits/da9382b4ebde74904edc143f6f53deb134a4d0a0/next

The print_cache_item_size feature should not pull in dep:lzzzz — that
dependency is only needed for the compressed-size variant and is already
correctly wired through print_cache_item_size_with_compressed.
Also revert default = ["print_cache_item_size"] to default = []:
the debug feature should not be on by default.

Co-Authored-By: Claude <[email protected]>
@sokra sokra marked this pull request as ready for review March 24, 2026 18:36
@sokra sokra requested a review from lukesandberg March 24, 2026 18:37
@sokra sokra merged commit 78f1625 into canary Mar 25, 2026
379 of 401 checks passed
@sokra sokra deleted the sokra/fix-print-cache-item-size branch March 25, 2026 07:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

created-by: Turbopack team PRs by the Turbopack team. Turbopack Related to Turbopack with Next.js.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants