turbo-tasks-backend: improve print_cache_item_size instrumentation#91742
turbo-tasks-backend: improve print_cache_item_size instrumentation#91742
Conversation
…with_compressed feature The lz4 compression stats add overhead. Split print_cache_item_size into two features: the base feature shows uncompressed sizes, while the new print_cache_item_size_with_compressed feature adds compressed size reporting. Enable print_cache_item_size by default for easier debugging.
Avoids the overhead of get_task_name by using the already-available inner reference. Also fixes the print_cache_item_size guard to trigger on encode_data || encode_meta instead of only encode_meta.
Extract FormatSizes struct and helper methods (task_name, sort_key, format_total/data/meta) onto TaskCacheStats to eliminate duplicated cfg-gated patterns and inline struct literals at the call site. Co-Authored-By: Claude <[email protected]>
Failing test suitesCommit: da9382b | About building and testing Next.js
Expand output● css-data-url-global-pages › should apply styles from data url correctly
Expand output● next-config-ts-tsconfig-extends-cjs › should support tsconfig extends (CJS) |
Merging this PR will improve performance by 4.64%
Performance Changes
Comparing Footnotes
|
Stats from current PR✅ No significant changes detected📊 All Metrics📖 Metrics GlossaryDev Server Metrics:
Build Metrics:
Change Thresholds:
⚡ Dev Server
📦 Dev Server (Webpack) (Legacy)📦 Dev Server (Webpack)
⚡ Production Builds
📦 Production Builds (Webpack) (Legacy)📦 Production Builds (Webpack)
📦 Bundle SizesBundle Sizes⚡ TurbopackClient Main Bundles
Server Middleware
Build DetailsBuild Manifests
📦 WebpackClient Main Bundles
Polyfills
Pages
Server Edge SSR
Middleware
Build DetailsBuild Manifests
Build Cache
🔄 Shared (bundler-independent)Runtimes
📎 Tarball URL |
The print_cache_item_size feature should not pull in dep:lzzzz — that dependency is only needed for the compressed-size variant and is already correctly wired through print_cache_item_size_with_compressed. Also revert default = ["print_cache_item_size"] to default = []: the debug feature should not be on by default. Co-Authored-By: Claude <[email protected]>
What?
Fixes a compilation hang that occurred when the
print_cache_item_sizedebug feature was enabled,and cleans up the surrounding instrumentation code.
Root cause of the hang (double lock / deadlock):
During
persist_snapshot, the storage iterates over all modified tasks. For each task it calls:The
processclosure receives&TaskStoragewhile that read lock is still held. Inside theclosure, the old code called:
get_task_namecreates anExecuteContext, then callsctx.task(task_id, TaskDataCategory::Data),which calls
storage.access_mut(task_id), which callsself.map.entry(key). Thatentry()calltries to acquire a write lock on the same DashMap shard — which is already read-locked on the
same thread.
parking_lot::RwLockis not reentrant, so the thread blocks forever waiting for itsown read lock to be released.
Fix: the
&TaskStoragereference (inner) is already in scope inside the closure. Callinner.get_persistent_task_type()directly instead of going throughget_task_name, which avoidsany additional lock acquisition.
Additional changes in this PR:
Make compressed-size reporting opt-in — split
print_cache_item_sizeinto two Cargo features.The base feature now shows uncompressed sizes only (no lz4 overhead). A new
print_cache_item_size_with_compressedfeature re-enables compressed-size reporting for whenyou need it.
Extract helpers to eliminate duplication — the output block repeated
#[cfg]-gatedFormatSizes { … }struct literals six times and duplicated thetask_namecomputation twice.Extract:
FormatSizesstruct +impl Display(with#[cfg]isolated inside)TaskCacheStats::task_name(storage)— single source for the grouping keyTaskCacheStats::sort_key()— encapsulates the#[cfg]-switched primary sort fieldTaskCacheStats::format_total/data/avg_data/meta/avg_meta()— each returningFormatSizesWhy?
The compilation was hanging and unusable. The lock ordering bug was non-obvious because the re-entrant
acquisition happened across an abstraction boundary (
get_task_namehiding theaccess_mutcall).How?
Avoid the redundant lock by using the
&TaskStoragealready provided to the closure.Pure refactor for everything else — no behaviour change for any given feature combination.