Skip to content

Avoid quadratic HMR queue shifts - #96137

Merged
marcoshernanz merged 2 commits into
canaryfrom
codex/perf-hmr-queue-index
Jul 25, 2026
Merged

marcoshernanz merged 2 commits into
canaryfrom
codex/perf-hmr-queue-index

Conversation

@marcoshernanz

Copy link
Copy Markdown
Contributor

What

  • Replace repeated Array.shift() calls in the HMR affected-module breadth-first traversal with a monotonically increasing queue index.
  • Clear each consumed slot immediately so copied dependency chains can be collected during deep traversals.

Why

getAffectedModuleEffects can enqueue a wide set of parents when a shared dependency changes. Repeatedly shifting the queue moves the remaining array contents on every iteration, making broad invalidations increasingly expensive. Indexed reads keep traversal FIFO and make queue consumption linear.

The slot clearing is important for deep graphs: it avoids retaining every processed item's copied dependencyChain until traversal returns.

Benchmark

Three independent 8-vCPU/16-GiB Vercel Sandbox workers ran Node.js 24.14.1 from exact commit 0950e8ae05db20c33302f02257114f5b857edabc. The balanced A1/B1/B2/A2 benchmark mirrors this loop for one changed shared dependency imported by N root modules, including dependency-chain allocation. Each phase used three warm-ups and nine samples.

Affected parents shift() median Indexed median Improvement Speedup Worker agreement Max baseline A/A drift
1,000 0.160 ms 0.089 ms 45.81% 1.85x 3/3 9.91%
20,000 27.895 ms 2.929 ms 89.43% 9.46x 3/3 7.85%
50,000 164.960 ms 7.342 ms 95.52% 22.33x 3/3 1.21%
100,000 665.564 ms 15.840 ms 97.62% 42.02x 3/3 2.23%

The small-graph effect is intentionally modest; this targets very broad HMR invalidations in large applications.

Validation

  • targeted Prettier: pass
  • targeted ESLint: pass
  • pnpm --filter @vercel/turbopack-ecmascript-runtime check:nodejs: pass
  • CARGO_TARGET_DIR=/home/vercel-sandbox/next.js/target cargo check -p turbopack-ecmascript-runtime: pass
  • GPT-5.6 Sol xhigh + Claude Opus 4.8 xhigh autoreview: clean after addressing a first-round dependency-chain retention finding (0 findings, correctness 0.99)

The aggregate runtime TypeScript check remains blocked by an existing runtime-utils.ts:239 EsmExport type mismatch. The identical check:browser-dev-client failure reproduces on the untouched baseline and is unrelated to this file.

@github-actions

github-actions Bot commented Jul 24, 2026 •

Copy link
Copy Markdown
Contributor

Tests Passed

Commit: 9df2c7c

@github-actions

github-actions Bot commented Jul 24, 2026 •

Copy link
Copy Markdown
Contributor

Stats from current PR

🔴 1 regression

Metric Canary PR Change Trend
Webpack Cold (First Request) 3.270s 3.407s 🔴 +137ms (+4%) ▂▁▁▅▁
📊 All Metrics
📖 Metrics Glossary

Dev Server Metrics:

  • Listen = TCP port starts accepting connections
  • First Request = HTTP server returns successful response
  • Cold = Fresh build (no cache)
  • Warm = With cached build artifacts

Build Metrics:

  • Fresh = Clean build (no .next directory)
  • Cached = With existing .next directory

Change Thresholds:

  • Time: Changes < 50ms AND < 10%, OR < 2% are insignificant
  • Size: Changes < 1KB AND < 1% are insignificant
  • All other changes are flagged to catch regressions

⚡ Dev Server

Metric Canary PR Change Trend
Cold (Listen) 813ms 811ms ✓ ▂▂█▇▁
Cold (Ready in log) 782ms 772ms ✓ ▂▁██▁
Cold (First Request) 1.279s 1.280s ✓ ▂▁█▇▁
Warm (Listen) 810ms 812ms ✓ ▂▂▇▇▂
Warm (Ready in log) 768ms 778ms ✓ ▁▁██▁
Warm (First Request) 1.252s 1.266s ✓ ▂▁██▁
📦 Dev Server (Webpack) (Legacy)

📦 Dev Server (Webpack)

Metric Canary PR Change Trend
Cold (Listen) 767ms 812ms ✓ ▄▄▅█▄
Cold (Ready in log) 766ms 782ms ✓ ▄▂▃█▂
Cold (First Request) 3.270s 3.407s 🔴 +137ms (+4%) ▂▁▁▅▁
Warm (Listen) 812ms 811ms ✓ ▄▄▄█▅
Warm (Ready in log) 780ms 775ms ✓ ▅▂▃█▃
Warm (First Request) 3.385s 3.335s ✓ ▃▁▂▅▂

⚡ Production Builds

Metric Canary PR Change Trend
Fresh Build 5.012s 4.958s ✓ ▃▁█▇▁
Cached Build 5.022s 5.023s ✓ ▂▁█▇▁
📦 Production Builds (Webpack) (Legacy)

📦 Production Builds (Webpack)

Metric Canary PR Change Trend
Fresh Build 23.352s 23.484s ✓ ▃▃▄█▁
Cached Build 23.393s 23.679s ✓ ▄▂▅█▃
node_modules Size 528 MB 528 MB ✓ ▁▁▁▁▁
📦 Bundle Sizes

Bundle Sizes

⚡ Turbopack

Client

Main Bundles
Canary PR Change
020ywijt5nj6h.js gzip 162 B N/A -
0a-p0meqxynjz.js gzip 10.3 kB N/A -
0avtke4yefj4-.js gzip 13.1 kB N/A -
0cz1d0mv5g_q7.js gzip 39.4 kB 39.4 kB ✓
0k0mowevltnd8.js gzip 156 B N/A -
0pt5of3ch6d2u.js gzip 13.6 kB N/A -
0xnsdo-6y9r3k.js gzip 5.72 kB N/A -
0yi7is8u4covm.js gzip 8.77 kB N/A -
13o5mq9-iq7w7.js gzip 65.6 kB N/A -
1d1l5ssh5rr7h.js gzip 157 B N/A -
1elt1qium-r2m.css gzip 115 B 115 B ✓
1prv-wg6jgtt2.js gzip 8.77 kB N/A -
1q2o3jxzuohdw.js gzip 157 B N/A -
1t47vod02e_4a.js gzip 8.75 kB N/A -
1tf1phijqlx9j.js gzip 220 B 220 B ✓
1tw002sq5bo4l.js gzip 10 kB N/A -
1wlerik1n7zbr.js gzip 8.7 kB N/A -
1xc6xtfjdu27q.js gzip 156 B N/A -
22q0om86f2f2y.js gzip 157 B N/A -
28koph4jjzmen.js gzip 45.2 kB N/A -
2l12bmavujito.js gzip 154 B N/A -
2lsr7u4ksfdec.js gzip 156 B N/A -
2lviwqiqxm904.js gzip 3.52 kB N/A -
2pevdqjpbf3xk.js gzip 9.45 kB N/A -
2qd2anu9eizlz.js gzip 8.75 kB N/A -
2sv39ae5_8vfj.js gzip 8.81 kB N/A -
2utx6n7w65xqa.js gzip 8.7 kB N/A -
3-ukivglqkfeq.js gzip 8.78 kB N/A -
325xisqrylugu.js gzip 450 B N/A -
3i1km7gub3184.js gzip 156 B N/A -
3jc9ghvds1spf.js gzip 2.29 kB N/A -
3juwkvkgvywfe.js gzip 10.6 kB N/A -
3kbove7q58uqh.js gzip 13.2 kB N/A -
3ku0nvgf4mxxb.js gzip 157 B N/A -
3lmdscffnd80s.js gzip 162 B N/A -
3qn7vntp30gaj.js gzip 1.47 kB N/A -
3u9ziwgnnaj5j.js gzip 7.4 kB N/A -
3xwty9p9n9ocu.js gzip 169 B N/A -
3yo-64802foqw.js gzip 71.1 kB N/A -
3z3gybhple7ca.js gzip 158 B N/A -
turbopack-0h..d21z.js gzip 3.81 kB N/A -
turbopack-0r..9xgy.js gzip 3.81 kB N/A -
turbopack-0w..1anq.js gzip 3.81 kB N/A -
turbopack-0y.._pk2.js gzip 3.81 kB N/A -
turbopack-19..c7bm.js gzip 3.81 kB N/A -
turbopack-1l..6x32.js gzip 3.82 kB N/A -
turbopack-1w..nfcm.js gzip 3.81 kB N/A -
turbopack-2_..xhvq.js gzip 3.81 kB N/A -
turbopack-26..pdsv.js gzip 3.79 kB N/A -
turbopack-2h..r2vy.js gzip 3.81 kB N/A -
turbopack-2t..o_0c.js gzip 3.81 kB N/A -
turbopack-2u..l8cy.js gzip 3.8 kB N/A -
turbopack-3_..r6ym.js gzip 3.81 kB N/A -
turbopack-39..ix4o.js gzip 3.81 kB N/A -
02sdbrhey7j7e.js gzip N/A 156 B -
030a1njeh5b77.js gzip N/A 10 kB -
0793r14k-42g0.js gzip N/A 71.1 kB -
0a8n-jguxfekn.js gzip N/A 65.6 kB -
0hvpxe74qwhsv.js gzip N/A 153 B -
0ot0qt5np20h7.js gzip N/A 8.78 kB -
0qpfmgpnybc5k.js gzip N/A 155 B -
1-qxxhh8-axib.js gzip N/A 45.2 kB -
12heqfhr2fq7h.js gzip N/A 157 B -
12m9rr7_7cexd.js gzip N/A 8.7 kB -
14e9p7ipvwu28.js gzip N/A 170 B -
186r3y_rfvr0b.js gzip N/A 8.78 kB -
19v97848yrjp7.js gzip N/A 8.81 kB -
1l1ravpajbxlq.js gzip N/A 162 B -
1nd_0rwgz2ozk.js gzip N/A 9.45 kB -
1oi9pltr_e10m.js gzip N/A 10.6 kB -
1rd9tzqgzsz2w.js gzip N/A 8.77 kB -
1rq1g_cwv642d.js gzip N/A 10.3 kB -
1uojw_uabeg93.js gzip N/A 450 B -
1w-7lusgyl81y.js gzip N/A 8.7 kB -
1xrsv7h-eh212.js gzip N/A 156 B -
2_yiudjsfz4l7.js gzip N/A 157 B -
21h56wdn9zddo.js gzip N/A 8.75 kB -
26oztzwo5776p.js gzip N/A 162 B -
28ezo_zo5_x-9.js gzip N/A 156 B -
2lg10omv7xu14.js gzip N/A 3.52 kB -
2ovff1533zvno.js gzip N/A 13.2 kB -
2s4z8jy0z7v93.js gzip N/A 1.46 kB -
2y6gb5bs4mnam.js gzip N/A 8.75 kB -
341h71ln1jpl3.js gzip N/A 2.29 kB -
34ttpt_n7605h.js gzip N/A 5.72 kB -
36ckfq9o-ys-a.js gzip N/A 13.1 kB -
38n1na-4zsfpn.js gzip N/A 7.41 kB -
3bsuzxfl28c9h.js gzip N/A 157 B -
3nc76t01leeue.js gzip N/A 156 B -
3su2d458ivoh-.js gzip N/A 157 B -
3vdh6_u50pu5t.js gzip N/A 13.6 kB -
turbopack-0-..ddc3.js gzip N/A 3.81 kB -
turbopack-0c..3fxe.js gzip N/A 3.81 kB -
turbopack-15..1-ev.js gzip N/A 3.81 kB -
turbopack-16..7lk6.js gzip N/A 3.8 kB -
turbopack-1k..ch4l.js gzip N/A 3.81 kB -
turbopack-1l..lxpb.js gzip N/A 3.81 kB -
turbopack-2v..cmeo.js gzip N/A 3.81 kB -
turbopack-2z..ib9y.js gzip N/A 3.8 kB -
turbopack-38..nxjf.js gzip N/A 3.81 kB -
turbopack-3b..-u7x.js gzip N/A 3.82 kB -
turbopack-3h..p6gp.js gzip N/A 3.79 kB -
turbopack-3l..j1et.js gzip N/A 3.81 kB -
turbopack-3s..bcea.js gzip N/A 3.8 kB -
turbopack-41..9dnj.js gzip N/A 3.81 kB -
Total 448 kB 448 kB ⚠️ +28 B

Server

Middleware
Canary PR Change
middleware-b..fest.js gzip 780 B 775 B ✓
Total 780 B 775 B ✅ -5 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 431 B 431 B ✓
Total 431 B 431 B ✓

📦 Webpack

Client

Main Bundles
Canary PR Change
3322-HASH.js gzip 63.7 kB N/A -
4191.HASH.js gzip 169 B N/A -
7920-HASH.js gzip 4.68 kB N/A -
9784-HASH.js gzip 5.63 kB N/A -
b1ad9f4c-HASH.js gzip 62.9 kB N/A -
framework-HASH.js gzip 59.7 kB 59.7 kB ✓
main-app-HASH.js gzip 254 B 253 B ✓
main-HASH.js gzip 40 kB 40 kB ✓
webpack-HASH.js gzip 1.68 kB 1.68 kB ✓
3577.HASH.js gzip N/A 168 B -
578-HASH.js gzip N/A 64.4 kB -
8590-HASH.js gzip N/A 5.61 kB -
9750-HASH.js gzip N/A 4.69 kB -
a8984546-HASH.js gzip N/A 62.9 kB -
Total 239 kB 239 kB ⚠️ +636 B
Polyfills
Canary PR Change
polyfills-HASH.js gzip 39.4 kB 39.4 kB ✓
Total 39.4 kB 39.4 kB ✓
Pages
Canary PR Change
_app-HASH.js gzip 194 B 193 B ✓
_error-HASH.js gzip 181 B 182 B ✓
css-HASH.js gzip 334 B 331 B ✓
dynamic-HASH.js gzip 1.81 kB 1.81 kB ✓
edge-ssr-HASH.js gzip 255 B 253 B ✓
head-HASH.js gzip 349 B 351 B ✓
hooks-HASH.js gzip 382 B 384 B ✓
image-HASH.js gzip 581 B 582 B ✓
index-HASH.js gzip 260 B 259 B ✓
link-HASH.js gzip 2.48 kB 2.48 kB ✓
routerDirect..HASH.js gzip 317 B 318 B ✓
script-HASH.js gzip 384 B 386 B ✓
withRouter-HASH.js gzip 316 B 315 B ✓
1afbb74e6ecf..834.css gzip 106 B 106 B ✓
Total 7.95 kB 7.95 kB ⚠️ +1 B

Server

Edge SSR
Canary PR Change
edge-ssr.js gzip 128 kB 128 kB ✓
page.js gzip 288 kB 287 kB ✓
Total 416 kB 415 kB ✅ -1.02 kB
Middleware
Canary PR Change
middleware-b..fest.js gzip 618 B 618 B ✓
middleware-r..fest.js gzip 156 B 156 B ✓
middleware.js gzip 45.3 kB 45.1 kB ✓
edge-runtime..pack.js gzip 842 B 842 B ✓
Total 46.9 kB 46.7 kB ✅ -212 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 718 B 718 B ✓
Total 718 B 718 B ✓
Build Cache
Canary PR Change
0.pack gzip 4.6 MB 4.6 MB ✓
index.pack gzip 121 kB 122 kB ✓
index.pack.old gzip 123 kB 122 kB 🟢 1.66 kB (-1%)
Total 4.85 MB 4.84 MB ✅ -4.89 kB

🔄 Shared (bundler-independent)

Runtimes
Canary PR Change
app-page-exp...dev.js gzip 366 kB 366 kB ✓
app-page-exp..prod.js gzip 202 kB 202 kB ✓
app-page-tur...dev.js gzip 366 kB 366 kB ✓
app-page-tur..prod.js gzip 202 kB 202 kB ✓
app-page-tur...dev.js gzip 362 kB 362 kB ✓
app-page-tur..prod.js gzip 200 kB 200 kB ✓
app-page.run...dev.js gzip 362 kB 362 kB ✓
app-page.run..prod.js gzip 200 kB 200 kB ✓
app-route-ex...dev.js gzip 81.8 kB 81.8 kB ✓
app-route-ex..prod.js gzip 55.6 kB 55.6 kB ✓
app-route-tu...dev.js gzip 81.8 kB 81.8 kB ✓
app-route-tu..prod.js gzip 55.6 kB 55.6 kB ✓
app-route-tu...dev.js gzip 81.4 kB 81.4 kB ✓
app-route-tu..prod.js gzip 55.4 kB 55.4 kB ✓
app-route.ru...dev.js gzip 81.4 kB 81.4 kB ✓
app-route.ru..prod.js gzip 55.4 kB 55.4 kB ✓
dist_client_...dev.js gzip 324 B 324 B ✓
dist_client_...dev.js gzip 326 B 326 B ✓
dist_client_...dev.js gzip 318 B 318 B ✓
dist_client_...dev.js gzip 317 B 317 B ✓
pages-api-tu...dev.js gzip 45.4 kB 45.4 kB ✓
pages-api-tu..prod.js gzip 34.1 kB 34.1 kB ✓
pages-api.ru...dev.js gzip 45.4 kB 45.4 kB ✓
pages-api.ru..prod.js gzip 34.1 kB 34.1 kB ✓
pages-turbo....dev.js gzip 54.9 kB 54.9 kB ✓
pages-turbo...prod.js gzip 39.7 kB 39.7 kB ✓
pages.runtim...dev.js gzip 54.8 kB 54.8 kB ✓
pages.runtim..prod.js gzip 39.7 kB 39.7 kB ✓
server.runti..prod.js gzip 68 kB 68 kB ✓
use-cache-pr...dev.js gzip 71.6 kB 71.6 kB ✓
use-cache-pr...dev.js gzip 71.6 kB 71.6 kB ✓
use-cache-pr...dev.js gzip 69.9 kB 69.9 kB ✓
use-cache-pr...dev.js gzip 69.9 kB 69.9 kB ✓
Total 3.51 MB 3.51 MB ⚠️ +1 B
📎 Tarball URL
https://vercel-packages.vercel.app/next/commits/9df2c7cdbe18ec144117cdd1a3430c8b2ede2f54/next

Commit: 9df2c7c

@marcoshernanz
marcoshernanz force-pushed the codex/perf-hmr-queue-index branch from adeba4a to a107ecb Compare July 24, 2026 08:30
Co-Authored-By: Marcos Hernanz <[email protected]>
@marcoshernanz
marcoshernanz marked this pull request as ready for review July 25, 2026 00:34
@marcoshernanz
marcoshernanz merged commit d5cc1e1 into canary Jul 25, 2026
129 checks passed
@marcoshernanz
marcoshernanz deleted the codex/perf-hmr-queue-index branch July 25, 2026 00:35
@bgw
bgw requested a review from wbinnssmith July 25, 2026 01:13
m-kawafuji pushed a commit to m-kawafuji/next.js that referenced this pull request Aug 8, 2026
## What

- Replace repeated `Array.shift()` calls in the HMR affected-module
breadth-first traversal with a monotonically increasing queue index.
- Clear each consumed slot immediately so copied dependency chains can
be collected during deep traversals.

## Why

`getAffectedModuleEffects` can enqueue a wide set of parents when a
shared dependency changes. Repeatedly shifting the queue moves the
remaining array contents on every iteration, making broad invalidations
increasingly expensive. Indexed reads keep traversal FIFO and make queue
consumption linear.

The slot clearing is important for deep graphs: it avoids retaining
every processed item's copied `dependencyChain` until traversal returns.

## Benchmark

Three independent 8-vCPU/16-GiB Vercel Sandbox workers ran Node.js
24.14.1 from exact commit `0950e8ae05db20c33302f02257114f5b857edabc`.
The balanced `A1/B1/B2/A2` benchmark mirrors this loop for one changed
shared dependency imported by N root modules, including dependency-chain
allocation. Each phase used three warm-ups and nine samples.

| Affected parents | `shift()` median | Indexed median | Improvement |
Speedup | Worker agreement | Max baseline A/A drift |
|---:|---:|---:|---:|---:|---:|---:|
| 1,000 | 0.160 ms | 0.089 ms | 45.81% | 1.85x | 3/3 | 9.91% |
| 20,000 | 27.895 ms | 2.929 ms | 89.43% | 9.46x | 3/3 | 7.85% |
| 50,000 | 164.960 ms | 7.342 ms | 95.52% | 22.33x | 3/3 | 1.21% |
| 100,000 | 665.564 ms | 15.840 ms | 97.62% | 42.02x | 3/3 | 2.23% |

The small-graph effect is intentionally modest; this targets very broad
HMR invalidations in large applications.

## Validation

- targeted Prettier: pass
- targeted ESLint: pass
- `pnpm --filter @vercel/turbopack-ecmascript-runtime check:nodejs`:
pass
- `CARGO_TARGET_DIR=/home/vercel-sandbox/next.js/target cargo check -p
turbopack-ecmascript-runtime`: pass
- GPT-5.6 Sol xhigh + Claude Opus 4.8 xhigh autoreview: clean after
addressing a first-round dependency-chain retention finding (0 findings,
correctness 0.99)

The aggregate runtime TypeScript check remains blocked by an existing
`runtime-utils.ts:239` `EsmExport` type mismatch. The identical
`check:browser-dev-client` failure reproduces on the untouched baseline
and is unrelated to this file.

---------

Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
@github-actions github-actions Bot added the locked label Sep 9, 2026
@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Sep 9, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants