Repository navigation
fix(DST-1904): run docs loaders in worker threads on Vercel - #5988
Conversation
…ild memory The docs build on Vercel's 8 GB machine is intermittently OOM-killed during Turbopack's compile. Its peak swings by about 1 GB between runs of the same code, and the Vite 8.3.3 bump tipped it over the limit. About half the compile-peak memory is the pool of Node child processes that run webpack loaders, mostly the fumadocs-mdx MDX compile. Running them in worker threads lowered the measured compile peak from 4.0 GB to 3.5 GB on average, with no change in build time. Vercel's Node 24.x is >= 24.13.1, where Next falls back to child processes unless 'forceWorkerThreads' is set. That bypasses a guard against nodejs/node#65100, whose known trigger is the macOS-only fsevents, so the setting is scoped to Vercel and local builds keep Next's default.
…builds VERCEL is also set by `vercel dev` on a developer machine. VERCEL_ENV is 'preview' or 'production' only on Vercel's build machines, so local runs on macOS, where fsevents can trigger nodejs/node#65100, keep Next's default.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
|
|
Accessibility tests executed. Download the report here. |
Coverage Report for Marigold Code Coverage
File CoverageNo changed files found. |
aromko
left a comment
There was a problem hiding this comment.
PR Review: #5988 — fix(DST-1904): run docs loaders in worker threads on Vercel
Overview
| Field | Value |
|---|---|
| Author | @sebald |
| Branch | fix/DST-1904-docs-build-memory → main |
| Files changed | 1 (+13 / -0) |
| Conventions source | CLAUDE.md |
| Visual regression | Not needed (build config only, no rendered output) |
Linked ticket
DST-1904: 🏗️ Keep the docs build on Vercel from running out of memory, In Review
The ticket asks for three things:
- Set
turbopackPluginRuntimeStrategy: 'forceWorkerThreads'. Done, and limited to Vercel preview and production builds. The ticket didn't ask for that limit, but it makes sense: the trigger in nodejs/node#65100 isfsevents, which is only installed on macOS. The PR description explains the choice well. - Set
VERCEL_BUILD_SYSTEM_REPORT=1on the project. The description says it's done (env idlG9sMyZmUQh7oBlx). - Redeploy 3–5 times. Still open: the
Vercel – marigold-docscheck is still deploying.
The ticket counts as done only when repeated preview builds and main builds succeed. That is a rollout condition, not something the code can show.
Verification
I loaded the PR's next.config.mjs through next/dist/server/config (Next 16.4.2) with each VERCEL_ENV:
VERCEL_ENV |
Resolved strategy |
|---|---|
| unset | childProcesses |
development |
childProcesses |
preview |
workerThreads |
production |
workerThreads |
This matches the PR description. Passing undefined lets Next fall back to its own default, so local builds are unaffected. I also read config.js:1086-1103: forceWorkerThreads maps to workerThreads and only logs the warning on Node >=24.13.1, as the code comment says.
Recommendation
Approve once the preview builds are green
It's a small, well-scoped change, and the investigation behind it is thorough: the A/B runs, the options that were ruled out, and a clear rollback signal (abort at worker teardown means revert). The code needs no changes. The one thing left is the condition the PR sets for itself: wait for 3–5 green marigold-docs preview builds in a row before merging. The build failed in 5 of 8 runs, so one green build doesn't prove much. If a build fails by aborting instead of being OOM-killed, revert as the description says.
Post-merge validation on VercelAll six
Build 6 reproduces the original failure scenario. The failed Before the change: 5 of 8 docs builds containing Vite 8.3.3 were OOM-killed. After: 6 of 6 green. Notes for future readers:
|
Description
The
marigold-docsbuild on Vercel is intermittently SIGKILLed without_of_memoryduring Turbopack's compile step. It started with the Vite 8.3.2 → 8.3.3 bump (#5980) and blocks the "Version Packages" release PR, becauseVercel – marigold-docsis a required status check.This PR lowers the build's peak memory instead of reverting any dependency. On Vercel preview and production builds only, Turbopack's webpack loaders (mostly the
fumadocs-mdxMDX compile) now run in worker threads instead of a pool of separate Node child processes:Local runs (
pnpm dev,pnpm build,vercel dev) keep Next's default. The investigation below is kept in full so it can be reused the next time the docs build runs out of memory.Closes DST-1904
Root cause
The docs build has been running close to the 8 GB limit of Vercel's standard build machine (4 cores, 8 GB), and its peak memory swings by about 1 GB between runs of identical code. The Vite bump didn't break anything; it tipped a borderline build over the edge.
Evidence from Vercel:
next build→ "Creating an optimized production build" (Turbopack compile), about 50s in. Green builds on the same machine compile in 26s–58s, so the compile phase itself is highly variable.e0b2e7ehas a tree byte-identical to the failingmaincommit62da88a(git diff --stat e0b2e7e 62da88ais empty) and built green.ERR_PNPM_OUTDATED_LOCKFILE,git_info_failor plain build errors). Since Vite 8.3.3 landed, 5 of 8 docs builds containing it were OOM-killed:dpl_6xTpNWTYEKhDZjfs7UwzhqinaRmTrenovate/npm-vite-vulnerability086ef40dpl_7mVoemwpYT7XPgr8oVXopNSA4Efcrenovate/npm-vite-vulnerabilitybd783dbdpl_2a4ZRERFRUQUxKHBZekhPeNiSe3Hdependabot/npm_and_yarn/vite-8.3.315898e6dpl_33nSKAvUnzuQZbgUZ8UTZKGUV3Z7dependabot/npm_and_yarn/vite-8.3.3e0b2e7edpl_BDeAi5f1ZkWvhEJev1gjvg4HtTGjmain(production)62da88adpl_JBrBfuadtPRRf28hGBQ7WLuh4vTJchangesets-ghcommit-temp/changeset-release/main96dd25ddpl_4vesPbs3knC2ztsZebk4oLohL6mYchangeset-release/main96dd25ddpl_Ef6eF4vcHBJ4owpF5MsJ2XEqQYgrdst-1751_select-all-for-none-table-collections2db3d3eThe last good production deployment is
dpl_DWb7szA6ukAdc5pLFs66gHXeEgSL(f8441c2); the live docs stay on it until this is fixed.What the bump changed for the docs build. The diff only touches
vitein the root andpackages/climanifests plus a lockfile dedupe. The one change inside the docs importer is thatfumadocs-mdxis re-resolved against new peers (rolldown 1.2.11 → 1.2.13,vite 8.3.2 → 8.3.3). Thefumadocs-mdxwebpack loader that Next uses does not import Vite or Rolldown; only itsdist/viteandrolldownadapters do. So there is no direct mechanism. The bump shifted a noisy distribution just enough.Where the memory goes
A snapshot of the
next buildprocess tree at its compile peak (local, macOS):pool_entry-[turbopack-node]_transforms_webpack-loaders)next-build(Rust compiler)More than half the peak is the pool of Node child processes that run JS loaders, each with its own Node runtime and heap. Across runs the pool grew to 35–51 processes on an 8-core machine. That pool is what this PR targets.
What was measured
Method. Local runs on macOS (16 GB, 8 cores, Node 24.11.0) in detached worktrees. Peak = maximum of the summed RSS of the
next buildprocess tree, sampled every 0.5s. "Compile peak" stops atCompiled successfully. Summed RSS counts shared pages once per process, so it overstates the child-process setup and the absolute numbers don't map 1:1 to Vercel's Linux container. Treat them as relative comparisons. The run-to-run noise is about ±1 GB, so every variant ran 2–3 times.1. Cold A/B: last green commit vs. failing commit
f8441c2(green on Vercel)62da88a(red on Vercel)Possibly slightly higher, but within noise. This matches "borderline build tipped over", not "regression".
2. Turbopack build cache (
experimental.turbopackFileSystemCacheForBuild, defaulttruein 16.4.2): ruled outSimulates Vercel exactly:
f8441c2was built in place, then62da88awas checked out at the same path and built on top of that cache.turbopackFileSystemCacheForBuild: falseDisabling the cache doesn't lower the peak. Side finding worth a separate look: a cache restored from a build under a different lockfile makes the compile about 2.5× slower than a cold build. Vercel restores the previous deployment's cache on every build, so dependency-bump builds pay this. Not a memory problem, but it costs build minutes.
3. Capping Turbopack's Rust thread pools: ruled out
TOKIO_WORKER_THREADS=2TOKIO_WORKER_THREADS=2 RAYON_NUM_THREADS=2Both names are compiled into the
@next/swcbinary, but Next configures its runtimes explicitly. The thread count doesn't change, so the env vars have no effect.4. Loader runtime: child processes vs. worker threads (this PR)
experimental.turbopackPluginRuntimeStrategy, added in Next 16.4.0, documented innode_modules/next/dist/docs/01-app/03-api-reference/05-config/01-next-config-js/turbopackPluginRuntimeStrategy.md.childProcesses(default)workerThreadsAbout 0.5 GB (~13%) lower on average, consistently lower in each pair, all builds green, build time unchanged. Because summed RSS overcounts child processes, the real saving on Vercel is probably somewhat smaller. That's why this PR also turns on Vercel's build system report (see Rollout).
Options checked and not available
memoryLimitdoesn't appear inconfig-schema.js).experimental.turbopackMemoryEviction: documented as "only effective in dev sessions".experimental.turbopackGc: prunes unreachable tasks from the persistent cache, which doesn't lower the peak.experimental.cpus/staticGenerationMaxConcurrency: these only affect static page generation, which runs after the compile step where the OOM happens.Why
forceWorkerThreads, and why only on Vercel buildsOn Node
>= 24.13.1, Next ignores'workerThreads', prints a warning and falls back to child processes (next/dist/server/config.js). That guards against nodejs/node#65100: the process can abort when a worker thread exits while a native addon holds a live Node-API threadsafe function. Vercel's24.xis newer than 24.13.1, so'forceWorkerThreads'is the only way to get worker threads there.The trigger Next names is
fsevents, the macOS file watcher. Vercel builds on Linux, wherefseventsisn't installed andnext builddoesn't watch files. On developer Macs,fseventsis installed and.node-versionpins only24, so a teammate on the latest Node 24 would be exposed. Hence the scoping:VERCEL_ENVpnpm dev/pnpm buildlocallychildProcesses(Next default)developmentvercel devlocallychildProcesses(Next default)previewworkerThreads(forced)productionworkerThreads(forced)Verified locally by loading the config through
next/dist/server/configwith eachVERCEL_ENVvalue.Both Vercel projects build the docs
marigold-docsmain(marigold-docs-git-main-…)marigold-productiondocsbranch (an ignore step cancels everything else), whichrelease.ymlupdates after each npm publishBoth run
pnpm buildindocs/with the samenext.config.mjs, so this change covers both:marigold-productionbuilds withVERCEL_ENV=production. It hasn't failed yet only because its last build (dpl_6z9aSsoQwrzPQTdBNhyzdDF91gBd, Oct 12, compile 55s) predates the Vite bump. The first build after the next release would have been the first one at risk, and a failure there would keep the live site on the old version.Rollout and validation
VERCEL_BUILD_SYSTEM_REPORT=1is now set on both projects:marigold-docs(production + preview, env idlG9sMyZmUQh7oBlx) andmarigold-production(production; its previews never build). Vercel's own OOM message recommends it. Every build log now ends with a build system report, so we get the real memory numbers from the Linux container instead of macOS estimates. It's a diagnostic setting and can be removed once things are stable.marigold-docs(16 GB build machine, paid).Test Instructions
marigold-docsVercel build log. In thenext buildheader, the experiments list should showturbopackPluginRuntimeStrategy: "workerThreads"along with Next's warning that'forceWorkerThreads'bypasses the Node.js guard. The log should end with the "Build system report".vercel redeploy <preview-url> --scope marigold). Every build must finish, and the report shows the peak memory.cd docs && pnpm build. The experiments list must not showturbopackPluginRuntimeStrategy, because local builds keep Next's default.unsetanddevelopment→childProcesses,previewandproduction→workerThreads.Breaking Changes
No
Checklist
component-testtag where applicable)pnpm changeset)N/A for this PR: stories, unit tests, component docs, accessibility, VRT and changeset. It's a build-config change to the private
@marigold/docspackage, with no published package and no UI change. The docs preview item is left for the reviewer, since it depends on the Vercel builds above.