Repository navigation
Turbopack: don't trace a package copy that a closer one shadows - #98798
Closed
davidbarratt wants to merge 1 commit into
Closed
davidbarratt wants to merge 1 commit into
davidbarratt wants to merge 1 commit into
Conversation
find_package walks every node_modules directory up the tree and returns each candidate it finds for a request. merge_alternatives then keeps only the first result for a given request key, so a candidate that a closer one shadows contributes nothing to the resolution - but its affecting sources were still attached to the merged result, and from there reached the output file trace. With pnpm that surfaces when a package has two copies in the store. Both the link in the importer's own node_modules and the hoisted link in node_modules/.pnpm/node_modules resolve, node uses the first, and the second was traced as a link to a copy whose files are not in the trace. Copied into a standalone build it becomes a symlink with no target, which fails a Vercel deploy during extraction with ENOENT on realpath. Carry the symlinks traversed to reach a candidate on the candidate itself, and skip candidates whose request keys are all claimed by a closer one. Candidates that resolve to nothing are kept, since node reads them too before falling through. Fixes #98791
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
find_packagewalks up thenode_moduleschain and returns every candidate it finds for a request, and the symlinks it traversed to reach each of them were collected into one list that got attached to the resolved result.merge_alternativeskeeps only the first result for a given request key, so a candidate that a closer one shadows contributes nothing to the resolution. Its symlinks were reported anyway, and from there they reached the output file trace.pnpm is where this shows up. It keeps a second copy of every transitive package in
node_modules/.pnpm/node_modules, so when a package has two copies in the store, both the link in the importer's ownnode_modulesand the hoisted link resolve. Node uses the closer one. The hoisted one was traced as a link into a copy whose files were never traced, andoutput: "standalone"copied it out as a symlink with nothing on the other end. Deploying that to Vercel fails during extraction, before the app runs:Symlinks are now carried on the candidate they belong to, and a candidate whose request keys are all claimed by a closer one is dropped along with them. Candidates that resolve to nothing are kept, since node reads those too before falling through to the next directory.
In the reproduction from #98791 this was every hoisted alias in the trace, not only the broken one — 47 of them.
scheduleris the one package there with two copies in the store, so it's the only alias whose link and traced files disagreed enough to break the deploy.Verification
cargo test -p turbopack-core. The new unit test builds a store with two copies of a package and asserts that only the link resolution actually used is reported. It fails without the change.pnpm test-start-turbo test/production/app-dir/tracing-shadowed-hoisted-package, and the same undertest-start-webpack. The fixture's store directory is named.storerather than.pnpmso the package manager running the test doesn't prune it. Nothing about the defect depends on the name.pnpm test-start-turbo test/production/standalone-mode,test/production/next-server-nftandtest/production/pnpm-support, pluspnpm test-start-webpack test/e2e/app-dir/app/standalone.test.ts, to check the trace hasn't lost anything it needs.next-swcwith and without the change and nothing else differing.find .next -xtype lgoes from one dangling symlink to none, and the files traced for the copy node resolves are unchanged.I wanted the integration test to assert on the standalone output directly, but it can't. The shadowed candidate's
package.jsonis read during resolution, so it gets traced and copied, and the link resolves. A dangling link needs the shadowed copy to contribute nothing at all, which is what happens toschedulerbut not in a fixture small enough to check in. The test asserts on the trace instead.I haven't bisected which release introduced this, and I don't know whether
.next/node_modules/<pkg>-<hash>in #95816 shares the cause. This change doesn't touch that path.Fixes #98791