Handle invalid sourcemaps by eps1lon · Pull Request #75713 · vercel/next.js

eps1lon · 2025-02-05T21:32:26Z

Sourcemaps can be user generated content (e.g. when modules are externals) so they need to be treated as untrusted. There are two places where an invalid sourcemap triggers errors:

Node.js' findSourceMap throws on invalid sourcemaps with "TypeError [ERR_INVALID_ARG_TYPE]: The "payload" argument must be of type object. Received null"
source-map throws when constructing the sourcemap consumer

Source map consumers are in large parts free to not abort parsing and just continue but Node.js and source-map are stricter in that regard. Chrome is more forgiving as far as I can tell. This strictness might also just be from a time where the spec wasn't finalized or didn't explicitly allow this level of lenience.

Closes https://linear.app/vercel/issue/NDX-505/

eps1lon · 2025-02-05T21:32:48Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

ijjk · 2025-02-06T22:25:32Z

Stats from current PR

Default Build (Increase detected ⚠️)

General Overall increase ⚠️

	vercel/next.js canary	vercel/next.js sebbie/02-05-handle_invalid_sourcemaps	Change
buildDuration	23.7s	20.5s	N/A
buildDurationCached	19s	16.9s	N/A
nodeModulesSize	393 MB	393 MB	⚠️ +29.5 kB
nextStartRea..uration (ms)	507ms	516ms	N/A

Client Bundles (main, webpack)

	vercel/next.js canary	vercel/next.js sebbie/02-05-handle_invalid_sourcemaps	Change
5306-HASH.js gzip	54.3 kB	54.3 kB	N/A
8276.HASH.js gzip	169 B	168 B	N/A
8377-HASH.js gzip	5.46 kB	5.46 kB	N/A
bccd1874-HASH.js gzip	53 kB	53 kB	✓
framework-HASH.js gzip	57.5 kB	57.5 kB	N/A
main-app-HASH.js gzip	245 B	245 B	✓
main-HASH.js gzip	34.6 kB	34.5 kB	N/A
webpack-HASH.js gzip	1.71 kB	1.71 kB	N/A
Overall change	53.2 kB	53.2 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	vercel/next.js sebbie/02-05-handle_invalid_sourcemaps	Change
polyfills-HASH.js gzip	39.4 kB	39.4 kB	✓
Overall change	39.4 kB	39.4 kB	✓

Client Pages

	vercel/next.js canary	vercel/next.js sebbie/02-05-handle_invalid_sourcemaps	Change
_app-HASH.js gzip	193 B	193 B	✓
_error-HASH.js gzip	193 B	193 B	✓
amp-HASH.js gzip	512 B	510 B	N/A
css-HASH.js gzip	343 B	342 B	N/A
dynamic-HASH.js gzip	1.84 kB	1.84 kB	✓
edge-ssr-HASH.js gzip	265 B	265 B	✓
head-HASH.js gzip	363 B	362 B	N/A
hooks-HASH.js gzip	393 B	392 B	N/A
image-HASH.js gzip	4.59 kB	4.58 kB	N/A
index-HASH.js gzip	268 B	268 B	✓
link-HASH.js gzip	2.35 kB	2.35 kB	N/A
routerDirect..HASH.js gzip	328 B	328 B	✓
script-HASH.js gzip	397 B	397 B	✓
withRouter-HASH.js gzip	323 B	326 B	N/A
1afbb74e6ecf..834.css gzip	106 B	106 B	✓
Overall change	3.59 kB	3.59 kB	✓

Client Build Manifests

	vercel/next.js canary	vercel/next.js sebbie/02-05-handle_invalid_sourcemaps	Change
_buildManifest.js gzip	748 B	747 B	N/A
Overall change	0 B	0 B	✓

Rendered Page Sizes

	vercel/next.js canary	vercel/next.js sebbie/02-05-handle_invalid_sourcemaps	Change
index.html gzip	522 B	523 B	N/A
link.html gzip	538 B	538 B	✓
withRouter.html gzip	518 B	520 B	N/A
Overall change	538 B	538 B	✓

Edge SSR bundle Size

	vercel/next.js canary	vercel/next.js sebbie/02-05-handle_invalid_sourcemaps	Change
edge-ssr.js gzip	130 kB	130 kB	N/A
page.js gzip	211 kB	211 kB	N/A
Overall change	0 B	0 B	✓

Middleware size

	vercel/next.js canary	vercel/next.js sebbie/02-05-handle_invalid_sourcemaps	Change
middleware-b..fest.js gzip	676 B	671 B	N/A
middleware-r..fest.js gzip	155 B	156 B	N/A
middleware.js gzip	31.3 kB	31.3 kB	N/A
edge-runtime..pack.js gzip	844 B	844 B	✓
Overall change	844 B	844 B	✓

Next Runtimes

	vercel/next.js canary	vercel/next.js sebbie/02-05-handle_invalid_sourcemaps	Change
app-page-exp...dev.js gzip	394 kB	394 kB	✓
app-page-exp..prod.js gzip	132 kB	132 kB	✓
app-page-tur..prod.js gzip	145 kB	145 kB	✓
app-page-tur..prod.js gzip	141 kB	141 kB	✓
app-page.run...dev.js gzip	381 kB	381 kB	✓
app-page.run..prod.js gzip	129 kB	129 kB	✓
app-route-ex...dev.js gzip	39.3 kB	39.3 kB	✓
app-route-ex..prod.js gzip	25.6 kB	25.6 kB	✓
app-route-tu..prod.js gzip	25.6 kB	25.6 kB	✓
app-route-tu..prod.js gzip	25.4 kB	25.4 kB	✓
app-route.ru...dev.js gzip	40.9 kB	40.9 kB	✓
app-route.ru..prod.js gzip	25.4 kB	25.4 kB	✓
dist_client_...dev.js gzip	356 B	356 B	✓
dist_client_...dev.js gzip	349 B	349 B	✓
pages-api-tu..prod.js gzip	9.69 kB	9.69 kB	✓
pages-api.ru...dev.js gzip	11.8 kB	11.8 kB	✓
pages-api.ru..prod.js gzip	9.68 kB	9.68 kB	✓
pages-turbo...prod.js gzip	21.9 kB	21.9 kB	✓
pages.runtim...dev.js gzip	31.5 kB	31.5 kB	✓
pages.runtim..prod.js gzip	21.9 kB	21.9 kB	✓
server.runti..prod.js gzip	60.6 kB	60.7 kB	N/A
Overall change	1.61 MB	1.61 MB	✓

build cache Overall increase ⚠️

	vercel/next.js canary	vercel/next.js sebbie/02-05-handle_invalid_sourcemaps	Change
0.pack gzip	2.11 MB	2.11 MB	⚠️ +2.8 kB
index.pack gzip	75.3 kB	76.8 kB	⚠️ +1.48 kB
Overall change	2.18 MB	2.19 MB	⚠️ +4.28 kB

Diff details

Diff for main-HASH.js

Diff too large to display

Diff for server.runtime.prod.js

Diff too large to display

Commit: bd2aece

eps1lon · 2025-02-06T23:25:04Z

packages/next/errors.json

  "633": "Dynamic route not found",
-  "634": "Route %s used \"searchParams\" inside \"use cache\". Accessing Dynamic data sources inside a cache scope is not supported. If you need this data inside a cached function use \"searchParams\" outside of the cached function and pass the required dynamic data in as an argument. See more info here: https://nextjs.org/docs/messages/next-request-in-use-cache"
+  "634": "Route %s used \"searchParams\" inside \"use cache\". Accessing Dynamic data sources inside a cache scope is not supported. If you need this data inside a cached function use \"searchParams\" outside of the cached function and pass the required dynamic data in as an argument. See more info here: https://nextjs.org/docs/messages/next-request-in-use-cache",
+  "635": "%s: Invalid source map. Only conformant source maps can be used to find the original code."


It's pretty jargon-y. The error cause is likely also not sufficient since the invalid sourcemap may have been produced by our bundler due to invalid input. But we also never have access to the source mapping URL. We only ever get access to the payload if it fails when constructing the source map consumer so we could theoretically dump the contents somewhere and link to that place. This feels excessive for such a rare issue that is in most cases caused by Next.js bugs (e.g. using turbopack://[project] as URLs so I feel comfortable with this error message.

Open to suggestions how to improve the wording though.

gaojude · 2025-02-07T00:46:59Z

test/e2e/app-dir/server-source-maps/server-source-maps.test.ts

+        '' +
+          '\nError: Boom!' +
+          // TODO(veil): Turbopack's sourcemap loader generates a wrong source entry here
+          // Should be not be sourcemapped or "custom://[badhost]/app/bad-sourcemap/page.js"


"should not be sourcemapped"

gaojude · 2025-02-07T00:48:50Z

test/e2e/app-dir/server-source-maps/fixtures/default/app/bad-sourcemap/page.js.map

@@ -0,0 +1,10 @@
+{


Interesting, I did not know that you could manually provide a source map in a .js.map file like this. What makes it work? Does compiler just copy it to the dist folder next to the .js?

The // #sourceMappingURL=... in the other file. The .map extension isn't required for sourcemapping to work.

You can also inline source maps via //# sourceMappingURL=data:application/json;base64,<base64EncodedSourcemapPayload>

gaojude · 2025-02-07T13:44:58Z

test/e2e/app-dir/server-source-maps/fixtures/default/app/bad-sourcemap/page.js

+// tsc compile errors can be ignored
+import { connection } from 'next/server'
+export default async function Page() {
+  await connection()


Is connection required here?

To make it dynamic. Otherwise we log during next build which complicates production testing. We only want to hit those errors when we actually navigate to the page.

ijjk · 2025-02-07T14:13:45Z

Tests Passed

ijjk added created-by: Next.js team PRs by the Next.js team. tests type: next labels Feb 5, 2025

eps1lon mentioned this pull request Feb 5, 2025

Send formatted error to frontend if sourcemapping fails #75712

Merged

eps1lon changed the title ~~Current behavior~~ Handle invalid sourcemaps Feb 5, 2025

eps1lon force-pushed the sebbie/02-05-send_formatted_error_to_frontend_if_sourcemapping_fails branch from 7119541 to 9a89990 Compare February 5, 2025 23:28

eps1lon force-pushed the sebbie/02-05-handle_invalid_sourcemaps branch from bbcd174 to f8960c2 Compare February 5, 2025 23:28

This was referenced Feb 5, 2025

Only disable Prettier for original Turbopack source #75716

Merged

Create valid URLs in source map sources with Turbopack #75719

Closed

eps1lon force-pushed the sebbie/02-05-send_formatted_error_to_frontend_if_sourcemapping_fails branch from 9a89990 to ccad701 Compare February 6, 2025 21:41

eps1lon force-pushed the sebbie/02-05-handle_invalid_sourcemaps branch from f8960c2 to b883440 Compare February 6, 2025 21:41

eps1lon mentioned this pull request Feb 6, 2025

Run full native build if native code changes #75767

Closed

eps1lon force-pushed the sebbie/02-05-handle_invalid_sourcemaps branch 2 times, most recently from 5ec9f44 to fe356cf Compare February 6, 2025 22:17

eps1lon force-pushed the sebbie/02-05-handle_invalid_sourcemaps branch 2 times, most recently from 148c515 to 0b1911c Compare February 6, 2025 23:18

eps1lon requested review from devjiwonchoi and gaojude and removed request for devjiwonchoi February 6, 2025 23:18

eps1lon marked this pull request as ready for review February 6, 2025 23:20

eps1lon commented Feb 6, 2025

View reviewed changes

eps1lon force-pushed the sebbie/02-05-handle_invalid_sourcemaps branch from 0b1911c to c66af5a Compare February 6, 2025 23:26

gaojude reviewed Feb 7, 2025

View reviewed changes

eps1lon changed the base branch from sebbie/02-05-send_formatted_error_to_frontend_if_sourcemapping_fails to graphite-base/75713 February 7, 2025 13:33

gaojude approved these changes Feb 7, 2025

View reviewed changes

gaojude reviewed Feb 7, 2025

View reviewed changes

eps1lon force-pushed the graphite-base/75713 branch from ccad701 to dc490a6 Compare February 7, 2025 14:03

eps1lon force-pushed the sebbie/02-05-handle_invalid_sourcemaps branch from c66af5a to 306b1c1 Compare February 7, 2025 14:03

eps1lon changed the base branch from graphite-base/75713 to canary February 7, 2025 14:03

eps1lon force-pushed the sebbie/02-05-handle_invalid_sourcemaps branch from 306b1c1 to 8377737 Compare February 7, 2025 14:03

eps1lon force-pushed the sebbie/02-05-handle_invalid_sourcemaps branch from 8377737 to 36685d8 Compare February 7, 2025 17:36

Current behavior

bd2aece

eps1lon force-pushed the sebbie/02-05-handle_invalid_sourcemaps branch from 36685d8 to bd2aece Compare February 9, 2025 11:39

eps1lon merged commit c003181 into canary Feb 9, 2025
131 checks passed

eps1lon deleted the sebbie/02-05-handle_invalid_sourcemaps branch February 9, 2025 12:40

eps1lon mentioned this pull request Feb 15, 2025

Create valid URLs in source map sources with Turbopack #76075

Merged

github-actions bot added the locked label Feb 24, 2025

github-actions bot locked as resolved and limited conversation to collaborators Feb 24, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Handle invalid sourcemaps#75713

Handle invalid sourcemaps#75713
eps1lon merged 1 commit intocanaryfrom
sebbie/02-05-handle_invalid_sourcemaps

eps1lon commented Feb 5, 2025 •

edited

Loading

Uh oh!

eps1lon commented Feb 5, 2025 •

edited

Loading

Uh oh!

ijjk commented Feb 6, 2025 •

edited

Loading

Uh oh!

eps1lon Feb 6, 2025

Uh oh!

gaojude Feb 7, 2025

Uh oh!

gaojude Feb 7, 2025 •

edited

Loading

Uh oh!

eps1lon Feb 7, 2025 •

edited

Loading

Uh oh!

gaojude Feb 7, 2025

Uh oh!

eps1lon Feb 7, 2025

Uh oh!

ijjk commented Feb 7, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

eps1lon commented Feb 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eps1lon commented Feb 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ijjk commented Feb 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Stats from current PR

Uh oh!

eps1lon Feb 6, 2025

Choose a reason for hiding this comment

Uh oh!

gaojude Feb 7, 2025

Choose a reason for hiding this comment

Uh oh!

gaojude Feb 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eps1lon Feb 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gaojude Feb 7, 2025

Choose a reason for hiding this comment

Uh oh!

eps1lon Feb 7, 2025

Choose a reason for hiding this comment

Uh oh!

ijjk commented Feb 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tests Passed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

eps1lon commented Feb 5, 2025 •

edited

Loading

eps1lon commented Feb 5, 2025 •

edited

Loading

ijjk commented Feb 6, 2025 •

edited

Loading

gaojude Feb 7, 2025 •

edited

Loading

eps1lon Feb 7, 2025 •

edited

Loading

ijjk commented Feb 7, 2025 •

edited

Loading