Skip to content

feat(pages-router): consume _next/data JSON endpoint from the client#1412

Merged
james-elicx merged 14 commits into
mainfrom
james/_next/data-client
May 22, 2026
Merged

feat(pages-router): consume _next/data JSON endpoint from the client#1412
james-elicx merged 14 commits into
mainfrom
james/_next/data-client

Conversation

@james-elicx

Copy link
Copy Markdown
Member

The server-side /_next/data/<buildId>/<page>.json endpoint landed in #1384, but the vinext client was still fetching the full HTML page on every navigation and regex-extracting __NEXT_DATA__ from it. This wires the client to actually use the JSON endpoint, matching what Next.js does and avoiding the HTML round trip.

Closes the follow-up half of #1330.

What changed

Client navigation (shims/router.ts). navigateClient is split into two paths:

  • navigateClientData (new): when window.__VINEXT_PAGE_LOADERS__ has a registered code-split loader for the target route AND we have a buildId, fetch /_next/data/<buildId>/<page>.json with Accept: application/json + x-nextjs-data: 1. On 200 parse the envelope, look up the page module via the in-memory loader map, and re-render.
  • navigateClientHtml: the previous HTML-fetch + regex-extract logic, unchanged. Used in dev (where the inline hydration script does not populate the loader map) and as a fallback for any route not in the client loader map.

Failure modes — 404, 5xx, network error, malformed JSON, soft redirect via x-nextjs-redirect, missing loader — all queue a hard navigation (window.location.href = url). This is the deploy-skew safety net: when the server's buildId has rotated, the data endpoint returns 404 and the client lands on the new build via a full document load.

Code-split loader manifest (entries/pages-client-entry.ts). Vite already code-splits each page into its own chunk via the dynamic imports in pageLoaders. The generated client entry now exposes that map on window.__VINEXT_PAGE_LOADERS__ (route pattern → loader thunk), plus window.__VINEXT_PAGE_PATTERNS__ (ordered list) and window.__VINEXT_APP_LOADER__ (the _app thunk, when present). This replaces Next.js' _buildManifest.js indirection — vinext already generates the equivalent at build time, we just had to expose it.

Data URL construction (shims/internal/pages-data-url.ts, new). Pure helper module mirroring getAssetPathFromRoute + getDataHref from Next.js:

Route Data URL
/ /_next/data/<id>/index.json
/about /_next/data/<id>/about.json
/index /_next/data/<id>/index/index.json
/blog/foo /_next/data/<id>/blog/foo.json

Round-trip tests verify the server's parseNextDataPathname agrees with the client's buildPagesDataPath for every shape — and surfaced a latent server bug: /_next/data/<id>/index/foo.json was parsing back to /index/foo instead of /foo. Fixed in pages-data-route.ts so the round-trip is now lossless.

Prefetch (shims/link.tsx + shims/router.ts). <Link> prefetch and Router.prefetch() now prefer the data path:

  • Inject <link rel=\"prefetch\" as=\"fetch\" crossorigin href={dataUrl}> for the JSON, matching the actual navigation's request shape.
  • Invoke the loader thunk to warm the chunk in parallel. Vite's import() caches the result, so the eventual click is a double cache hit.

When no loader matches (dev, unmapped route), fall back to the legacy <link rel=\"prefetch\" as=\"document\"> so the browser at least preloads the HTML.

This also fixes a long-standing prefetch bug as a side effect: the previous Pages branch of link.tsx's prefetch check (__NEXT_DATA__.__vinext?.pageModuleUrl) was only populated by the dev server's inline hydration script, so prefetch in production was a no-op.

Ports from Next.js

  • getDataHref in .nextjs-ref/packages/next/src/client/page-loader.ts:154
  • fetchNextData in .nextjs-ref/packages/next/src/shared/lib/router/router.ts:498
  • getAssetPathFromRoute in .nextjs-ref/packages/next/src/shared/lib/router/utils/get-asset-path-from-route.ts

Tests

  • tests/pages-data-url.test.ts (new): 27 tests covering URL construction and the client/server round trip.
  • tests/shims.test.ts (+8 new tests): JSON-path navigation success, 404 hard-reload (build-skew), x-nextjs-redirect soft-redirect handling, HTML fallback when no loader matches, dynamic param extraction, malformed JSON hard-reload, prefetch JSON path, prefetch fallback.

Verified locally: 1393 vitest tests pass across shims, pages-router, link/navigation, prefetch, data-route, data-url, entry-templates, and routing test files.

Out of scope (potential follow-ups)

  • Dev server JSON-path navigation. Dev injects per-page inline hydration scripts and does not populate the loader map, so dev navigation still uses the HTML path. To bring dev up to parity we would emit a small loaders manifest alongside the dev hydration script — a separate small PR.
  • Soft-redirect handled as a client-side Router.replace instead of a hard reload. Today the x-nextjs-redirect header hard-loads; a future PR can wire it through Router.replace to keep the redirect a client-side navigation.

The server-side `/_next/data/<buildId>/<page>.json` endpoint landed in
#1384, but the vinext client was still fetching the full HTML page on
every navigation and regex-extracting `__NEXT_DATA__` from it. This
wires the client to actually use the JSON endpoint, matching what
Next.js does and avoiding the HTML round trip.

## Client navigation (shims/router.ts)

`navigateClient` is split into two paths:

- `navigateClientData` (new): when `window.__VINEXT_PAGE_LOADERS__` has
  a registered code-split loader for the target route AND we have a
  buildId, fetch `/_next/data/<buildId>/<page>.json` with
  `Accept: application/json` + `x-nextjs-data: 1`. On 200 parse the
  envelope, look up the page module via the in-memory loader map (Vite
  has already split each page into its own chunk), and re-render.
- `navigateClientHtml`: the previous HTML-fetch + regex-extract logic,
  unchanged. Used in dev (where the inline hydration script does not
  populate the loader map) and as a fallback for any route not in the
  client loader map.

Failure modes — 404, 5xx, network error, malformed JSON, soft
redirect via `x-nextjs-redirect`, missing loader — all queue a hard
navigation (`window.location.href = url`). This is the deploy-skew
safety net: when the server's buildId has rotated, the data endpoint
returns 404 and the client lands on the new build via a full document
load.

## Code-split loader manifest (entries/pages-client-entry.ts)

Vite already code-splits each page into its own chunk via the dynamic
imports in `pageLoaders`. The generated client entry now exposes that
map on `window.__VINEXT_PAGE_LOADERS__` (route pattern → loader thunk),
plus `window.__VINEXT_PAGE_PATTERNS__` (ordered list) and
`window.__VINEXT_APP_LOADER__` (the `_app` thunk, when present). This
replaces Next.js' `_buildManifest.js` indirection — vinext already
generates the equivalent at build time, we just had to expose it.

## Data URL construction (shims/internal/pages-data-url.ts)

New pure helper module mirroring `getAssetPathFromRoute` +
`getDataHref` from Next.js. Handles:

- `/`            → `/_next/data/<id>/index.json`
- `/about`       → `/_next/data/<id>/about.json`
- `/index`       → `/_next/data/<id>/index/index.json` (round-trip)
- `/blog/foo`    → `/_next/data/<id>/blog/foo.json`

Also exports `matchPagesPattern` which uses the existing
`matchRoutePattern` to map a URL pathname to a registered route
pattern.

Round-trip tests verify the server's `parseNextDataPathname` agrees
with the client's `buildPagesDataPath` for every shape, and surfaced a
latent server bug: `/_next/data/<id>/index/foo.json` was parsing back
to `/index/foo` instead of `/foo`. Fixed in `pages-data-route.ts` so
the round-trip is now lossless.

## Prefetch (shims/link.tsx + shims/router.ts)

`<Link>` prefetch and `Router.prefetch()` now prefer the data path:

- Inject `<link rel="prefetch" as="fetch" crossorigin href={dataUrl}>`
  for the JSON, matching the actual navigation's request shape.
- Invoke the loader thunk to warm the chunk in parallel. Vite's
  `import()` caches the result, so the eventual click is a double
  cache hit.

When no loader matches (dev, unmapped route), fall back to the legacy
`<link rel="prefetch" as="document">` so the browser at least preloads
the HTML.

Notably this also fixes a long-standing prefetch bug: the previous
Pages branch of `link.tsx`'s prefetch check
(`__NEXT_DATA__.__vinext?.pageModuleUrl`) was only populated by the
dev server's inline hydration script, so prefetch in production was a
no-op.

## Tests

- `tests/pages-data-url.test.ts` (new): 27 tests covering URL
  construction and the client/server round trip.
- `tests/shims.test.ts` (+8): JSON-path navigation success, 404
  hard-reload (build-skew), `x-nextjs-redirect` soft-redirect handling,
  HTML fallback when no loader matches, dynamic param extraction,
  malformed JSON hard-reload, prefetch JSON path, prefetch fallback.
- Ports from Next.js: `getDataHref` in
  `.nextjs-ref/packages/next/src/client/page-loader.ts:154`,
  `fetchNextData` in
  `.nextjs-ref/packages/next/src/shared/lib/router/router.ts:498`,
  `getAssetPathFromRoute` in
  `.nextjs-ref/packages/next/src/shared/lib/router/utils/get-asset-path-from-route.ts`.

Verified: 1393 vitest tests pass across shims, pages-router,
link/navigation, prefetch, data-route, data-url, entry-templates, and
routing.

Closes #1330 follow-up.
@pkg-pr-new

pkg-pr-new Bot commented May 21, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/vinext@1412

commit: 4a900fd

@github-actions

Copy link
Copy Markdown
Contributor
Example Preview Production Original
app-router-cloudflare preview production
pages-router-cloudflare preview production
app-router-playground preview production original
realworld-api-rest preview production
nextra-docs-template preview production
benchmarks preview production
hackernews preview production original
web preview production

Knip flagged the exported `PagesPatternMatch` type as unused. It is only
referenced as the return type of `matchPagesPattern` in the same file, so
narrow it to a local type.
@james-elicx

Copy link
Copy Markdown
Member Author

/bigbonk review

@ask-bonk ask-bonk Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid PR — the split into navigateClientData / navigateClientHtml with a shared orchestrator in navigateClient is clean, the round-trip tests between buildPagesDataPath and parseNextDataPathname are exactly the right thing to do, and the parseNextDataPathname bug fix for index/-prefixed paths is a good catch.

A few things I noticed:

query on __NEXT_DATA__ is missing search params after data-path navigation

In navigateClientData, the synthesised __NEXT_DATA__ sets:

query: { ...target.params },

This only includes dynamic route params. After navigating to /about?foo=bar, window.__NEXT_DATA__.query will be {} instead of { foo: "bar" }.

The user-facing useRouter().query happens to still work because getPathnameAndQuery() independently reads window.location.search and merges. But any library or user code that reads window.__NEXT_DATA__.query directly (which is a public API surface in Next.js) will see stale/incomplete data. Next.js' __NEXT_DATA__.query always contains both dynamic params and search params merged.

target.search is already available on the resolved target — just needs to be parsed and spread into query alongside the params. Something like:

const searchQuery: Record<string, string | string[]> = {};
const searchParams = new URLSearchParams(target.search);
for (const [key, value] of searchParams) {
  const existing = searchQuery[key];
  if (existing !== undefined) {
    searchQuery[key] = Array.isArray(existing) ? [...existing, value] : [existing, value];
  } else {
    searchQuery[key] = value;
  }
}
// ...
query: { ...searchQuery, ...target.params },

Dynamic params should win over search params (spread order), matching Next.js.

gSSP redirect returns HTTP 3xx, not soft-redirect — x-nextjs-redirect check is dead code

navigateClientData checks for x-nextjs-redirect header (the soft-redirect protocol), but the server side (pages-page-data.ts resolvePagesPageData) returns raw HTTP 307/308 with a Location header for gSSP/gSP redirects, even on /_next/data requests. The browser's fetch() transparently follows 3xx redirects, so:

  1. The x-nextjs-redirect check at line ~673 never fires for server-originated redirects.
  2. After the browser follows the 3xx, the response the client sees is whatever lives at the redirect target — likely an HTML page, not a JSON payload — which will fail the res.json() parse and trigger a hard reload. So the end result is correct (user lands on the right page), but the error path is noisy/indirect.

Next.js handles this by returning 200 with x-nextjs-redirect header on data requests instead of a raw 3xx. This is documented as the "soft redirect" protocol. Not a blocker for this PR since the current behavior is functionally correct (fails to JSON, hard-reloads), but worth tracking as a follow-up to make the server emit soft-redirect responses for data requests specifically.

__VINEXT_PAGE_PATTERNS__ ordering

The comment in pages-client-entry.ts says the patterns list is exposed "so the router can sort it once (longest match first) without re-keying the loader map." But Object.keys(pageLoaders) returns insertion order from the generated code, which is whatever order pageRoutes.map(...) produces. I don't see any sorting happening — neither in the entry generator nor in matchPagesPattern (which explicitly says "Patterns are tried in patterns order — callers should pre-sort").

Is the route scanner already returning routes in priority order? If so the comment is misleading (it's not about the router sorting, it's about the scanner having already sorted). If not, there's a latent priority bug for apps with overlapping patterns like /posts/[id] and /posts/[...rest].

Minor nits

The navigateClientData and navigateClientHtml functions both have their own if (!root) guard + fallback. Since both are only called from navigateClient which already checks typeof window !== "undefined", the root check could be hoisted into navigateClient before the try block — would remove 8 lines of duplication. Not a big deal though.

Overall this is well-structured and the test coverage is thorough. The __NEXT_DATA__.query search-params gap is the main thing I'd want addressed before merge.

@ask-bonk

ask-bonk Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

Review posted on PR #1412. Here's a summary of the key findings:

  1. __NEXT_DATA__.query missing search params (should fix before merge) — navigateClientData sets query: { ...target.params } which only includes dynamic route params. Navigating to /about?foo=bar leaves window.__NEXT_DATA__.query as {}. The user-facing useRouter().query still works (because getPathnameAndQuery() reads window.location.search separately), but direct reads of window.__NEXT_DATA__.query will be wrong. Fix: parse target.search and merge into query.

  2. x-nextjs-redirect check is dead code (follow-up) — The client checks for the soft-redirect header, but the server returns raw HTTP 3xx for gSSP/gSP redirects on data requests. The browser follows the redirect transparently, the client gets HTML back, JSON parse fails, and hard-reloads — functionally correct but noisy. The fix is server-side (emit 200 + x-nextjs-redirect for data requests), which is a separate PR.

  3. __VINEXT_PAGE_PATTERNS__ ordering (clarification needed) — The comment says patterns are exposed for the router to sort, but no sorting happens. Either the route scanner already returns sorted routes (in which case the comment is misleading) or there's a latent priority bug with overlapping patterns.

  4. Minor: duplicated !root guard — Both navigateClientData and navigateClientHtml check for __VINEXT_ROOT__ independently; could be hoisted into navigateClient.

github run

…ta-path nav

bigbonk review found that navigateClientData was synthesising
__NEXT_DATA__.query from dynamic route params only, missing the URL search
params. Code reading window.__NEXT_DATA__.query directly (a public Next.js
surface) would see incomplete data after a JSON navigation, even though
useRouter().query happened to still work because getPathnameAndQuery()
re-reads window.location.search.

Parse target.search and merge it into query alongside the route params,
with route params winning on key collision — same merge order as
getPathnameAndQuery() and Next.js itself.

Also fix a misleading comment in pages-client-entry.ts: the patterns list
is already sorted by pagesRouter()/compareRoutes (static → dynamic →
catch-all), not by the router at runtime.
@james-elicx

Copy link
Copy Markdown
Member Author

Thanks for the review. Addressed in 9029611:

  • __NEXT_DATA__.query missing search paramsnavigateClientData now merges parsed search params with the dynamic route params (route params win on key collision, mirroring getPathnameAndQuery() / Next.js). Added a regression test in tests/shims.test.ts that navigates to /blog/my-post?slug=spoofed&q=hello&tag=a&tag=b and asserts query is { slug: "my-post", q: "hello", tag: ["a", "b"] }.
  • __VINEXT_PAGE_PATTERNS__ ordering — confirmed pagesRouter() already sorts via compareRoutes (static → dynamic → catch-all → optional catch-all), so Object.keys(pageLoaders) inherits that order and matchPagesPattern() can trust first-match. The comment in pages-client-entry.ts was misleading about where the sort happens — updated it to reference the build-time sort.
  • x-nextjs-redirect dead code — agree, follow-up; the current hard-reload-on-bad-JSON path is functionally correct but indirect. Tracking as out-of-scope for this PR (along with the soft-redirect-as-client-side-replace work already called out in the description).
  • if (!root) duplication — leaving as-is; the guard is in two places but each only runs once per nav, and hoisting requires both navigation paths to share a pre-check site that today doesn't exist.

/bigbonk review

@ask-bonk ask-bonk Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the fixes in 9029611:

__NEXT_DATA__.query missing search params — Fix is correct. mergeRouteParamsIntoQuery(parseQueryString(target.search), target.params) properly parses the search string into a multi-value-aware record, then spreads route params on top so they win on collision. The utility functions in utils/query.ts are clean and well-factored. The regression test covers the key case: /blog/my-post?slug=spoofed&q=hello&tag=a&tag=b correctly produces { slug: "my-post", q: "hello", tag: ["a", "b"] } — route param slug beats the spoofed search param, single-value and multi-value search params both work.

__VINEXT_PAGE_PATTERNS__ ordering — Confirmed: pagesRouter() calls routes.sort(compareRoutes) at lines 85 and 234 of routing/pages-router.ts, so insertion order in the generated pageLoaders is already sorted by specificity. The updated comment now accurately describes this ("which pagesRouter() has already sorted by specificity … via compareRoutes").

x-nextjs-redirect dead code and !root duplication — fine as follow-ups / leave-as-is respectively.

LGTM.

@ask-bonk

ask-bonk Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

Review posted. All three addressed items check out:

  1. Search params fixmergeRouteParamsIntoQuery(parseQueryString(target.search), target.params) correctly merges search params into __NEXT_DATA__.query with route params winning on collision. The regression test passes and covers single-value, multi-value, and spoofed-key scenarios.

  2. Comment update — The __VINEXT_PAGE_PATTERNS__ comment now accurately attributes the sort to pagesRouter()'s compareRoutes call at build time, rather than implying the router sorts at runtime.

  3. Deferred itemsx-nextjs-redirect dead code and !root duplication are reasonable to track as follow-ups / leave as-is.

Approved the PR.

github run

Adds a Playwright assertion that a Link click on a hydrated Pages Router
app on Cloudflare Workers fires a request to /_next/data/<buildId>/<page>.json
and does NOT trigger a full HTML reload. If the build pipeline ever stops
exposing __VINEXT_PAGE_LOADERS__ (or the loader map mis-keys), every other
navigation test still passes — they all assert the URL changed and the new
page rendered, neither of which fails when navigateClient silently falls
back to the HTML path. This test fails loudly in that scenario.
@james-elicx

Copy link
Copy Markdown
Member Author

Added an E2E regression guard in 5529ccc (tests/e2e/cloudflare-pages-router/navigation.spec.ts). Unit tests covered the navigateClientData logic, but none of the existing browser-level tests asserted that a Link click on a hydrated app actually hits /_next/data/<buildId>/<page>.json — they only checked the URL changed and the new page rendered, both of which still pass if the loader manifest silently regresses and we fall back to the HTML path. The new test fails loudly in that scenario by sniffing the network: ≥1 request to the JSON endpoint, 0 document-type requests to /ssr.

/bigbonk review

@ask-bonk ask-bonk Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean, well-structured PR. The split into navigateClientData / navigateClientHtml with a shared orchestrator in navigateClient reads clearly, the round-trip test suite between buildPagesDataPath and parseNextDataPathname is exactly the right thing for a wire-format contract, and the index/-prefix parsing fix in pages-data-route.ts:83 is a good catch.

E2E regression guard (5529ccc)

The new Playwright test at tests/e2e/cloudflare-pages-router/navigation.spec.ts:61 is the right shape: sniff the network for /_next/data/<buildId>/ssr.json requests and assert zero document-type requests to /ssr. This catches the exact failure mode where the loader manifest silently regresses and navigation falls back to HTML — something every other existing test would miss since they only assert on URL change + rendered content.

One thing worth noting: the test waits for __VINEXT_HYDRATED_AT (line 64) which is set at the end of hydrate() in the client entry (line 145 of pages-client-entry.ts), after __VINEXT_PAGE_LOADERS__ is already assigned (line 90). So the hydration gate is correct — by the time the test proceeds, the loader manifest is guaranteed to be populated.

Items verified from prior review rounds

  • Search params merge (9029611): mergeRouteParamsIntoQuery(parseQueryString(target.search), target.params) at router.ts:766 correctly parses multi-value search params via addQueryParam and spreads route params on top. The regression test for /blog/my-post?slug=spoofed&q=hello&tag=a&tag=b covers the key collision case.

  • Pattern ordering comment updated in pages-client-entry.ts:87-89 now correctly attributes the sort to pagesRouter()'s compareRoutes at build time.

Minor observations (no action needed)

  1. parseQueryString accepts a full URL, not just search stringparseQueryString(target.search) at router.ts:766 passes a string like ?foo=bar, and the function splits on ? to extract the query portion. This works but is slightly redundant since target.search already starts with ?. Not a bug — URLSearchParams("foo=bar") and URLSearchParams("?foo=bar".split("?")[1]) produce the same result — just noting the indirection.

  2. Prefetch duplication between link.tsx and router.tslink.tsx:259-281 and router.ts:1209-1231 both independently resolve the data target and build prefetch links. The code is structurally identical. Not a blocker (the link.tsx path runs during component render and the router.ts path runs from Router.prefetch()), but if the prefetch shape ever changes, both sites need updating.

LGTM. The test coverage is thorough across unit and E2E, the failure modes all funnel to hard-reload correctly, and the addressed review items check out.

@ask-bonk

ask-bonk Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

Review posted and PR approved. Here's a summary of the review:

Approved — the E2E regression guard in 5529ccc is the right shape. It sniffs the network for /_next/data/<buildId>/ssr.json requests and asserts zero document-type requests to /ssr, catching the exact failure mode where the loader manifest silently regresses. The hydration gate (__VINEXT_HYDRATED_AT) is correctly ordered after __VINEXT_PAGE_LOADERS__ is assigned.

Verified the fixes from prior review rounds (search params merge, pattern ordering comment). Noted two minor observations — parseQueryString receives a search string that it redundantly re-splits on ?, and the prefetch logic is duplicated between link.tsx and router.ts — neither requiring action.

github run

…sniffing

The original assertion 'no document-resource request to /ssr' caught false
positives: <link rel="prefetch" as="document"> can fire on initial paint
for unrelated reasons, and Chromium does not always classify prefetch hints
in a stable way. Swap to checking that __VINEXT_HYDRATED_AT survives the
click — a document reload would reset it, an in-place re-render preserves it.
Combined with the data-URL network assertion, this remains a tight guard on
the JSON nav path without the flake.
Timestamps are timing-related signals; swap to a window-scoped sentinel
that a document reload would wipe and an in-place re-render preserves.
Same guarantee, no flake.
…xample

The Node prod-server normalizes /_next/data/<buildId>/<page>.json before
middleware and flags it as a data request so renderPage emits the JSON
envelope. The Cloudflare worker example never did — data requests hit
matchPageRoute which can't match /_next/data/... and returned 404, so
navigateClientData saw the 404 and triggered a hard navigation. The
client-side JSON nav path was effectively dead on Workers.

Mirror the prod-server normalization in the worker: detect the data path
before middleware, validate the buildId, strip it to the page path, and
pass { isDataReq: true } through to renderPage.

Export ./server/pages-data-route from the vinext package so user-written
workers can import the helpers.
The Node prod-server normalized /_next/data/<buildId>/<page>.json before
middleware so renderPage could emit the JSON envelope. Cloudflare Workers
go through a user-written worker entry that doesn't (and shouldn't) know
about the data endpoint protocol, so data requests were reaching
matchPageRoute as /_next/data/... URLs, returning 404, and the client
fell back to a hard navigation — defeating the whole JSON-path
optimisation on Workers.

Move the detection into the generated server entry so it works for any
caller. runMiddleware now normalizes the URL for middleware matcher and
signals via result.rewriteUrl, and renderPage auto-detects data requests
from request.url so the JSON envelope fires whether or not middleware
rewrote the URL. Mismatched buildId short-circuits to a JSON 404 in
both paths so stale clients hard-navigate onto the new build.
# Conflicts:
#	packages/vinext/src/entries/pages-server-entry.ts
@james-elicx

Copy link
Copy Markdown
Member Author

Moved the _next/data request handling out of the example worker and into the generated server entry per feedback — the user-written worker template stays thin. runMiddleware and renderPage now auto-detect data URLs via a shared __normalizePagesDataRequest helper in the virtual entry; middleware sees the normalized page path, the worker sees the normalized URL via result.rewriteUrl, and renderPage falls back to detecting from request.url when there's no middleware. Buildid mismatches short-circuit to a JSON 404 in both paths.

Merged latest main and resolved the conflict in pages-server-entry.ts (kept both my __dataNorm.request change and main's new trailingSlash arg to __runGeneratedMiddleware).

All 37 CI checks green including the new E2E regression guard.

/bigbonk review

@ask-bonk ask-bonk Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thorough review of the full PR after the latest rebase and the move of _next/data normalization into the generated server entry.

Architecture

The split into navigateClientData / navigateClientHtml with a shared orchestrator in navigateClient is well-structured. The resolvePagesDataNavigationTarget function cleanly encapsulates the "can we use the fast path?" decision, and returning null to fall through to HTML is the right escape hatch. The __normalizePagesDataRequest helper in the generated server entry correctly centralizes data URL normalization so user-written worker entries stay thin — verified that all three server contexts (prod-server, dev-server, Cloudflare worker/deploy.ts) handle _next/data requests correctly with no parity gap.

The round-trip test contract between buildPagesDataPath (client) and parseNextDataPathname (server) is exactly the right thing to do for a wire format, and it caught the index/-prefix bug in pages-data-route.ts:83.

The E2E regression guard (navigation.spec.ts) is well-designed — the sentinel + network sniff approach catches the exact failure mode where the loader manifest silently regresses and the client falls through to HTML without any visible breakage.

Issues found

1. gSSP/gSP redirect on data requests returns raw 3xx instead of JSON body (follow-up, not blocking)

resolvePagesPageData returns a raw HTTP 307/308 with Location header for gSSP/gSP redirects, even on _next/data requests. The browser's fetch() transparently follows the 3xx per the Fetch spec, so the client never sees the redirect — it gets whatever the redirect destination serves (likely HTML), the res.json() parse fails, and the client hard-reloads. The end result is correct but indirect and noisy.

Next.js handles this differently: on data requests, it returns HTTP 200 with { __N_REDIRECT: "/destination", __N_REDIRECT_STATUS: 307 } in the JSON body. The client router interprets those fields and performs a client-side navigation to the redirect target, avoiding a full page reload.

The x-nextjs-redirect check at router.ts:684 is only reachable for middleware redirects (which do emit the header). For gSSP/gSP redirects, it's dead code on the current server implementation.

Already called out as out-of-scope in the PR description and prior reviews. Noting for completeness — the server-side fix (emit 200 + JSON body for data-request redirects) is the real fix and belongs in a separate PR.

2. Locale transitions silently skip the data path (latent, not blocking)

resolvePagesDataNavigationTarget calls matchPagesPattern(pagePath, patterns) where pagePath still includes the locale prefix (e.g., /fr/about). Route patterns in __VINEXT_PAGE_PATTERNS__ are locale-unaware (/about), so the match always fails for non-default-locale URLs. resolvePagesDataNavigationTarget returns null and the code correctly falls through to the HTML path.

This means locale transitions (and all non-default-locale navigations) never benefit from the JSON data path — they always take the slower HTML round trip. Not a correctness bug (the fallback works), but a performance gap worth knowing about.

If this is ever fixed (stripping locale prefix before pattern matching), navigateClientData also needs to be updated: it currently inherits locale fields from ...prev (the previous __NEXT_DATA__), which would be stale after a locale transition. The data JSON response only contains { pageProps } with no locale metadata.

3. Prefetch duplication between link.tsx and router.ts (nit)

The prefetch logic in link.tsx:309-360 and router.ts:1234-1267 is structurally identical — both resolve the data target, inject <link rel="prefetch" as="fetch">, and invoke the loader thunk. If the prefetch shape changes (e.g., adding Accept header hints, or switching from <link> to fetch() with keepalive), both sites need updating. Not blocking, but worth extracting into a shared helper at some point.

Verified from prior rounds

  • Search params merge (9029611): mergeRouteParamsIntoQuery(parseQueryString(target.search), target.params) correctly merges search params with route params winning on collision. Regression test covers the key case.
  • Pattern ordering comment updated to reference build-time sort via compareRoutes.
  • Server parity: __normalizePagesDataRequest in the generated entry handles the Cloudflare Workers path (deploy.ts) correctly. prod-server.ts does its own normalization before calling the entry. dev-server.ts (index.ts) does its own normalization before the SSR handler. No gaps.
  • E2E guard (5529ccc): hydration gate via __VINEXT_HYDRATED_AT is correctly ordered after __VINEXT_PAGE_LOADERS__ assignment. Window sentinel approach is more robust than resource-type sniffing.

LGTM — the items above are tracked follow-ups or latent issues, not blockers. The core navigation/prefetch/normalization logic is correct and well-tested.

@ask-bonk

ask-bonk Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

Review posted and PR approved. Here's a summary of the key findings:

Architecture: Clean and correct. The navigateClientData/navigateClientHtml split with navigateClient as orchestrator is well-structured. The __normalizePagesDataRequest helper in the generated server entry correctly centralizes data URL normalization across all three server contexts (prod, dev, Cloudflare Workers) with no parity gaps. The round-trip test contract between client URL builder and server parser is the right approach.

Three items noted (none blocking):

  1. gSSP/gSP redirect on data requests — Server returns raw 3xx instead of Next.js's JSON body protocol ({ __N_REDIRECT, __N_REDIRECT_STATUS }). Works accidentally via hard-reload-on-bad-JSON but is indirect. The x-nextjs-redirect check in the client is dead code for gSSP/gSP redirects. Already tracked as out-of-scope.

  2. Locale transitions skip the data pathmatchPagesPattern receives locale-prefixed URLs (/fr/about) but route patterns are locale-unaware (/about), so the match fails and falls through to the HTML path. Not a correctness bug (fallback works), but locale navigations never get the JSON fast path. If fixed later, navigateClientData also needs locale field updates (currently inherits stale locale from ...prev).

  3. Prefetch duplicationlink.tsx and router.ts have structurally identical prefetch logic. Worth extracting to a shared helper eventually.

github run

…nd Router

Extract `resolvePagesDataNavigationTarget` + `prefetchPagesData` into
`shims/internal/pages-data-target.ts` so the Link prefetch path and the
Router (`navigateClient` + `Router.prefetch`) share one decision helper
instead of maintaining two structurally identical copies.

Also strip the locale prefix before matching `__VINEXT_PAGE_PATTERNS__`
so locale transitions (e.g. `/fr/about`) hit the JSON data fast path
instead of falling through to the slower HTML round-trip — route
patterns are locale-unaware (`/about`), but the browser URL for a
locale-prefixed page includes the locale. The data URL itself keeps
the locale prefix because the server uses it to pick locale-specific
gSSP data.

When the data path is used and the URL implies a different locale than
the current `__NEXT_DATA__.locale`, refresh the locale fields on the
synthesised `__NEXT_DATA__` (the JSON envelope itself only carries
`pageProps`, so the locale must be derived from the URL). Without this,
`useRouter().locale` stays stale after a client-side locale transition.
@james-elicx

Copy link
Copy Markdown
Member Author

Addressed two of the three review nits in 0a2760a:

  • Prefetch duplication — extracted resolvePagesDataNavigationTarget + prefetchPagesData into shims/internal/pages-data-target.ts. Both Router.prefetch() and <Link> prefetch now go through the shared helper; the navigateClient hot path uses the same resolver, so there is only one decision site for "can we use the JSON fast path?".
  • Locale transitions skip the data path — the target resolver now strips the locale prefix before matching __VINEXT_PAGE_PATTERNS__, so navigations like /fr/about resolve the locale-unaware /about loader and take the JSON fast path. The data URL keeps the locale prefix so the server still picks locale-specific gSSP data. When the URL implies a different locale than the current __NEXT_DATA__.locale, the synthesised __NEXT_DATA__ now refreshes its locale field (the JSON envelope only carries pageProps, so the locale has to come from the URL) — without this, useRouter().locale stayed stale after a client-side locale transition. Added two regression tests in tests/shims.test.ts: locale-prefixed nav (/fr/about/_next/data/<id>/fr/about.json, page /about, locale "fr") and default-locale-restore (Router.push("/about", undefined, { locale: "en" }) from /fr drops the locale prefix and resets locale back to "en").
  • gSSP/gSP redirect protocol — leaving as-is. The PR description and the prior reviews already track this as out-of-scope; the real fix is server-side (emit 200 + { __N_REDIRECT, __N_REDIRECT_STATUS } JSON for data-request redirects instead of raw 3xx), which belongs in a separate PR.

All 4738 unit tests pass locally including the new locale regression tests.

/bigbonk review

@ask-bonk ask-bonk Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the refactor in 0a2760a. All three items check out.

Prefetch deduplication

Clean extraction. resolvePagesDataNavigationTarget + prefetchPagesData in shims/internal/pages-data-target.ts is the right factoring — the module lives under shims/internal/ so neither link.tsx nor router.ts pulls in the other at init time (no circular imports, no SSR side effects from the Router singleton). Both the Link prefetch path (line 324 of link.tsx) and the Router prefetch/navigation path (lines 930 and 1193 of router.ts) now go through the same resolver, so there is exactly one decision site for "can we use the JSON fast path?".

The link.tsx diff is a clean 40-line deletion with 6 lines replacing it — the inline try/catch + URL parse + stripBasePath + matchPagesPattern + buildPagesDataHref + createElement sequence is gone, replaced by two function calls. Good.

Locale transitions

The locale-stripping logic at pages-data-target.ts:84-87 is correct:

const locale = getLocalePathPrefix(pagePath, window.__VINEXT_LOCALES__);
const pathForMatch = locale ? pagePath.slice(locale.length + 1) || "/" : pagePath;
  • getLocalePathPrefix returns the matched locale string (e.g. "fr") or undefined. The + 1 accounts for the leading / in the /<locale> segment. The || "/" handles the edge case where the URL is just /<locale> (e.g. /fr) which should resolve to the root page.
  • pagePath (with locale prefix intact) is passed through to buildPagesDataHref, so the data URL keeps the prefix (e.g. /_next/data/<id>/fr/about.json), which is what the server expects for locale-specific gSSP data. Correct.

The locale refresh in navigateClientData (router.ts:729-732) handles both directions:

const hasI18n = (window.__VINEXT_LOCALES__?.length ?? 0) > 0;
const nextLocale = hasI18n
  ? (target.locale ?? window.__VINEXT_DEFAULT_LOCALE__)
  : (prev as VinextNextData | undefined)?.locale;
  • Locale transition (/en/about/fr/about): target.locale is "fr", so nextLocale is "fr". The spread ...(nextLocale !== undefined ? { locale: nextLocale } : {}) overrides prev.locale.
  • Back to default (/fr/about/about): target.locale is undefined (no prefix on an unprefixed URL), so nextLocale falls back to window.__VINEXT_DEFAULT_LOCALE__ (e.g. "en"). Correct — without this, locale would stay stale at "fr".
  • No i18n configured: hasI18n is false, so nextLocale inherits from prev.locale unchanged. Also correct — no locale to refresh.

Then applyVinextLocaleGlobals(window, nextData) writes the new locale to window.__VINEXT_LOCALE__, so useRouter().locale (which reads from that global via the Router.locale getter at line 1497) reflects the transition immediately.

Tests

Both regression tests are well-structured:

  1. Locale-prefixed nav (strips the locale prefix): navigates to /fr/about, asserts the fetch URL is /_next/data/test-build/fr/about.json (locale prefix preserved in data URL), the matched page is /about (locale stripped for pattern match), and both __NEXT_DATA__.locale and window.__VINEXT_LOCALE__ flip to "fr".

  2. Default-locale restore (restores the default locale): starts on /fr, navigates to /about with { locale: "en" }, asserts the data URL has no locale prefix (/_next/data/test-build/about.json), and locale globals reset to "en".

Both tests properly clean up globalThis.window and globalThis.fetch in finally blocks, and call vi.resetModules() to avoid cross-test pollution from the router module's singleton state.

LGTM.

@ask-bonk

ask-bonk Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

Review posted and PR approved. Here's a summary of what I verified:

Prefetch deduplication -- Clean extraction of resolvePagesDataNavigationTarget + prefetchPagesData into shims/internal/pages-data-target.ts. Both link.tsx and router.ts now share a single decision site for the JSON fast path. The shims/internal/ placement avoids circular imports and SSR-side router-init side effects. The link.tsx diff removes ~40 lines of inline URL parsing/matching/prefetching and replaces them with two function calls.

Locale transitions -- The locale-stripping logic correctly peels off the /<locale> prefix before pattern matching while preserving it in the data URL for server-side locale-specific gSSP resolution. The locale refresh in navigateClientData handles all three cases: locale transition (override with target locale), back-to-default (fallback to __VINEXT_DEFAULT_LOCALE__), and no-i18n (inherit from previous __NEXT_DATA__). The applyVinextLocaleGlobals call ensures window.__VINEXT_LOCALE__ stays in sync so useRouter().locale reflects the change immediately.

Tests -- Both regression tests are well-structured with proper cleanup and cover the key scenarios (locale-prefixed navigation and default-locale restore). All 11 _next/data tests and all 25 pages-data-url tests pass.

github run

@james-elicx
james-elicx marked this pull request as ready for review May 22, 2026 14:46
@james-elicx
james-elicx merged commit d6bf641 into main May 22, 2026
36 checks passed
@james-elicx
james-elicx deleted the james/_next/data-client branch May 22, 2026 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant