Skip to content

Commit 16eab3a

Browse files
authored
[backport] Fix: Optimistic routing bugs leading to repeated prefetch loops (#97325)
Backports #97128 ("Fix: Optimistic routing bugs leading to repeated prefetch loops") to `next-16-3`. Fixes an infinite prefetch loop in optimistic routing. A proxy that rewrites every URL to inject a leading path segment — i18n with the default locale hidden from the URL, so `/one/two` is served as `/en/one/two` — combined with a fully dynamic target route like `/[locale]/[...pages]`, produces a prefetch that never settles. When a prefetch response came back, the client never checked whether the rendered tree matched the one it had requested. When the params baked into the request were wrong because of the rewrite, the task kept re-attempting to fulfill the missing data and looped forever. The prefetch path now records a dynamic-rewrite marker on the local route definition when the response's tree diverges from the requested one, which disables further optimistic resolution of that route and invalidates the entries derived from it. This is the same strategy normal navigation responses already used, now applied to the prefetch path. A second, related bug is also fixed: parallel routes with conflicting dynamic params at the same level (`@modal/[...catchAll]` next to `[username]`) can't be distinguished by the traversal algorithm, which assumes each segment resolves independently without inspecting sibling branches. When that scenario is detected, optimistic routing is now disabled for the conflicting route rather than binding one branch's URL parts to another branch's params. ### Verification on this branch Cherry-picked from `5942b37a42`. Three source files conflicted, all because `next-16-3` predates the post-16.3.0 refactor that split the client's response decoder into its own module: - `segment-cache/decode-server-response.ts` — this file does not exist on this branch; it was extracted out of `navigation.ts` after 16.3.0. Its change (tracking whether the decoded tree's segment identities diverge from the base request tree) was ported into `navigation.ts`, where the equivalent logic still lives. Canary decodes a compact transport tree and compares segment identities inline during the walk; this branch instead merges the response patch onto the base `FlightRouterState` before converting it to a `RouteTree`. The check is therefore re-expressed as a read-only walk in `convertServerPatchToFullTree` that compares each server patch against the base tree it is applied to, following canary's descent (`convertServerPatchToFullTreeImpl`) exactly and using the same comparison rules: segment identity via `matchSegment`, page segments matched modulo embedded search params, server-filled `__DEFAULT__` segments ignored, and inactive parallel-route branches (those carrying a refresh state in the base) skipped. The result is threaded out through `NavigationSeed.treeDivergedFromBase`. - `segment-cache/cache.ts` — the consuming block in `fetchSegmentPrefetchesUsingDynamicRequest` (mark-as-dynamic-rewrite, invalidate derived entries, reject with immediate expiry) is byte-identical to canary. The surrounding conflicts were only incidental: the `RouteTreeAccumulator` type gaining `treeDivergedFromBase`, and canary's transport-decoder call vs. this branch's `convertRootFlightRouterStateToRouteTree`. Resolved by keeping this branch's calls and adding the new field to the accumulators. Canary's inline `NavigationSeed` construction in `writeDynamicTreeResponseIntoCache` does not apply here because that function delegates to `convertServerPatchToFullTree`, which now carries the field. - `router-reducer/create-initial-router-state.ts` — a one-line accumulator initializer; kept this branch's `convertRootFlightRouterStateToRouteTree` call and added `treeDivergedFromBase: false`. The other touched source files applied cleanly and are byte-identical to canary: `segment-cache/optimistic-routes.ts` (the parallel-route conflict detection) and `route-params.ts` (exporting `canonicalizeURLPart`), along with `ppr-navigations.ts`, the small cleanly-applied hunk in `navigation.ts`, and every test fixture. `tsc` passes on the `next` package for all files touched here. Behavioral verification is left to CI on this branch; the change carries the `proxy-prefix-rewrite-prefetch-loop` and `modal-catchall-sibling-dynamic-prefetch-loop` end-to-end suites plus additions to the `optimistic-routing` suite. Opened as a draft because the `decode-server-response.ts` → `navigation.ts` port is a manual re-implementation across the refactor boundary rather than a mechanical cherry-pick. The divergence walk in `convertServerPatchToFullTree` is the part most worth a close review. <!-- NEXT_JS_LLM -->
1 parent 75548c9 commit 16eab3a

25 files changed

Lines changed: 951 additions & 14 deletions

File tree

‎packages/next/src/client/components/router-reducer/create-initial-router-state.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,7 @@ export function createInitialRouterState({
8383
// these trees with InliningHintsStale, which causes the route cache entry
8484
// to be immediately expired. The next prefetch will re-fetch the tree with
8585
// correct hints from the /_tree response.
86-
const acc = { metadataVaryPath: null }
86+
const acc = { metadataVaryPath: null, treeDivergedFromBase: false }
8787
const initialRouteTree = convertRootFlightRouterStateToRouteTree(
8888
initialTree,
8989
initialRenderedSearch as NormalizedSearch,

‎packages/next/src/client/components/router-reducer/ppr-navigations.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -932,7 +932,7 @@ function reuseActiveSegmentInDefaultSlot(
932932
reusedRenderedSearch = oldRootRefreshState.renderedSearch
933933
}
934934

935-
const acc = { metadataVaryPath: null }
935+
const acc = { metadataVaryPath: null, treeDivergedFromBase: false }
936936
const reusedRouteTree = convertReusedFlightRouterStateToRouteTree(
937937
parentRouteTree,
938938
parallelRouteKey,

‎packages/next/src/client/components/segment-cache/cache.ts‎

Lines changed: 44 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1534,6 +1534,9 @@ function rejectSegmentCacheEntry(
15341534

15351535
type RouteTreeAccumulator = {
15361536
metadataVaryPath: PageVaryPath | null
1537+
// Whether the decoded tree's segment identities diverged from the base
1538+
// tree it was overlaid onto. See NavigationSeed.treeDivergedFromBase.
1539+
treeDivergedFromBase: boolean
15371540
}
15381541

15391542
function convertRootTreePrefetchToRouteTree(
@@ -2097,7 +2100,10 @@ export async function fetchRouteOnCacheMiss(
20972100
//
20982101
// During this traversal, we accumulate additional data into this
20992102
// "accumulator" object.
2100-
const acc: RouteTreeAccumulator = { metadataVaryPath: null }
2103+
const acc: RouteTreeAccumulator = {
2104+
metadataVaryPath: null,
2105+
treeDivergedFromBase: false,
2106+
}
21012107
const routeTree = convertRootTreePrefetchToRouteTree(
21022108
serverData,
21032109
renderedPathname,
@@ -3305,6 +3311,39 @@ export async function fetchSegmentPrefetchesUsingDynamicRequest(
33053311
// Not needed for prefetch responses; pass unknown to use the default.
33063312
UnknownDynamicStaleTime
33073313
)
3314+
3315+
if (
3316+
navigationSeed.treeDivergedFromBase &&
3317+
// A head-only request uses the MetadataOnlyRequestTree stub rather than
3318+
// a tree derived from the route entry, so divergence from it carries
3319+
// no signal.
3320+
// TODO: This special case goes away once convertServerPatchToFullTree
3321+
// diffs against the base RouteTree (route.tree) instead of the
3322+
// request tree.
3323+
dynamicRequestTree !== MetadataOnlyRequestTree
3324+
) {
3325+
// The server rendered a different route tree than the one we requested:
3326+
// the URL has a rewrite that behaves dynamically, so the params baked
3327+
// into the request are wrong and the server can never fulfill it. Mark
3328+
// the route entry — which doubles as the stored prediction pattern, so
3329+
// this also disables a bad prediction (see matchKnownRoute) that would
3330+
// otherwise be re-derived on every retry — and invalidate entries that
3331+
// were derived from it. This mirrors dispatchRetryDueToTreeMismatch on
3332+
// the navigation path. It can't loop: the refetched route entry is
3333+
// built from the server's response, so it only mismatches again if the
3334+
// rewrite's behavior changes again.
3335+
markRouteEntryAsDynamicRewrite(route)
3336+
invalidateRouteCacheEntries(key.nextUrl, task.treeAtTimeOfPrefetch)
3337+
// Reject with an immediate expiration instead of the usual backoff: the
3338+
// invalidation above triggers a re-prefetch, which per the above does
3339+
// not loop.
3340+
// TODO: Consider also bounding retries with a counter on the task
3341+
// object, so a prefetch that repeatedly fails to settle backs off
3342+
// regardless of the reason.
3343+
rejectSegmentEntriesIfStillPending(spawnedEntries, -1)
3344+
return null
3345+
}
3346+
33083347
// Aside from writing the data into the cache, this function also returns
33093348
// the entries that were fulfilled, so we can streamingly update their sizes
33103349
// in the LRU as more data comes in.
@@ -3407,7 +3446,10 @@ function writeDynamicTreeResponseIntoCache(
34073446
//
34083447
// During this traversal, we accumulate additional data into this
34093448
// "accumulator" object.
3410-
const acc: RouteTreeAccumulator = { metadataVaryPath: null }
3449+
const acc: RouteTreeAccumulator = {
3450+
metadataVaryPath: null,
3451+
treeDivergedFromBase: false,
3452+
}
34113453
const routeTree = convertRootFlightRouterStateToRouteTree(
34123454
flightRouterState,
34133455
renderedSearch,

‎packages/next/src/client/components/segment-cache/navigation.ts‎

Lines changed: 121 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ import type {
33
FlightRouterState,
44
FlightSegmentPath,
55
ScrollRef,
6+
Segment,
67
} from '../../../shared/lib/app-router-types'
78
import type { CacheNode } from '../../../shared/lib/app-router-types'
89
import type { HeadData } from '../../../shared/lib/app-router-types'
@@ -50,6 +51,11 @@ import { computeChangedPath } from '../router-reducer/compute-changed-path'
5051
import { isJavaScriptURLString } from '../../lib/javascript-url'
5152
import { UnknownDynamicStaleTime, computeDynamicStaleAt } from './bfcache'
5253
import { createLinkPrefetchPartialError } from '../../../shared/lib/instant-messages'
54+
import { matchSegment } from '../match-segments'
55+
import {
56+
DEFAULT_SEGMENT_KEY,
57+
PAGE_SEGMENT_KEY,
58+
} from '../../../shared/lib/segment'
5359

5460
/**
5561
* Navigate to a new URL, using the Segment Cache to construct a response.
@@ -421,6 +427,8 @@ function navigateUsingPrefetchedRouteTree(
421427
data: null,
422428
head: null,
423429
dynamicStaleAt: computeDynamicStaleAt(now, UnknownDynamicStaleTime),
430+
// Not derived from a server response; no base to diverge from.
431+
treeDivergedFromBase: false,
424432
}
425433
return navigateToKnownRoute(
426434
now,
@@ -884,6 +892,16 @@ export type NavigationSeed = {
884892
data: CacheNodeSeedData | null
885893
head: HeadData | null
886894
dynamicStaleAt: number
895+
// Whether the response rendered a segment whose identity differs from the
896+
// base tree's at the same position (inactive parallel route branches are
897+
// expected to differ and don't count). Only meaningful when the base is a
898+
// request tree derived from a cached route entry, as during a prefetch:
899+
// divergence then means the entry doesn't describe what the server renders
900+
// — the URL has a rewrite that behaves dynamically (see
901+
// fetchSegmentPrefetchesUsingDynamicRequest). During a navigation the base
902+
// is the current page's tree, so divergence carries no signal. False when
903+
// there was no base to compare against.
904+
treeDivergedFromBase: boolean
887905
}
888906

889907
export function convertServerPatchToFullTree(
@@ -912,13 +930,25 @@ export function convertServerPatchToFullTree(
912930
let baseTree: FlightRouterState = currentTree
913931
let baseData: CacheNodeSeedData | null = null
914932
let head: HeadData | null = null
933+
// Whether any patch rendered a segment whose identity differs from the base
934+
// (request) tree at the same position. See NavigationSeed.treeDivergedFromBase.
935+
// Compared against the original `currentTree`, not the progressively-merged
936+
// `baseTree`, so each patch is checked against what was actually requested.
937+
let treeDivergedFromBase = false
915938
if (flightData !== null) {
916939
for (const {
917940
segmentPath,
918941
tree: treePatch,
919942
seedData: dataPatch,
920943
head: headPatch,
921944
} of flightData) {
945+
if (!treeDivergedFromBase) {
946+
treeDivergedFromBase = didServerPatchDivergeFromBase(
947+
currentTree,
948+
segmentPath,
949+
treePatch
950+
)
951+
}
922952
const result = convertServerPatchToFullTreeImpl(
923953
baseTree,
924954
baseData,
@@ -943,7 +973,7 @@ export function convertServerPatchToFullTree(
943973
// TODO: Eventually, FlightRouterState will evolve to being a transport format
944974
// only. The RouteTree type will become the main type used for dealing with
945975
// routes on the client, and we'll store it in the state directly.
946-
const acc = { metadataVaryPath: null }
976+
const acc = { metadataVaryPath: null, treeDivergedFromBase: false }
947977
const routeTree = convertRootFlightRouterStateToRouteTree(
948978
finalFlightRouterState,
949979
renderedSearch as NormalizedSearch,
@@ -957,7 +987,97 @@ export function convertServerPatchToFullTree(
957987
renderedSearch,
958988
head,
959989
dynamicStaleAt: computeDynamicStaleAt(now, dynamicStaleTimeSeconds),
990+
treeDivergedFromBase,
991+
}
992+
}
993+
994+
// Whether a server patch's rendered tree diverges in segment identity from the
995+
// base (request) tree it was applied to. Mirrors the comparison performed while
996+
// decoding a server response in newer versions of the client (see
997+
// NavigationSeed.treeDivergedFromBase). Divergence means the server rendered a
998+
// different route than the one we requested — e.g. a URL rewrite that behaves
999+
// dynamically — so a prefetch built from the base tree can never be fulfilled.
1000+
//
1001+
// The base-tree descent mirrors convertServerPatchToFullTreeImpl: segmentPath
1002+
// is a repeating [parallelRouteKey, segment, ...] pattern; segmentPath[i] keys
1003+
// into the children, segmentPath[i + 1] is the server's segment at that
1004+
// position.
1005+
function didServerPatchDivergeFromBase(
1006+
currentTree: FlightRouterState,
1007+
segmentPath: FlightSegmentPath,
1008+
treePatch: FlightRouterState
1009+
): boolean {
1010+
let baseNode: FlightRouterState = currentTree
1011+
for (let i = 0; i + 1 < segmentPath.length; i += 2) {
1012+
const parallelRouteKey: string = segmentPath[i]
1013+
const serverSegment: Segment = segmentPath[i + 1]
1014+
const childBase: FlightRouterState | undefined =
1015+
baseNode[1][parallelRouteKey]
1016+
if (childBase === undefined) {
1017+
// The base tree doesn't have this branch. Unless the server merely
1018+
// filled it with a default, the trees have different structures.
1019+
return serverSegment !== DEFAULT_SEGMENT_KEY
1020+
}
1021+
if (segmentIdentityDivergesFromBase(serverSegment, childBase[0])) {
1022+
return true
1023+
}
1024+
baseNode = childBase
1025+
}
1026+
return detectTreeDivergenceFromBase(treePatch, baseNode)
1027+
}
1028+
1029+
// Recursively compares a server-rendered subtree against the base subtree at
1030+
// the same position. Inactive parallel route branches — which carry a refresh
1031+
// state in the base — are expected to differ and are skipped.
1032+
function detectTreeDivergenceFromBase(
1033+
serverNode: FlightRouterState,
1034+
baseNode: FlightRouterState
1035+
): boolean {
1036+
if (segmentIdentityDivergesFromBase(serverNode[0], baseNode[0])) {
1037+
return true
1038+
}
1039+
const serverChildren = serverNode[1]
1040+
const baseChildren = baseNode[1]
1041+
for (const parallelRouteKey in serverChildren) {
1042+
const childServer = serverChildren[parallelRouteKey]
1043+
const childBase = baseChildren[parallelRouteKey]
1044+
if (childBase === undefined) {
1045+
// A slot the base tree doesn't have. Unless the server merely filled it
1046+
// with a default, the trees have different structures.
1047+
if (childServer[0] !== DEFAULT_SEGMENT_KEY) {
1048+
return true
1049+
}
1050+
} else if ((childBase[2] ?? null) !== null) {
1051+
// The base branch carries a refresh state: an inactive parallel route
1052+
// reused from a different route (e.g. a "default" slot). The server's
1053+
// answer is expected to differ, so skip the branch.
1054+
} else if (detectTreeDivergenceFromBase(childServer, childBase)) {
1055+
return true
1056+
}
1057+
}
1058+
return false
1059+
}
1060+
1061+
// Whether two segments at the same position claim different identities. Page
1062+
// segments match modulo embedded search params (validated separately, see
1063+
// getRenderedSearch), and a default filled in by the server is not a claim
1064+
// about the position's identity.
1065+
function segmentIdentityDivergesFromBase(
1066+
serverSegment: Segment,
1067+
baseSegment: Segment
1068+
): boolean {
1069+
if (
1070+
typeof serverSegment === 'string' &&
1071+
typeof baseSegment === 'string' &&
1072+
serverSegment.startsWith(PAGE_SEGMENT_KEY) &&
1073+
baseSegment.startsWith(PAGE_SEGMENT_KEY)
1074+
) {
1075+
return false
1076+
}
1077+
if (serverSegment === DEFAULT_SEGMENT_KEY) {
1078+
return false
9601079
}
1080+
return !matchSegment(baseSegment, serverSegment)
9611081
}
9621082

9631083
function convertServerPatchToFullTreeImpl(

0 commit comments

Comments
 (0)