perf(router): render/caching pipeline optimizations - #658

Problem

Three independent audits of the render/caching pipeline (server request path, cache layer, client navigation) surfaced redundant hot-path work. The most consumer-observable costs: every document request resolved the route twice (classification + match), every "use cache" call — including fresh hits — paid the React Flight reply encoder just to build its key, N concurrent misses on one key after a tag invalidation each re-ran the cached function and re-wrote the store, the Vercel store serialized every large envelope twice per write and rewrote the whole payload on every stale read, and every production client navigation eagerly built O(segments) debug strings that were thrown away unread.

What a consumer sees differently

Nothing to adopt — no API changes are required. Observable effects:

• Lower TTFB on document loads (route resolution, manifest restore, and URL parsing each happen once).
• "use cache" under concurrent traffic: a cold or just-invalidated popular key executes once, not N times.
• MemorySegmentCacheStore no longer grows without bound: new maxEntries option, default 1000 per family (segments / responses / items / shells), FIFO eviction with tag-index cleanup.

// Before: unbounded
new MemorySegmentCacheStore({ defaults: { ttl: 60, swr: 300 } });

// After: bounded by default; tune or raise per deployment
new MemorySegmentCacheStore({ defaults: { ttl: 60, swr: 300 }, maxEntries: 5000 });

End-to-end example

// api/get-product.ts
export async function getProduct(ctx: HandlerContext, id: string) {
  "use cache";
  return db.products.find(id);
}

Before: every call — hit or miss — ran encodeReply([ctxKeyParts, id]) to build the key; 20 concurrent first requests for product 42 ran db.products.find 20 times and wrote the store 20 times. After: the key is a stable JSON string (use-cache:<id>:j:[...]), one leader runs the query, 19 followers deserialize the leader's serialized result into their own responses, and the store is written once.

Semantics table

Code changes

45 files, +2563/−364. Server: request-classification.ts (resolve under the render's isSSR flag), match-api.ts (full-path snapshot reuse mirroring the partial path), manifest.ts (alias cached Map; reassign-not-clear safety pin), middleware.ts/match-handlers.ts (clean URL from request context), match-result.ts (fused passes). Cache: cache-runtime.ts (in-flight Map<key, Promise<CacheEnvelope>> + jsonSafeKeyPart fast key), memory-segment-store.ts (maxEntries), vercel-cache-store.ts (single-serialize + claimRevalidation lock key), document-cache.ts, segment-codec.ts. Client: debug-flag guards across segment-reconciler/partial-update/etc., navigation-store.ts (container-only cloneHandleData + owner-guarded single-scan write), prefetch/cache.ts (expiry watermark), event-controller.ts (memoized referentially-stable snapshots), Link.tsx, navigation-client.ts/prefetch/fetch.ts (single cookie parse).

Deliberately deferred (design passes needed, tracked separately): Flight-string hit splicing (serve cached segments without deserialize→reserialize), renderSegments per-segment element reuse, temporalLazyRefs streaming.

Test plan

• Unit: 4867 router tests + all consumer-app suites + RSC configs (pnpm run test:unit:all) green. New suites: match-api-classified-reuse (snapshot reuse/fallback matrix), manifest-cache poisoning regression (proven red-before-green against in-place clear()), cache-runtime-inflight (leader/follower/rejection/tags+handles), cache-runtime-fastkey (determinism, fallback, tainted-ctx keys), memory-store-eviction, segment-codec (multibyte split), Vercel store legacy-envelope + lock tests, clone-handle-data-cow, prefetch-cache-watermark, event-controller-snapshot, rango-state read-once tests.
• Semantic matrix: 64 passed (mandatory — middleware.ts touched).
• Typecheck (15/15 projects), oxlint --deny-warnings, oxfmt, e2e-bucketing, .gen.ts clean.
• CI full e2e matrix (dev + production across test-app, cloudflare-basic, vercel apps) — existing suites pin the affected contracts in both modes (blog-cache/cache-tag read x-document-cache-status in production; use-cache*, cache*, prefetch and handle-streaming suites cover the touched paths).

Notes

• The "use cache" key-namespace change means one cold miss per JSON-safe-args entry on first deploy; TTL windows make this a non-event, but flagging it.
• Vercel stale-read behavior intentionally changed from "every stale reader revalidates (dampened by re-stamp)" to "one claimant revalidates, others serve stale": fewer duplicate revalidations, no payload rewrite; the lock is best-effort (no CAS) exactly like the mechanism it replaces.
• maxEntries default (1000) is the only default-behavior change; deployments intentionally holding more per family should raise it.