Skip to content

Commit 34dd2c7

Browse files
authored
feat(ppr): shell fast path — HIT tails replay the handler layer through the cache() hit lane (#695)
Only loaders are dynamic: an eligible shell HIT's tail match hits an implicit doc-level segment record (recorded into the shell entry at capture, SnapshotOnlySegmentStore) and replays path/layout/parallel handlers byte-identically; middleware and DSL loaders run live. Eligibility (handlerLiveHoles): handler-scoped nested-thenable handle pushes, pushes pending at the putShell barrier, or handler-invoked loaders (#672 consumption lane) decline the fast path and keep the full tail. Route-derived cache()/cache(false) scopes always win. Capture context isolates _explicitTaggedStores so ephemeral stores never pin the handler-lifetime tag registry. Red-proven execution-matrix e2e in test-app + cloudflare-basic (dev + production): middleware+loader counters advance per HIT, handler counters frozen.
1 parent 55eec49 commit 34dd2c7

24 files changed

Lines changed: 1080 additions & 11 deletions

‎docs/design/shell-fast-path.md‎

Lines changed: 253 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,253 @@
1+
# Shell Fast Path: the shell entry as a cache() of the handler layer
2+
3+
Status: v1 IMPLEMENTED on `feat/shell-fast-path` (2026-07-05); the "v1 as
4+
built" section below is the authoritative record of what shipped and why it
5+
needed almost no new machinery. Prerequisite work (capture-time
6+
nested-thenable masking) shipped in PR #692; the instrumentation that produced
7+
the numbers below shipped in PR #691 and PR #693.
8+
9+
## Why (the measurements that motivated this)
10+
11+
On a consumer storefront (workerd, PPR homepage, SFCC + Builder.io + CIO
12+
upstreams), a shell HIT commits headers at ~25ms — and then spends its entire
13+
tail re-running the handler layer to produce a payload the capture already
14+
produced once:
15+
16+
- `ctx.router.match()` on the tail re-executes middleware, segment
17+
resolution, and every route handler. With the app's handler data sources
18+
`"use cache"`'d, the re-run costs ~26-36ms — cheap, but pure waste.
19+
- The full Flight render re-serializes the whole tree per HIT (the payload
20+
cannot be resumed — a React limitation), even though only the loader
21+
carve-outs differ from the capture's payload.
22+
- Prefetch renders pay the full path too: each PDP prefetch off the homepage
23+
measured ~2.5s (cold Builder cache key + CIO), starving the interactive
24+
tail in the same isolate.
25+
- The remaining tail latency was the LIVE lane (per-session upstream auth
26+
awaited inside chrome loader containers: 709ms fresh session → 247ms warm)
27+
— work this design deliberately keeps, because it is the honest floor.
28+
29+
So the question: why does a HIT run the handler layer at all?
30+
31+
## The consumer contract, in one sentence
32+
33+
**Only loaders are dynamic.** Middleware guards the request (live control),
34+
loaders load the data (live data), handlers render the page (cached).
35+
Everything else — handles, props, trees — inherits its lane from which of
36+
those three it rides. That is the entire PPR / cache() / prerender story a
37+
consumer has to learn, and it requires NO API change: the primitives already
38+
mean this; the fast path just makes the runtime agree with the mental model.
39+
40+
## The model
41+
42+
**The shell entry is a cache() of the handler layer, with loader records as
43+
the live carve-outs.**
44+
45+
A HIT becomes:
46+
47+
```
48+
middleware → loaders → replay stored payload fragments
49+
+ generate fresh loader records
50+
+ regenerate per-request metadata rows
51+
→ stream (prelude bytes + spliced payload)
52+
```
53+
54+
No handler execution. No full Flight render. Prelude/payload parity holds by
55+
construction — the payload IS the capture's payload, holes filled fresh.
56+
57+
What stays on the full path (one code path, entered less often): MISS,
58+
preview/editing modes, actions and progressive enhancement, and anything the
59+
eligibility rules below exclude.
60+
61+
## Eligibility is the cache purity rule — and it already shipped, piecemeal
62+
63+
The objection "handlers do per-request work (notFound, redirects, preview
64+
branches), you cannot bake them" dissolves because capture eligibility
65+
already encodes the purity contract:
66+
67+
- We only capture 200s. A 404/redirect decision never becomes a shell entry.
68+
- The identity guard refuses captures whose render reads request identity
69+
(`cookies()`/`headers()`), so an entry never encodes a handler that
70+
branches on the requester.
71+
- Per-REQUEST divergence (auth walls, entitlement) lives in middleware —
72+
which runs live on every HIT.
73+
- Per-TIME divergence (a URL mapping gains a redirect mid-TTL) heals by TTL
74+
plus recapture — ordinary cache staleness, same blast radius as the prelude
75+
already has today.
76+
77+
No new consumer contract is required. The contract exists; this design names
78+
it and rides it.
79+
80+
## v1 as built: the implicit doc-cache scope
81+
82+
The investigation that preceded the build collapsed the design to something
83+
much smaller than fragment splicing: **the cache() hit lane already IS the
84+
fast path**, end to end, in production today — for routes wrapped in
85+
`cache()`. `withCacheLookup` (match-middleware/cache-lookup.ts) is the
86+
innermost pipeline phase; on a hit it deserializes the stored segments,
87+
replays handle data, and runs `resolveLoadersOnly` — loaders fresh, handlers
88+
never executed. The doc-level entry (`doc:{host}{pathname}…`, ALL non-loader
89+
segments in one `CachedEntryData`) is written by `cacheRoute`, and the shell
90+
snapshot machinery (PR #691) already records segment-family writes into the
91+
`ShellCacheEntry.snapshot` at capture and replays them through the
92+
`SeededShellStore` on a HIT.
93+
94+
What was missing was only the connective tissue — the shell had no way to say
95+
"treat the whole matched route as a cache() boundary" for routes that never
96+
opted into `cache()`. v1 is exactly that, four small pieces:
97+
98+
1. **`resolveShellImplicitCacheScope`** (cache/cache-scope.ts), applied at the
99+
FULL MatchContext construction site (match-api.ts): when the route tree
100+
derived NO cache scope and the request context carries the
101+
`_shellImplicitCache` marker, substitute an enabled doc-level scope. A
102+
route-derived scope — including an explicit `cache(false)` — always wins.
103+
2. **Capture side** (shell-capture.ts): the derived capture context sets the
104+
marker with a `SnapshotOnlySegmentStore` — the capture's `cacheRoute`
105+
write records the doc segment entry into the snapshot ONLY, never the real
106+
store (a passthrough write would make the NEXT capture's lookup hit the
107+
previous generation and never re-run handlers, breaking SWR recapture
108+
freshness).
109+
3. **Serve side** (rsc-rendering.ts serveShellHit): the seeded tail context
110+
sets the marker when the entry is eligible; the tail's `ctx.router.match()`
111+
then HITs the seeded doc record and the handler layer is REPLAYED, not
112+
re-executed. Per-request payload metadata (initialTheme replay,
113+
locationState, …) is rebuilt by `buildFullPayload` exactly as before —
114+
there is no `root` on the wire; only `metadata.segments` content replays.
115+
4. **Eligibility** (`ShellCacheEntry.handlerLiveHoles`): the capture's
116+
handle-push funnel (the PR #692 mask) now also attributes each push —
117+
`isInsideLoaderScope()` read synchronously at push time. A nested thenable
118+
in a push made OUTSIDE a DSL loader scope (handler body, handler-invoked
119+
`ctx.use(loader)`, defers), a handler push still pending at the putShell
120+
barrier, or a HANDLER-INVOKED LOADER executing during the capture (the
121+
consumption lane, #672 — its "fresh per serve" slot value would freeze
122+
under replay) sets the flag: those holes only a handler re-run can fill,
123+
so the serve side declines the fast path and keeps today's full tail.
124+
DSL-loader pushes never disqualify — loaders re-run on every HIT — and
125+
their captured values are filtered OUT of the snapshot's handle records
126+
(they would duplicate the fresh push and stall the Flight handle encode on
127+
their masks).
128+
129+
## Why the original splice framing was dropped
130+
131+
- **Whole-payload Flight round-trip is fragile**: `metadata.handles` is an
132+
AsyncGenerator (not plain Flight data), live promises hang the buffered
133+
`streamToString` drain, and temporary references don't survive a store
134+
round-trip. Per-segment records — the shipped codec — avoid all three.
135+
- **Version coupling**: stored records are Flight wire format, coupled to the
136+
react-server-dom version and the client chunk graph. The buildVersion gate
137+
(PR #687) already invalidates shell entries on deploy; the doc record rides
138+
INSIDE the entry, so it dies with it. Same for cacheTag eviction: shell
139+
tags are unioned at the putShell barrier (#676/#680) and evict the whole
140+
entry, record included.
141+
- **Deterministic carve-outs**: capture-time masking (mask-nested.ts,
142+
PR #692) guarantees every per-request value is a hole — never a settled
143+
value — in both the prelude and the recorded segments. The correctness fix
144+
is the prerequisite for the fast path, not an orthogonal nicety.
145+
146+
## Parity: why this cannot introduce hydration errors
147+
148+
The dynamic surface on a fast-path HIT is exactly what it is on today's HIT —
149+
narrower, in fact:
150+
151+
| Layer | Today's HIT (full tail) | Fast-path HIT |
152+
| -------------------------------------------- | ------------------------------------------ | ------------------------------------------------ |
153+
| Middleware | live | live (unchanged — runs outside the shell branch) |
154+
| Handlers (path/layout/parallel/intercept) | re-executed; parity via snapshot pinning | REPLAYED byte-identically (never executed) |
155+
| DSL loaders | re-run; plain paths pinned by seed overlay | re-run; same overlay, same pinning |
156+
| `useLoader()` promise-shaped paths | live (stream into holes) | live (identical funnel) |
157+
| Handler-pushed handles (settled) | re-pushed by handler re-run | replayed from the entry |
158+
| Handler-pushed handles (nested promise) | live via handler re-run | **entry ineligible** → full tail |
159+
| Handler-invoked `ctx.use(loader)` (#672) | re-consumed by handler re-run | **entry ineligible** → full tail |
160+
| Loader-pushed handles | re-pushed by loader re-run | re-pushed by loader re-run (identical) |
161+
| Per-request metadata (theme/locationState/…) | rebuilt per request | rebuilt per request (buildFullPayload) |
162+
163+
Plain (non-promise) loader values stay PINNED to the capture on document HITs
164+
— the `_shellLoaderSeed` overlay's "recorded wins" rule — because the frozen
165+
prelude already displays the capture-time value; serving fresh plain data in
166+
the payload is precisely the hydration-mismatch class the snapshot machinery
167+
exists to kill. The nested-promise shape is the one and only opt-in to
168+
document-visible freshness. Partial navigations and action revalidations are
169+
untouched and always fresh.
170+
171+
Replay is byte-identical by construction, which is STRONGER than the full
172+
tail's re-render-and-pin approach: a handler that computes something the
173+
snapshot doesn't pin (a timestamp, a random id) drifts today; replayed it
174+
cannot.
175+
176+
## Unification with pre-rendering
177+
178+
`prerender-api-design.md`'s core principle — _pre-rendering is caching at
179+
build time_ — stops being an analogy. One entry format, two producers, one
180+
consumer:
181+
182+
- **Producer A (runtime)**: the background capture, triggered by traffic,
183+
TTL-bound.
184+
- **Producer B (build time)**: Prerender runs the same capture under
185+
`BuildContext`. Build-time production is the SAFER producer: there is no
186+
ambient user identity at build, so the identity guard is trivially
187+
satisfied and the capture-credential defense-in-depth concern vanishes for
188+
build-time entries.
189+
- **Consumer**: the fast path above. The worker cannot tell whether an entry
190+
came from a capture or from the build — extending the existing hard rule
191+
("the browser can't tell a route was pre-rendered") one layer deeper, into
192+
the payload.
193+
194+
The current two mental models — "PPR shell cache" vs "prerendered route" —
195+
collapse into one property of the entry: when it was produced and what its
196+
TTL is. A static marketing page is a build-time entry with infinite TTL whose
197+
basket badge still streams live; the homepage is a runtime capture with a
198+
300s TTL; both serve through identical code.
199+
200+
## What it buys (against the measurements above)
201+
202+
- Handler re-run + full Flight render on HITs: ~30-50ms → ~0.
203+
- PDP prefetch renders: full Dispatcher render (~2.5s cold) → fragment
204+
replay + loaders. Prefetch starvation of the interactive tail mostly
205+
disappears.
206+
- Tail wall-clock becomes exactly "your slowest live loader" — the honest
207+
floor. (The app-side lever for THAT: move session/token acquisition inside
208+
the nested promise so containers return instantly; see the field notes.)
209+
- Parity bugs between prelude and payload become impossible by construction
210+
rather than maintained by the overlay's "recorded wins" rule.
211+
212+
## The five open questions, as resolved in v1
213+
214+
1. **Fragment granularity → per-segment, via the shipped codec.**
215+
`CachedEntryData` (all non-loader segments of the match in one doc-keyed
216+
record) — the exact shape `cache()` serves in production. Whole-payload
217+
was rejected: `metadata.handles` is an AsyncGenerator and live promises
218+
hang the buffered Flight drain.
219+
2. **Per-request metadata rows → free.** There is no `root` on the wire; the
220+
payload object is rebuilt per HIT by `buildFullPayload`, so
221+
initialTheme/locationState/diff/redirect regenerate exactly as today. Only
222+
`metadata.segments` content replays.
223+
3. **Actions/PE → untouched.** The fast path arms only inside serveShellHit
224+
(document GET, non-partial, non-RSC); actions and PE never reach it, and
225+
`withCacheLookup`/`withCacheStore` both bail on `ctx.isAction`.
226+
4. **Eviction → inherited.** The doc record rides INSIDE the shell entry:
227+
TTL, SWR recapture, buildVersion gate, and cacheTag/updateTag (shell tags
228+
unioned at the putShell barrier) all evict prelude and record together.
229+
The SnapshotOnlySegmentStore guarantees no doc record ever outlives its
230+
entry in the real store.
231+
5. **Failure posture → self-healing, inherited.** A corrupt record fails
232+
`deserializeSegments` inside `lookupRoute`, which evicts and returns null
233+
— a MISS — so the tail falls through to full segment resolution (handler
234+
re-run), never a broken splice. A missing record, a handler-live entry, or
235+
a route-derived cache scope degrade the same way: full tail, today's
236+
behavior.
237+
238+
## What stays out of v1 (deliberate)
239+
240+
- **Routes with their own `cache()` keep their semantics** — including
241+
`cache(false)` (opt-out respected; full tail) and nested boundaries
242+
(their records already fast-path the covered subtree today).
243+
- **Prefetch renders and partial navigations** — unchanged; the fast path is
244+
document-HIT-tail only. The PDP-prefetch starvation lever is follow-up.
245+
- **Prerender unification (producer B)** — the entry format now carries
246+
everything a build-time producer needs; wiring `matchForPrerender` to emit
247+
shell entries is its own PR.
248+
- **Loader-pushed plain handle values consumed in shell content** drift on
249+
HITs (fresh push vs frozen prelude) exactly as they do today — the shape
250+
rule is the contract; not a fast-path regression.
251+
- **Shell key vs doc key divergence**: both sides derive the doc key from the
252+
same request context, so a mismatch (exotic keyGenerator) just misses the
253+
seeded record and degrades to the full tail.

‎packages/rangojs-router/e2e/shell-cache.test.ts‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -813,6 +813,61 @@ function runShellCacheSpec(f: Fixture): void {
813813
await warmToHit(request, url);
814814
});
815815
}
816+
817+
// --- Shell fast path: the execution matrix (docs/design/shell-fast-path.md).
818+
// On a fast-path HIT the tail match hits the captured doc segment record and
819+
// REPLAYS the handler layer: only middleware and DSL loaders execute. The
820+
// fixture's per-layer counters ride out through the loader (live lane), so
821+
// two consecutive HITs expose exactly which layers ran in between. The SWR
822+
// recapture is the deliberate exception (it re-runs handlers in a background
823+
// capture); the fixture's ttl (300s) keeps it out of the test window.
824+
test("fast-path HIT executes ONLY middleware and loaders — path/layout/parallel handlers are replayed, not run", async ({
825+
request,
826+
}) => {
827+
const url = f.url("/shell-cache/exec-matrix?probe=exec");
828+
await warmToHit(request, url);
829+
830+
const readHit = async (): Promise<{
831+
counters: {
832+
middleware: number;
833+
layout: number;
834+
parallel: number;
835+
path: number;
836+
loader: number;
837+
};
838+
html: string;
839+
}> => {
840+
const res = await request.get(url, { headers: HTML_HEADERS });
841+
expect(res.status()).toBe(200);
842+
expect(res.headers()["x-rango-shell"]).toBe("HIT");
843+
const html = await res.text();
844+
const match = html.match(
845+
/data-testid="shell-exec-counters"[^>]*>(\{[^<]+\})</,
846+
);
847+
expect(
848+
match,
849+
"streamed counters JSON present in the document",
850+
).toBeTruthy();
851+
// React escapes double quotes in text nodes; decode before parsing.
852+
return { counters: JSON.parse(match![1].replace(/&quot;/g, '"')), html };
853+
};
854+
855+
const first = (await readHit()).counters;
856+
const second = await readHit();
857+
858+
// Live layers: exactly one execution per HIT.
859+
expect(second.counters.middleware).toBe(first.middleware + 1);
860+
expect(second.counters.loader).toBe(first.loader + 1);
861+
862+
// Handler layers: replayed from the captured record — frozen across HITs.
863+
expect(second.counters.path).toBe(first.path);
864+
expect(second.counters.layout).toBe(first.layout);
865+
expect(second.counters.parallel).toBe(first.parallel);
866+
867+
// The replayed handler output still renders (structure intact, not blank).
868+
expect(second.html).toContain("Exec matrix static chrome");
869+
expect(second.html).toContain("exec badge");
870+
});
816871
}
817872

818873
test.describe("shell-cache (dev)", () => {
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
"use client";
2+
3+
import { useLoader } from "@rangojs/router/client";
4+
import type { LoaderDefinition } from "@rangojs/router";
5+
import type { ShellExecCounters } from "../urls/shell-cache.defs.js";
6+
7+
// Execution-matrix consumer (docs/design/shell-fast-path.md): renders the
8+
// loader's per-layer counter snapshot as JSON text so the e2e can parse which
9+
// layers executed on each serve straight out of the streamed document body.
10+
export function ShellExecMatrix({
11+
loader,
12+
}: {
13+
loader: LoaderDefinition<ShellExecCounters>;
14+
}) {
15+
const { data } = useLoader(loader);
16+
return <div data-testid="shell-exec-counters">{JSON.stringify(data)}</div>;
17+
}

‎packages/rangojs-router/e2e/test-app/src/router.named-routes.gen.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -325,6 +325,7 @@ export const NamedRoutes = {
325325
shellCacheActionPe: "/shell-cache-action/pe",
326326
shellCacheCorrupt: "/shell-cache/__corrupt",
327327
shellCacheDrift: "/shell-cache/drift",
328+
shellCacheExecMatrix: "/shell-cache/exec-matrix",
328329
shellCacheGuard: "/shell-cache/guard",
329330
shellCacheLayoutLoader: "/shell-cache/layout-loader",
330331
shellCacheLayoutLoaderBare: "/shell-cache/layout-loader-bare",

‎packages/rangojs-router/e2e/test-app/src/urls/shell-cache.defs.ts‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -252,3 +252,41 @@ export const ShellSrvChipLoader = createLoader(async (): Promise<string> => {
252252
shellSrvChipSeq += 1;
253253
return `srv-chip-${shellSrvChipSeq}-${visitor}`;
254254
});
255+
256+
// Shell fast-path EXECUTION MATRIX fixture (docs/design/shell-fast-path.md).
257+
// Per-layer module counters, incremented by the exec-matrix route's middleware,
258+
// layout handler, parallel handler, path handler, and DSL loader. The loader —
259+
// the LIVE lane (loading() on the route) — returns a snapshot of ALL counters,
260+
// so every serve reports which layers executed. The e2e takes two consecutive
261+
// HITs and asserts the deltas: middleware and loader advance (live), the three
262+
// handler counters stay frozen (replayed from the captured doc segment record,
263+
// never executed). An SWR recapture is the documented exception — it re-runs
264+
// handlers in a background capture — so the fixture uses a TTL long past the
265+
// test window. The ~150ms delay keeps the loader honest real-I/O (masked at
266+
// capture) and ensures a full tail's same-request handler increments land
267+
// before the snapshot, so the delta assertions read cleanly either way.
268+
const SHELL_EXEC_DELAY_MS = 150;
269+
270+
export interface ShellExecCounters {
271+
middleware: number;
272+
layout: number;
273+
parallel: number;
274+
path: number;
275+
loader: number;
276+
}
277+
278+
export const shellExecCounters: ShellExecCounters = {
279+
middleware: 0,
280+
layout: 0,
281+
parallel: 0,
282+
path: 0,
283+
loader: 0,
284+
};
285+
286+
export const ShellExecLoader = createLoader(
287+
async (): Promise<ShellExecCounters> => {
288+
shellExecCounters.loader += 1;
289+
await new Promise((resolve) => setTimeout(resolve, SHELL_EXEC_DELAY_MS));
290+
return { ...shellExecCounters };
291+
},
292+
);

0 commit comments

Comments
 (0)