|
| 1 | +# Shell Fast Path: the shell entry as a cache() of the handler layer |
| 2 | + |
| 3 | +Status: v1 IMPLEMENTED on `feat/shell-fast-path` (2026-07-05); the "v1 as |
| 4 | +built" section below is the authoritative record of what shipped and why it |
| 5 | +needed almost no new machinery. Prerequisite work (capture-time |
| 6 | +nested-thenable masking) shipped in PR #692; the instrumentation that produced |
| 7 | +the numbers below shipped in PR #691 and PR #693. |
| 8 | + |
| 9 | +## Why (the measurements that motivated this) |
| 10 | + |
| 11 | +On a consumer storefront (workerd, PPR homepage, SFCC + Builder.io + CIO |
| 12 | +upstreams), a shell HIT commits headers at ~25ms — and then spends its entire |
| 13 | +tail re-running the handler layer to produce a payload the capture already |
| 14 | +produced once: |
| 15 | + |
| 16 | +- `ctx.router.match()` on the tail re-executes middleware, segment |
| 17 | + resolution, and every route handler. With the app's handler data sources |
| 18 | + `"use cache"`'d, the re-run costs ~26-36ms — cheap, but pure waste. |
| 19 | +- The full Flight render re-serializes the whole tree per HIT (the payload |
| 20 | + cannot be resumed — a React limitation), even though only the loader |
| 21 | + carve-outs differ from the capture's payload. |
| 22 | +- Prefetch renders pay the full path too: each PDP prefetch off the homepage |
| 23 | + measured ~2.5s (cold Builder cache key + CIO), starving the interactive |
| 24 | + tail in the same isolate. |
| 25 | +- The remaining tail latency was the LIVE lane (per-session upstream auth |
| 26 | + awaited inside chrome loader containers: 709ms fresh session → 247ms warm) |
| 27 | + — work this design deliberately keeps, because it is the honest floor. |
| 28 | + |
| 29 | +So the question: why does a HIT run the handler layer at all? |
| 30 | + |
| 31 | +## The consumer contract, in one sentence |
| 32 | + |
| 33 | +**Only loaders are dynamic.** Middleware guards the request (live control), |
| 34 | +loaders load the data (live data), handlers render the page (cached). |
| 35 | +Everything else — handles, props, trees — inherits its lane from which of |
| 36 | +those three it rides. That is the entire PPR / cache() / prerender story a |
| 37 | +consumer has to learn, and it requires NO API change: the primitives already |
| 38 | +mean this; the fast path just makes the runtime agree with the mental model. |
| 39 | + |
| 40 | +## The model |
| 41 | + |
| 42 | +**The shell entry is a cache() of the handler layer, with loader records as |
| 43 | +the live carve-outs.** |
| 44 | + |
| 45 | +A HIT becomes: |
| 46 | + |
| 47 | +``` |
| 48 | +middleware → loaders → replay stored payload fragments |
| 49 | + + generate fresh loader records |
| 50 | + + regenerate per-request metadata rows |
| 51 | + → stream (prelude bytes + spliced payload) |
| 52 | +``` |
| 53 | + |
| 54 | +No handler execution. No full Flight render. Prelude/payload parity holds by |
| 55 | +construction — the payload IS the capture's payload, holes filled fresh. |
| 56 | + |
| 57 | +What stays on the full path (one code path, entered less often): MISS, |
| 58 | +preview/editing modes, actions and progressive enhancement, and anything the |
| 59 | +eligibility rules below exclude. |
| 60 | + |
| 61 | +## Eligibility is the cache purity rule — and it already shipped, piecemeal |
| 62 | + |
| 63 | +The objection "handlers do per-request work (notFound, redirects, preview |
| 64 | +branches), you cannot bake them" dissolves because capture eligibility |
| 65 | +already encodes the purity contract: |
| 66 | + |
| 67 | +- We only capture 200s. A 404/redirect decision never becomes a shell entry. |
| 68 | +- The identity guard refuses captures whose render reads request identity |
| 69 | + (`cookies()`/`headers()`), so an entry never encodes a handler that |
| 70 | + branches on the requester. |
| 71 | +- Per-REQUEST divergence (auth walls, entitlement) lives in middleware — |
| 72 | + which runs live on every HIT. |
| 73 | +- Per-TIME divergence (a URL mapping gains a redirect mid-TTL) heals by TTL |
| 74 | + plus recapture — ordinary cache staleness, same blast radius as the prelude |
| 75 | + already has today. |
| 76 | + |
| 77 | +No new consumer contract is required. The contract exists; this design names |
| 78 | +it and rides it. |
| 79 | + |
| 80 | +## v1 as built: the implicit doc-cache scope |
| 81 | + |
| 82 | +The investigation that preceded the build collapsed the design to something |
| 83 | +much smaller than fragment splicing: **the cache() hit lane already IS the |
| 84 | +fast path**, end to end, in production today — for routes wrapped in |
| 85 | +`cache()`. `withCacheLookup` (match-middleware/cache-lookup.ts) is the |
| 86 | +innermost pipeline phase; on a hit it deserializes the stored segments, |
| 87 | +replays handle data, and runs `resolveLoadersOnly` — loaders fresh, handlers |
| 88 | +never executed. The doc-level entry (`doc:{host}{pathname}…`, ALL non-loader |
| 89 | +segments in one `CachedEntryData`) is written by `cacheRoute`, and the shell |
| 90 | +snapshot machinery (PR #691) already records segment-family writes into the |
| 91 | +`ShellCacheEntry.snapshot` at capture and replays them through the |
| 92 | +`SeededShellStore` on a HIT. |
| 93 | + |
| 94 | +What was missing was only the connective tissue — the shell had no way to say |
| 95 | +"treat the whole matched route as a cache() boundary" for routes that never |
| 96 | +opted into `cache()`. v1 is exactly that, four small pieces: |
| 97 | + |
| 98 | +1. **`resolveShellImplicitCacheScope`** (cache/cache-scope.ts), applied at the |
| 99 | + FULL MatchContext construction site (match-api.ts): when the route tree |
| 100 | + derived NO cache scope and the request context carries the |
| 101 | + `_shellImplicitCache` marker, substitute an enabled doc-level scope. A |
| 102 | + route-derived scope — including an explicit `cache(false)` — always wins. |
| 103 | +2. **Capture side** (shell-capture.ts): the derived capture context sets the |
| 104 | + marker with a `SnapshotOnlySegmentStore` — the capture's `cacheRoute` |
| 105 | + write records the doc segment entry into the snapshot ONLY, never the real |
| 106 | + store (a passthrough write would make the NEXT capture's lookup hit the |
| 107 | + previous generation and never re-run handlers, breaking SWR recapture |
| 108 | + freshness). |
| 109 | +3. **Serve side** (rsc-rendering.ts serveShellHit): the seeded tail context |
| 110 | + sets the marker when the entry is eligible; the tail's `ctx.router.match()` |
| 111 | + then HITs the seeded doc record and the handler layer is REPLAYED, not |
| 112 | + re-executed. Per-request payload metadata (initialTheme replay, |
| 113 | + locationState, …) is rebuilt by `buildFullPayload` exactly as before — |
| 114 | + there is no `root` on the wire; only `metadata.segments` content replays. |
| 115 | +4. **Eligibility** (`ShellCacheEntry.handlerLiveHoles`): the capture's |
| 116 | + handle-push funnel (the PR #692 mask) now also attributes each push — |
| 117 | + `isInsideLoaderScope()` read synchronously at push time. A nested thenable |
| 118 | + in a push made OUTSIDE a DSL loader scope (handler body, handler-invoked |
| 119 | + `ctx.use(loader)`, defers), a handler push still pending at the putShell |
| 120 | + barrier, or a HANDLER-INVOKED LOADER executing during the capture (the |
| 121 | + consumption lane, #672 — its "fresh per serve" slot value would freeze |
| 122 | + under replay) sets the flag: those holes only a handler re-run can fill, |
| 123 | + so the serve side declines the fast path and keeps today's full tail. |
| 124 | + DSL-loader pushes never disqualify — loaders re-run on every HIT — and |
| 125 | + their captured values are filtered OUT of the snapshot's handle records |
| 126 | + (they would duplicate the fresh push and stall the Flight handle encode on |
| 127 | + their masks). |
| 128 | + |
| 129 | +## Why the original splice framing was dropped |
| 130 | + |
| 131 | +- **Whole-payload Flight round-trip is fragile**: `metadata.handles` is an |
| 132 | + AsyncGenerator (not plain Flight data), live promises hang the buffered |
| 133 | + `streamToString` drain, and temporary references don't survive a store |
| 134 | + round-trip. Per-segment records — the shipped codec — avoid all three. |
| 135 | +- **Version coupling**: stored records are Flight wire format, coupled to the |
| 136 | + react-server-dom version and the client chunk graph. The buildVersion gate |
| 137 | + (PR #687) already invalidates shell entries on deploy; the doc record rides |
| 138 | + INSIDE the entry, so it dies with it. Same for cacheTag eviction: shell |
| 139 | + tags are unioned at the putShell barrier (#676/#680) and evict the whole |
| 140 | + entry, record included. |
| 141 | +- **Deterministic carve-outs**: capture-time masking (mask-nested.ts, |
| 142 | + PR #692) guarantees every per-request value is a hole — never a settled |
| 143 | + value — in both the prelude and the recorded segments. The correctness fix |
| 144 | + is the prerequisite for the fast path, not an orthogonal nicety. |
| 145 | + |
| 146 | +## Parity: why this cannot introduce hydration errors |
| 147 | + |
| 148 | +The dynamic surface on a fast-path HIT is exactly what it is on today's HIT — |
| 149 | +narrower, in fact: |
| 150 | + |
| 151 | +| Layer | Today's HIT (full tail) | Fast-path HIT | |
| 152 | +| -------------------------------------------- | ------------------------------------------ | ------------------------------------------------ | |
| 153 | +| Middleware | live | live (unchanged — runs outside the shell branch) | |
| 154 | +| Handlers (path/layout/parallel/intercept) | re-executed; parity via snapshot pinning | REPLAYED byte-identically (never executed) | |
| 155 | +| DSL loaders | re-run; plain paths pinned by seed overlay | re-run; same overlay, same pinning | |
| 156 | +| `useLoader()` promise-shaped paths | live (stream into holes) | live (identical funnel) | |
| 157 | +| Handler-pushed handles (settled) | re-pushed by handler re-run | replayed from the entry | |
| 158 | +| Handler-pushed handles (nested promise) | live via handler re-run | **entry ineligible** → full tail | |
| 159 | +| Handler-invoked `ctx.use(loader)` (#672) | re-consumed by handler re-run | **entry ineligible** → full tail | |
| 160 | +| Loader-pushed handles | re-pushed by loader re-run | re-pushed by loader re-run (identical) | |
| 161 | +| Per-request metadata (theme/locationState/…) | rebuilt per request | rebuilt per request (buildFullPayload) | |
| 162 | + |
| 163 | +Plain (non-promise) loader values stay PINNED to the capture on document HITs |
| 164 | +— the `_shellLoaderSeed` overlay's "recorded wins" rule — because the frozen |
| 165 | +prelude already displays the capture-time value; serving fresh plain data in |
| 166 | +the payload is precisely the hydration-mismatch class the snapshot machinery |
| 167 | +exists to kill. The nested-promise shape is the one and only opt-in to |
| 168 | +document-visible freshness. Partial navigations and action revalidations are |
| 169 | +untouched and always fresh. |
| 170 | + |
| 171 | +Replay is byte-identical by construction, which is STRONGER than the full |
| 172 | +tail's re-render-and-pin approach: a handler that computes something the |
| 173 | +snapshot doesn't pin (a timestamp, a random id) drifts today; replayed it |
| 174 | +cannot. |
| 175 | + |
| 176 | +## Unification with pre-rendering |
| 177 | + |
| 178 | +`prerender-api-design.md`'s core principle — _pre-rendering is caching at |
| 179 | +build time_ — stops being an analogy. One entry format, two producers, one |
| 180 | +consumer: |
| 181 | + |
| 182 | +- **Producer A (runtime)**: the background capture, triggered by traffic, |
| 183 | + TTL-bound. |
| 184 | +- **Producer B (build time)**: Prerender runs the same capture under |
| 185 | + `BuildContext`. Build-time production is the SAFER producer: there is no |
| 186 | + ambient user identity at build, so the identity guard is trivially |
| 187 | + satisfied and the capture-credential defense-in-depth concern vanishes for |
| 188 | + build-time entries. |
| 189 | +- **Consumer**: the fast path above. The worker cannot tell whether an entry |
| 190 | + came from a capture or from the build — extending the existing hard rule |
| 191 | + ("the browser can't tell a route was pre-rendered") one layer deeper, into |
| 192 | + the payload. |
| 193 | + |
| 194 | +The current two mental models — "PPR shell cache" vs "prerendered route" — |
| 195 | +collapse into one property of the entry: when it was produced and what its |
| 196 | +TTL is. A static marketing page is a build-time entry with infinite TTL whose |
| 197 | +basket badge still streams live; the homepage is a runtime capture with a |
| 198 | +300s TTL; both serve through identical code. |
| 199 | + |
| 200 | +## What it buys (against the measurements above) |
| 201 | + |
| 202 | +- Handler re-run + full Flight render on HITs: ~30-50ms → ~0. |
| 203 | +- PDP prefetch renders: full Dispatcher render (~2.5s cold) → fragment |
| 204 | + replay + loaders. Prefetch starvation of the interactive tail mostly |
| 205 | + disappears. |
| 206 | +- Tail wall-clock becomes exactly "your slowest live loader" — the honest |
| 207 | + floor. (The app-side lever for THAT: move session/token acquisition inside |
| 208 | + the nested promise so containers return instantly; see the field notes.) |
| 209 | +- Parity bugs between prelude and payload become impossible by construction |
| 210 | + rather than maintained by the overlay's "recorded wins" rule. |
| 211 | + |
| 212 | +## The five open questions, as resolved in v1 |
| 213 | + |
| 214 | +1. **Fragment granularity → per-segment, via the shipped codec.** |
| 215 | + `CachedEntryData` (all non-loader segments of the match in one doc-keyed |
| 216 | + record) — the exact shape `cache()` serves in production. Whole-payload |
| 217 | + was rejected: `metadata.handles` is an AsyncGenerator and live promises |
| 218 | + hang the buffered Flight drain. |
| 219 | +2. **Per-request metadata rows → free.** There is no `root` on the wire; the |
| 220 | + payload object is rebuilt per HIT by `buildFullPayload`, so |
| 221 | + initialTheme/locationState/diff/redirect regenerate exactly as today. Only |
| 222 | + `metadata.segments` content replays. |
| 223 | +3. **Actions/PE → untouched.** The fast path arms only inside serveShellHit |
| 224 | + (document GET, non-partial, non-RSC); actions and PE never reach it, and |
| 225 | + `withCacheLookup`/`withCacheStore` both bail on `ctx.isAction`. |
| 226 | +4. **Eviction → inherited.** The doc record rides INSIDE the shell entry: |
| 227 | + TTL, SWR recapture, buildVersion gate, and cacheTag/updateTag (shell tags |
| 228 | + unioned at the putShell barrier) all evict prelude and record together. |
| 229 | + The SnapshotOnlySegmentStore guarantees no doc record ever outlives its |
| 230 | + entry in the real store. |
| 231 | +5. **Failure posture → self-healing, inherited.** A corrupt record fails |
| 232 | + `deserializeSegments` inside `lookupRoute`, which evicts and returns null |
| 233 | + — a MISS — so the tail falls through to full segment resolution (handler |
| 234 | + re-run), never a broken splice. A missing record, a handler-live entry, or |
| 235 | + a route-derived cache scope degrade the same way: full tail, today's |
| 236 | + behavior. |
| 237 | + |
| 238 | +## What stays out of v1 (deliberate) |
| 239 | + |
| 240 | +- **Routes with their own `cache()` keep their semantics** — including |
| 241 | + `cache(false)` (opt-out respected; full tail) and nested boundaries |
| 242 | + (their records already fast-path the covered subtree today). |
| 243 | +- **Prefetch renders and partial navigations** — unchanged; the fast path is |
| 244 | + document-HIT-tail only. The PDP-prefetch starvation lever is follow-up. |
| 245 | +- **Prerender unification (producer B)** — the entry format now carries |
| 246 | + everything a build-time producer needs; wiring `matchForPrerender` to emit |
| 247 | + shell entries is its own PR. |
| 248 | +- **Loader-pushed plain handle values consumed in shell content** drift on |
| 249 | + HITs (fresh push vs frozen prelude) exactly as they do today — the shape |
| 250 | + rule is the contract; not a fast-path regression. |
| 251 | +- **Shell key vs doc key divergence**: both sides derive the doc key from the |
| 252 | + same request context, so a mismatch (exotic keyGenerator) just misses the |
| 253 | + seeded record and degrades to the full tail. |
0 commit comments