perf(ppr): fragment passthrough — HIT payloads emit stored snapshot fragments verbatim (#700) - #706

PPR fast path: emit stored snapshot fragments verbatim into the HIT payload (closes #700)

Problem

On a PPR shell HIT, the fast path (#695) already skips handler re-execution — but it still re-SERIALIZES the whole page per request. The tail deserialized the stored segment records into element trees and renderToReadableStream re-encoded the entire payload into __FLIGHT_DATA, so a consumer with a 3.77MB storefront homepage paid ~1.09MB of Flight generation per request (245 chunks) whose bytes are CONSTANT for the lifetime of the entry — at the measured ~5MB/s workerd stream-generation throughput, roughly 200-300ms of per-request isolate CPU that scales with page weight, not dynamism, and competes with concurrent renders (the PDP-prefetch starvation class). Waku has the same limitation: they killed re-execution but re-encode their stored slots into every document response. The stored snapshot already contains per-segment RSC-encoded fragment strings — the serialization work was already done once, at capture time, off the request path.

Fix (consumer-invisible; zero API change)

A HIT tail now emits those stored fragment strings VERBATIM into the outgoing payload. Replayed segments travel as fragment envelopes on their ReactNode fields; the outer Flight render copies each string into a raw T-row text chunk instead of re-encoding an element tree. Fresh serialization covers only what is genuinely live: loader rows, handle records, per-request metadata rows (pathname/params/theme/etc.). Both payload consumers — the SSR resume pass and browser hydration — expand each envelope through their own Flight deserializer before anything reads the segments, the same per-record decode the segment codec has always used (each fragment is its own row space, so there is no Flight row-id collision or shared-row dedupe hazard). The browser cannot tell: hydration sees the same trees the old deserialize→re-serialize round trip produced — now with one FEWER Flight round trip between capture and client.

Nothing changes for a consumer. No option, no new API, no entry-format change (producer A and producer B entries are consumed identically). Routes that don't use ppr see a single extra field scan per initial payload.

Before vs. after (the wire, /shell-cache/outlined HIT)

Before — the tail re-serializes the baked tree per request:

<script>self.__FLIGHT_DATA.push("...2:[\"$\",\"a\",\"row-0\",{...}]...  // 10,000 re-encoded element rows, every request
...(1.09MB of entry-constant bytes, re-generated each time)...")</script>

After — the stored fragment rides as one verbatim string copy; live rows stay live:

<script>self.__FLIGHT_DATA.push("...7:T1f4a2,<stored fragment bytes, copied>...   // {\"__rangoFragment\":1,\"f\":\"$7\"}
...loader rows / handles / metadata serialized fresh per request...")</script>

End-to-end consumer example

// routes.tsx — nothing new to write; any existing ppr route benefits.
export const router = createRouter((r) => [
  layout(StoreChrome, () => [
    parallel({
      "@cartBadge": {
        handler: () => <CartBadge />,
        use: () => [loader(CartCountLoader), loading(<BadgePending />)],
      },
    }),
    path("/", HomePage, { ppr: { ttl: 300, swr: 120 } }),
  ]),
]);

First request: MISS, background capture stores prelude + postponed + snapshot (unchanged). Every later request: prelude bytes flush at ~0ms; middleware and CartCountLoader run live; HomePage/StoreChrome are neither executed NOR re-serialized — their stored Flight fragments are string-copied into the payload. Hydration and the resume tree are identical to before.

Semantics table

Alternatives weighed (design inputs from vinext / Waku / Next.js)

• vinext-style whole-payload byte splice (store the baked payload bytes; serve verbatim; append fresh rows): unsound here. Rango's HIT payload interleaves live loader rows, the metadata.handles AsyncGenerator, and per-request metadata in the SAME Flight document, and the capture's payload stream is deliberately frozen mid-document (masked-loader rows never emit) — reusing its bytes means raw Flight row-id surgery, the exact hazard PPR fast path: emit stored snapshot fragments verbatim into the HIT payload (stop re-serializing baked segments per request) #700 rules out. vinext can serve whole documents verbatim because its prerendered pages are fully static; Rango's are not.
• Per-segment fragments expanded by consumers (this PR): stored bytes reach the wire without re-encoding (raw T-row copy), live rows stay genuinely live, and each fragment decodes in its own row space — the in-tree serializeResult/deserializeResult precedent.
• Next.js-style navigation-surface byte-reuse (serve stored flight to client-nav requests only): the honorable fallback if in-document splicing had proven unsound. Not needed — the document splice holds (battery below).

Code-change summary

• src/segment-fragments.ts (new, shared client/server, dependency-free): the { __rangoFragment: 1, f } envelope, isSegmentFragment, expandSegmentFragments(segments, decode) (in-place, parallel, Promise.resolve() adoption because Flight clients return thenable Chunks whose .then returns undefined), hasSegmentFragments. Envelopes ride ONLY ReactNode fields (component/layout/loading) — a plain object is never a valid ReactNode, so no collision with consumer data; loader data fields are never enveloped.
• src/cache/segment-codec.ts: fragmentSegments(data) — builds ResolvedSegments carrying the stored strings verbatim; decodes only loader-data fields (as before); preserves the loading: "null" sentinel server-side.
• src/server/request-context.ts: internal _shellFragmentPayload marker (+ PublicRequestContext omit).
• src/rsc/rsc-rendering.ts (serveShellHit): every HIT tail runs under a derived context with the marker — on the snapshot-seeded branch AND the no-snapshot branch (producer B entries often have no snapshot; their splice happens on the prerender-store lane). Never set on the shared reqCtx, so a scheduled capture cannot inherit it (an envelope reaching serializeSegments would store a double-encoded fragment).
• src/cache/cache-scope.ts (lookupRoute) + src/router/match-middleware/cache-lookup.ts (yieldFromStore): under the marker, build segments via fragmentSegments instead of deserializeSegments — covering the implicit doc record, route-scoped cache() hits, and the prerender store.
• src/ssr/ssr-root.tsx: the payload promise chains expandSegmentFragments through the SSR createFromReadableStream; onPayloadSettled fires after expansion so the capture's fizz-readiness gate includes fragment module loads.
• src/browser/rsc-router.tsx (initBrowserApp): expands fragments right after the initial payload decode, before the store seed / renderSegments / history cache see the segments.
• Docs: docs/design/shell-fast-path.md ("The fragment splice, as built"), packages/rangojs-router/docs/design/ppr-shell-resume.md (out-of-scope bullet now records PPR fast path: emit stored snapshot fragments verbatim into the HIT payload (stop re-serializing baked segments per request) #700 as shipped), docs/internal/feature-map.md, docs/internal/feature-file-map.md.

Why these locations: the splice must happen where the stored strings are already in hand (the two decode sites), and expansion must happen once per consumer environment at the single point each consumes the payload — everything between (match pipeline, buildFullPayload, gateTransitions, the render barrier) treats segments as opaque and needed no changes.

Measured win

Subject: /shell-cache/outlined (the 2MB-baked-section fixture from #703), test-app production build, Node preview server, M-series laptop, steady-state HITs. Method: process CPU delta (ps -o cputime) across 40 sequential HITs; latency via a streaming reader (n=15, median).

The eliminated CPU is exactly the two per-request Flight passes over entry-constant data (RSC-side deserializeSegments + whole-payload re-serialization); what remains is the SSR resume, the live loaders, and a string copy. On workerd — the issue's measured environment, ~5MB/s Flight generation — the eliminated re-serialization alone was ~200-300ms/request on the 1.09MB-payload storefront homepage; the Node numbers above are the same work at Node speed. Wire-shape verification: the fragments ride the outer stream as raw T-row text chunks (\d+:T[0-9a-f]+, present in the tail), and __rangoFragment envelopes appear on HITs only, never in the prelude, never on MISS.

Test plan

• Unit (pnpm run test:unit:all, all packages green):
  • src/__tests__/segment-fragments.test.ts — envelope detection (no false positives), in-place expansion across fields, loading null/undefined preservation, zero decoder calls on envelope-free payloads, decode-failure rejection, loaderData never expanded.
  • src/cache/__tests__/segment-codec-fragments.test.ts — fragmentSegments carries stored strings verbatim; loading sentinel; metadata/loader-data handling; expansion parity: fragmentSegments + expandSegmentFragments ≡ deserializeSegments; lookupRoute emits envelopes only under the tail marker.
  • src/rsc/__tests__/rsc-rendering-shell-ppr.test.ts — _shellFragmentPayload armed on the HIT tail context (snapshot-seeded AND no-snapshot branches) and never on the shared reqCtx (capture-leak guard).
  • Real-Flight round trip (test:unit:rsc, via the public @rangojs/router/testing/flight surface): src/testing/__tests__/segment-fragments.rsc-test.tsx — a stored fragment expands through the real vendored Flight client to the same tree as a direct decode: client island preserved as an I-row, typed props (Date/Map) intact, text content byte-identical; layout/loading fragments decode in independent row spaces.
• E2e, dev + production, test-app (shell-cache, shell-secure, shell-cache-action, prerender-ppr):
  • NEW HIT hydration payload carries verbatim fragment envelopes; MISS carries none (outlined route, 2MB section) — envelope only in the resumed tail, never the prelude; baked rows still delivered for hydration; MISS carries no envelopes.
  • NEW fragment-envelope HIT hydrates with zero errors and the slot hole stays live (slot-hole route) — guardHydrationErrors + live badge through an expanded-fragment shell.
  • NEW HIT payload carries verbatim fragment envelopes (prerender-store splice) (prerender-ppr) — producer-B composition lane.
  • Existing exec-matrix (fast-path HIT executes ONLY middleware and loaders), outlined-bake guarantee, drift/theme/hydration suites, shell-secure, shell-cache-action: green — the execution guarantees and hydration-cleanliness hold unchanged over fragment payloads.
• E2e, dev + production, cloudflare-basic (ppr-shell, incl. drift tests, ppr-tag-eviction):
  • NEW HIT payload carries verbatim fragment envelopes; the page hydrates cleanly on workerd.
  • ppr-tag-eviction's hydration test: the bare blog-sidebar locator is now scoped to the layout — see the streaming-race note below.
• Full battery counts: test-app production 43 passed (+1 retry-passed local pnpm-wrapper infra flake), dev 44 passed; semantic-matrix dev 21 / production 24 passed; cloudflare-basic dev 30 passed + tag-eviction production 4 passed, ppr-shell production 25 passed; tag-eviction dev hydration test 10/10 after the locator scoping.
• pnpm run typecheck (repo), pnpm run lint, pnpm run format, pnpm check:e2e-bucketing, pnpm check:docs-api — clean.
• No route changes — no *.gen.ts churn.

Notes

• Fragment strings ride as raw T rows. react-server-dom emits large strings as length-prefixed text chunks — a byte copy, no JSON escaping of the fragment body — so "stored bytes reach the wire without re-encoding" holds literally, not just asymptotically.
• Producer B needed the prerender-store lane, not the entry. Build-time entries share the entry format unchanged; but a Prerender+ppr tail serves segments from the prerender store (the lookup runs before the cache scope), so the splice is applied there too — one consumer, both producers.
• temporaryReferences caveat (pre-existing class). A baked segment containing a value only serializable via temporary references was already broken under replay (the store round trip cannot revive a temp ref); the failure point moves from a server-side error row to a consumer-side decode error, and the failed-tail recapture path covers it. No supported shape regresses.
• One timing shift this PR does cause (dev-workerd-visible, functionally benign): the dehydrated-boundary vs $RC race widens. The baked payload is now a byte copy, so __FLIGHT_DATA finishes further ahead of the fizz resume. When a hole's Flight data lands client-side before its $RC script executes, React client-renders the dehydrated Suspense boundary and fizz's hidden segment container (<div hidden id="S:n">) stays orphaned in the DOM — a standard React streaming artifact (raw responses verified complete and correct: $RC present, content once). Visible tree, hydration, and interactivity are identical; zero hydration errors. Observed only on the cloudflare dev worker's first-HITs (2/14 runs vs 0/14 on main); production clean everywhere (25/25 + 4/4). The one existing assertion that strict-mode-collided with the hidden node (ppr-tag-eviction sidebar toBeVisible) is now scoped to the layout; 10/10 after.
• Two pre-existing dev-only behaviors surfaced while testing (both reproduced on main, unrelated to this diff): (1) /shell-cache/outlined has a dev hydration mismatch — its module-level render counter bakes one value into the prelude and one more into the doc record; (2) /shell-cache's capture write-chain exceeds the 1s snapshot write-settle window in dev even warm, so its dev entries store snapshot-less and those dev HITs never arm the feat(ppr): shell fast path — HIT tails replay the handler layer; only loaders are dynamic #695 fast path (production is unaffected). The new tests route around both (slot-hole arms deterministically; a sacrificial warm-up absorbs cold-graph captures) — worth follow-up issues.
• No feature flag. The battery proves parity; per pre-release API hygiene no transitional option was added. The self-healing posture for a corrupt record shifts from pre-commit MISS to post-commit recapture (documented in the semantics table) — same class the shell already handles for a mismatched postponed blob.