fix(router): RSC flight wire format becomes explicit opt-in — generic clients get the HTML document - #705

Problem

curl https://app.example.com/products/1 returned the RSC flight wire format (text/x-component) instead of the page. The render layer classified any Accept header lacking the literal substring text/html as a flight request — so curl (Accept: */*), clients sending no Accept header, Accept: application/json on a page with no JSON variant, health checks, uptime monitors, and link unfurlers all received an internal transport payload they cannot render. On PPR routes this also hid the x-rango-shell header from curl, which read as "PPR is off" (the ppr skill carried a workaround note for exactly this).

Two internal defects rode on the same rule:

• document-cache.ts:303 keys HTML-vs-RSC response slots via mayNeedSSR, but the response body was decided by isRscRequest — and the two disagreed on a missing Accept header, so a no-Accept GET on a document-cached route stored a flight body under the :html cache key.
• On a negotiated route with a variant defined before the RSC page, an explicit Accept: text/x-component fell through to the definition-order fallback and answered with the variant (JSON) instead of the flight stream.

Before vs after

# Before
curl https://app.example.com/products/1
# → 200 text/x-component;charset=utf-8   (flight wire format)

# After
curl https://app.example.com/products/1
# → 200 text/html;charset=utf-8          (the rendered page)

Fetching the flight payload now requires the explicit wire-format Accept (scripts relying on bare curl for flight must add it):

curl -H "Accept: text/x-component" https://app.example.com/products/1
# → 200 text/x-component;charset=utf-8

End-to-end consumer example

// urls.tsx — one URL, two representations + a JSON variant
export const urlpatterns = urls(({ path }) => [
  path("/products/:id", ProductPage, { name: "product" }),
  path.json("/products/:id", (ctx) => db.getProduct(ctx.params.id), {
    name: "productJson",
  }),
]);

curl https://shop.example.com/products/42                                   # HTML page
curl -H "Accept: application/json"    https://shop.example.com/products/42  # JSON variant
curl -H "Accept: text/x-component"   https://shop.example.com/products/42  # flight stream
curl -s -D - -o /dev/null https://shop.example.com/products/42 | grep x-rango-shell
# x-rango-shell: HIT — PPR shells are now observable from a bare curl

Semantics table

Code changes

• src/rsc/ssr-setup.ts — mayNeedSSR and isRscRequest now share one acceptsFlightExplicitly predicate. The Accept rule flips from "no text/html substring → flight" to "explicit text/x-component, ranked above the document, → flight". A includes() fast path keeps generic traffic allocation-free. Because both functions share the predicate, the document-cache key split can no longer diverge from the render decision.

  function acceptsFlightExplicitly(request: Request, url: URL): boolean {
    if (url.searchParams.has("__html")) return false;
    const accept = request.headers.get("accept");
    if (accept === null || !accept.includes(RSC_WIRE_MIME)) return false;
    return prefersFlightRepresentation(parseAcceptTypes(accept));
  }

• src/router/content-negotiation.ts — the RSC candidate registers under both of its representations via a data-level RSC_MIMES = ["text/html", RSC_WIRE_MIME] list, and the representation ranking (prefersFlightRepresentation) is co-located with it so registration and ranking cannot drift. Location rationale: rsc/ → router/ is the established import direction (response-route-handler.ts already imports RESPONSE_TYPE_MIME), so the shared MIME knowledge lives in the lower layer.

• src/rsc/handler.ts — comment update only (the mayNeedSSR/isRscRequest asymmetry note is obsolete).

• Skills: mime-routes rewritten around the two-representation model (rules table + MIME mapping); ppr "Verifying it works" bare-curl gotcha removed — bare curl now sees x-rango-shell directly.

Test plan

• Red-before-green: 6 e2e failures captured on unfixed code — Accept: */*, no-Accept, and application/json receiving text/x-component; JSON-first route answering an explicit wire-format request with JSON (packages/rangojs-router/e2e/content-negotiation.test.ts:143,171,220,239,252,263 under the old code). All pass with the fix.
• Unit: new isRscRequest matrix + mayNeedSSR cases incl. q-ranking, q=0 refusal, wildcard-outranks, and a cross-predicate agreement test pinning isRscRequest === !mayNeedSSR across the Accept matrix (src/rsc/__tests__/ssr-setup.test.ts); pickNegotiateVariant wire-format selection + no-shadowing (src/router/__tests__/content-negotiation.test.ts); one old-semantics kickoff test updated — no-Accept now correctly starts SSR setup (src/rsc/__tests__/handler-ssr-kickoff.test.ts). Full test:unit:all: 5077+ tests pass.
• E2e dev + production, both apps (Hard rule 3): router content-negotiation 39/39; cloudflare-basic content-negotiation 40/40 (new: wire-format on JSON-first route, plain-route */*→HTML, plain-route wire-format→flight). Contract pinned: flight is explicit-opt-in; generic clients get HTML; internal transports unchanged.
• Adjacent e2e suites pass dev + production: mw-chain, content-ownership, loader-fetchable-guard, loader-fetch-thrown-response, stream-mode, prefetch-cache-control, origin-guard, app-middleware, shell-cache, progressive-enhancement.
• Production semantics additionally verified by curl matrix against a real vite build + vite preview of the e2e test-app, including x-rango-shell: HIT on bare curl.
• typecheck (16/16), lint (0 warnings), format, check:e2e-bucketing; no *.gen.ts drift (no route changes).

CI round 1 findings (amended into this PR)

The first CI run surfaced one real regression and three tests pinning the old transport implicitly:

• Real regression (fixed in isRscRequest): the 404 fallback plan hardcodes mode: "full-render" even for _rsc_partial navigations (handler.ts RouteNotFoundError catch), so partial 404s reached the render decision with isPartial=false. The old Accept rule masked this by classifying */* as flight; without it, client navigations to missing routes received an HTML 404 they cannot apply and never committed (multi-router soft-404, popstate not-found). isRscRequest now also reads _rsc_partial from the URL — the canonical transport marker, same as mayNeedSSR — closing the class for any caller that loses the plan. Unit test pins it; multi-router + not-found suites green dev+prod.
• Tests re-pinned to explicit flight (intent unchanged): shell-cache.test.ts / cf-basic ppr-shell.test.ts "bypasses non-document requests" (were Accept: */*), cf-basic cache-signal.test.ts miss→hit (was default Accept). Verified the cache-signal miss→hit signal is a flight-transport contract: Accept: text/html twice was miss/miss before this PR too — browsers were always on that path; */* clients now join it.
• ppr-shell:244 and shell-cache:962 failures were serial knock-on from the above; both pass in full local suite runs.

Notes

• Migration: any out-of-tree script fetching flight with default headers must add Accept: text/x-component (or ?__rsc=1). The router's own client runtime is unaffected — it always marks flight requests with transport params.
• Bots/monitors now trigger SSR instead of the cheaper flight render; intended — they were previously getting an unusable payload.
• Deliberately unchanged: Accept: text/html;q=0 still counts as accepting HTML (pre-existing substring limitation, out of scope); */* definition-order fallback on negotiated routes; emitted text/x-component;charset=utf-8 response-header literals across handlers (pre-existing duplication, separate concern from the matching MIME).
• The testing primitives (dispatch) cannot reach trie-driven variant selection or the HTML/flight render decision (pre-existing limitation documented in src/testing/__tests__/dispatch.test.ts:1355); per that precedent the selection behavior is pinned by dev+prod e2e and the pure functions by unit tests.