fix(router): dev PPR capture backoff cap + cloudflare-basic ppr warmup (#652 item 3) - #659

Problem

On a cold CI runner the [dev] ppr-shell.test.ts suite hard-fails: warmToHit polls x-rango-shell for 20s and never sees HIT, on the test attempt AND both Playwright retries (the backoff state lives in the dev-server process and persists across retries). Four independent CI occurrences: the main run for SHA 2586ea9c (job cloudflare-basic-e2e), the PR #657 runs, and — after the first iteration of this PR — #659's own first CI run (physics hole failed 3x at ~20s each).

Three compounding causes, fixed together here, plus an unrelated CI-blocking unit flake:

(a) Backoff arithmetic. The refused-capture backoff window is exponential in the consecutive-failure count — min(BASE * 2^(failures-1), ceiling) — and the ceiling was a flat 60s in every mode. On a cold runner each cold re-probe climbs the count (1s → 2s → 4s → 8s → 16s → …); once the window exceeds ~16s it outlasts the 20s poll — every request inside it hits isCaptureBackedOff and is skipped → eternal MISS, even though the modules are warm by then.

(b) Cold module graph outruns the capture budget. #659's first CI run proved the dev cap alone is necessary but not sufficient: one probe cycle ≈ 5s attempt (SHELL_CAPTURE_MAX_WAIT_MS) + 5s in-place retry + 2s capped backoff ≈ 12s, so the 20s window fits only ~2 cycles — and today's cold runners need more, because the dev module graph is compiled on demand during the capture render, repeatedly blowing the 5s budget.

(c) CI budget — the 600s owner. tests/cloudflare-basic/playwright.config.ts set globalTimeout: 10 * 60 * 1000 on CI. The merged PPR shell stack grew the sequential dev → production → hmr playwright run past 10 minutes, so healthy runs brushed the cap and any flake retry tipped it over — Playwright then killed the run mid-flight with its own "Timed out waiting 600s for the test suite to run", failing whichever test happened to be executing. This mechanism explains the remaining CI deaths that were NOT eternal-MISS shaped: the 0ms hmr-warmup "failure" skip-cascade, the mid-hmr kill at 10:31:06 while hmr tests were passing, and contributed to the main 2586ea9c run (where the warmToHit retries' wall time cascaded into the cap).

(d) shell-snapshot.test.ts 1ms flake. The same #659 run also failed SeededShellStore > serves snapshot values AS FRESH — expiresAt ...881 vs ...880. The test builds the seeded store and the expected value from two separate snapshotOf() calls, and segData embeds Date.now() + 60_000; a millisecond tick between the calls fails the deep-equal.

This is item 3 of #652. Items 1–2 (barrier narrowing, drain unification) remain open — #652 must stay open after this PR merges (referenced here without closing keywords, deliberately).

Fix (four coordinated changes)

1. Dev backoff cap (runtime safety net)

The backoff ceiling is now mode-dependent: 60s in production (a genuinely ineligible route — no loading(), a cookie-reading handler — should be re-probed rarely), ~2s in dev, where the dominant no-shell cause is a cold graph that warms on the very attempt that failed.

// markCaptureBackoff — before: flat 60s ceiling in every mode
const window = Math.min(REFUSED_CAPTURE_BASE_MS * 2 ** (failures - 1), REFUSED_CAPTURE_MAX_MS);

// after: ceiling depends on mode
const window = Math.min(REFUSED_CAPTURE_BASE_MS * 2 ** (failures - 1), refusedCaptureCeilingMs());

function refusedCaptureCeilingMs(): number {
  return isDevMode() ? REFUSED_CAPTURE_DEV_MAX_MS /* 2_000 */ : REFUSED_CAPTURE_MAX_MS /* 60_000 */;
}
function isDevMode(): boolean {
  return process.env.NODE_ENV !== "production"; // build-folded to a literal; no runtime probe
}

The dev signal matches the rest of the RSC runtime (handler.ts, server-action.ts, progressive-enhancement.ts) — no new env probe.

2. cloudflare-basic PPR dev warmup (the sufficient fix, test-side)

Warm the module graph BEFORE the ppr suite runs, so the first real capture attempt runs against a hot graph and completes inside its 5s budget on attempt 1. Mechanism: extend the suite's existing dev-warmup setup project (tests/cloudflare-basic/e2e/dev-warmup.setup.ts), which the dev project already depends on — the same pattern packages/rangojs-router/playwright.config.ts uses. No config change needed; the new setup test is picked up by the project's testMatch.

The warmup:

• GETs every ppr fixture route from src/urls.tsx (/ppr-shell, /ppr-shell/stream, /ppr-shell/no-hole, /ppr-drift, /ppr-blog, /ppr-blog/getting-started-with-rsc) with Accept: text/html, compiling each route's page/loader modules AND the capture-only path (loader mask, captureShellHTML, fizz prerender) via the background capture each GET schedules;
• uses an isolated ?probe=warmup query on every GET — the shell key includes the query, so no test-visible shell entry is ever pre-stored and every test's own MISS→HIT sequence is untouched;
• then polls each capturable fixture (/ppr-shell, /ppr-shell/stream, /ppr-drift, /ppr-blog) to x-rango-shell: HIT, proving the full capture pipeline is hot — not merely requested. /ppr-shell/no-hole is excluded (it can never HIT; the suite pins that negative);
• is serialized after the base dep-optimizer warmup (setup.describe.configure({ mode: "serial" })) so local 2-worker runs cannot race the optimizer churn.

Cap + warmup together: the warmup makes first attempts succeed; the cap remains the safety net for anything cold the warmup missed. warmToHit windows and all other runtime constants are untouched. The production project needs nothing — build output is always warm (CI consistently passes prod).

(No nonce ppr fixture exists in cloudflare-basic's urls.tsx; the route list above is the complete ppr-declared set.)

3. CI budget: 14m playwright globalTimeout, 20m job cap

• tests/cloudflare-basic/playwright.config.ts — globalTimeout raised from 10m to 14m on CI, with a sizing comment: the PPR stack grew the suite past the old 10m (and the ppr warmup adds dev-time work); 14m playwright budget + ~3-4m of job setup (install, router build, browser deps) fits the workflow job cap.
• .github/workflows/e2e.yml cloudflare-basic-e2e — timeout-minutes raised from 15 to 20 ("20 (was 15): ..." comment mirroring the rangojs-router job's "30 (was 20)" style), so the workflow cap comfortably contains setup + the new playwright budget.

The other two configs with the same 600000 pattern were checked against recent CI runs and deliberately left unchanged — their jobs shard/split, so no single playwright run is near the cap:

(Measured on the main 2586ea9c run 28694625718 and this PR's run 28703125311.)

4. Deterministic shell-snapshot.test.ts

Pin the clock with vi.useFakeTimers() + setSystemTime (the src/cache/__tests__/shell-cache.test.ts pattern) in the racy test, so both snapshotOf() calls see the same Date.now(). The whole file was audited for siblings: no other assertion deep-equals values derived from two Date.now() instants (the other snapshotOf/segData uses assert partial fields or compare a single instance; settleWrites is bounded intentionally uses real timers with a generous bound).

Consumer-observable end-to-end

A consumer declaring PPR on an eligible route:

// route: /ppr-shell — a layout shell + a route-level loading() boundary (the hole)
export const ppr = { key: (url) => `${url.host}${url.pathname}:shell`, ttl: 300 };

On a cold dev worker the first document GET is x-rango-shell: MISS and schedules a background capture. Before this change, on a slow machine the exponential backoff could freeze the route on MISS for up to 60s. After, a cold-but-eligible route re-probes every ~2s in dev and flips to HIT as soon as the modules warm. No API change; only the dev-mode recovery cadence changes. The warmup is internal to this repo's e2e suite.

Semantics

Code change

• packages/rangojs-router/src/rsc/shell-capture.ts — add REFUSED_CAPTURE_DEV_MAX_MS (2s), isDevMode(), refusedCaptureCeilingMs(); markCaptureBackoff clamps to the mode ceiling. Backoff internals exported for unit tests (same convention as the existing runShellCapture test export). This is the single layer owning the backoff lifecycle, so the cap lives here.
• tests/cloudflare-basic/e2e/dev-warmup.setup.ts — the ppr warmup setup test (see above). Chosen over a new Playwright project or global-setup because the dev-warmup project → dev dependency already exists and its testMatch picks the file up with zero config churn.
• packages/rangojs-router/src/cache/__tests__/shell-snapshot.test.ts — fake timers in the racy test.
• packages/rangojs-router/src/rsc/__tests__/shell-capture.test.ts — new refused-capture backoff policy describe (see test plan).
• packages/rangojs-router/docs/design/ppr-shell-resume.md — "Refused-capture backoff" section documents the mode-dependent ceiling and the cold-CI failure.

Test plan

• Unit — shell-capture.test.ts (34 tests). New pins:
  • dev cap honored: the dev window never exceeds REFUSED_CAPTURE_DEV_MAX_MS however high the failure count climbs.
  • prod growth intact: window exceeds the dev cap, climbs to and clamps at 60s.
  • stored clears backoff: a stored capture resets the failure count.
  • simulated cold start (regression pin): walks the exact warmToHit cadence — capture fails at t0, one request per second for 20s, every probe also cold and re-marking. Dev cap: ≥8 probes total, ≥1 in the last 4s. The sibling prod-mode test asserts the same sequence freezes the tail to zero probes — the eternal MISS being fixed — making the dev pin non-vacuous.
• Unit — shell-snapshot.test.ts (14 tests): the previously-flaky deep-equal now runs under a pinned clock; contract unchanged (seeded values served AS FRESH, shouldRevalidate: false, real store untouched).
• E2e (dev + production):
  • cloudflare-basic: --project=dev with the warmup dependency enforced (dev-warmup → dev): 16 passed, twice consecutively; --project=production --no-deps: 14 passed. (An intermediate local repro that ran warmup CONCURRENTLY with the dev tests via --no-deps flaked ppr-blog — an artifact of stripping the project dependency, impossible on CI where dev-warmup always completes first.)
  • test-app shell-cache + shell-cache-action + shell-secure — 42 passed (dev+prod).
  • mini shell suites — green via test:unit:all.
• pnpm run typecheck
• pnpm run test:unit:all (297 router unit files + all consumer app suites + RSC suites)
• pnpm run lint
• pnpm run format
• pnpm check:e2e-bucketing
• Config sanity after the CI-budget change: playwright test --list --project=production loads the cloudflare-basic config and resolves the full project/dependency chain.

Notes

• Production runtime behavior is entirely unchanged; the built preview worker captures on attempt 1 (pre-built modules) and was never affected.
• The warmup deliberately does NOT pre-store any test-visible shell entry (?probe=warmup isolation), so the suite's MISS-first assertions (?probe=miss, ?probe=seq) and the no-hole negative are untouched.
• The recurring renders and hydrates on the live path, and soft-nav duplicate-ppr-price flake (4th sighting) is tracked on PPR capture pipeline hardening: narrow the write barrier, unify bounded drains, tame the backoff-vs-CI flake #652's comments and is out of scope here.
• PPR capture pipeline hardening: narrow the write barrier, unify bounded drains, tame the backoff-vs-CI flake #652 items 1–2 (barrier narrowing, drain unification) remain open — this PR fixes only item 3.