Skip to content

Commit 5add969

Browse files
ryansolidclaude
andauthored
test(web): async-gen refetch oracle counts only answers the payload carried (#4446)
* test(web): async app generator — generated Loading/Errored/Show trees over async memos, with safety and liveness oracles An opt-in campaign (ASYNC_GEN=1, pnpm async-gen) that generates small apps (signals, sync and async memos with manual flights, latest()/isPending() reads, Loading / Errored / Show / leaves with grouped or split attribute effects) and scripts (writes, actions, settling flights in any order, flush / microtask / task). Oracles: nothing throws or logs, no unhandled rejection, no hole shows undefined/NaN, and once every flight has settled the screen equals a fresh synchronous render of the final inputs. Survey, shrink and minimize modes; the minimizer reduces the app as well as the script. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_0117BoWKJDzwaZPgaGWE181b * test(web): async-gen equivalence pairs (grouping, identity memo, inert Errored, store twin, flush mode, owned memos) ASYNC_GEN_PAIRS=E1,E2,E2e,E3,E6,E7 runs each scenario a second time rewritten without changing its meaning and compares the screens at every settle point (after the first render, each task step and the end); differences are grouped by what each side shows. Scenarios also choose whether memos are created at module level or inside the root component. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_0117BoWKJDzwaZPgaGWE181b * test(web): async-gen regression gate against a recorded baseline 1,500 generated scenarios (seed 3289, ~15 s) run with the web suite; each case's safety and liveness laws are compared to baseline.json. A case that newly breaks a law fails the suite (a change broke a neighbouring shape); a recorded failure that no longer reproduces is reported, and ASYNC_GEN_UPDATE=1 records it. The baseline holds today's 20 failing cases (#4362-#4367 shapes). Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_0117BoWKJDzwaZPgaGWE181b * test(web): async-gen hydration phase Each generated scenario is emitted as a TSX module, rendered through renderToStream (server half), then hydrated chunk by chunk (client half). Oracles: no refetch of a read memo before the first write, no console.error or hydration warning, hydrated screen equals a fresh CSR render, then the scenario's steps and final liveness. Opt-in via ASYNC_GEN_HY=1 (pnpm async-gen:hydrate); minimize-hydration.mts reduces a failing case through both halves. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_0117BoWKJDzwaZPgaGWE181b * test(web): async-gen differential mode; fix test-types ASYNC_GEN_RECORD on a reference build writes every case's screens, final screen and broken laws; ASYNC_GEN_DIFF on a candidate build replays the same seed and fails on any difference. Both the client campaign and the hydration campaign take it: the gate for the reader-seat carve. Same-build record/diff is clean (client 500 cases, hydration 300). minimize-hydration.mts loads minimize.ts through a computed specifier so tsc (no allowImportingTsExtensions) accepts it; runCase's return type names screens/final. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_0117BoWKJDzwaZPgaGWE181b * test(web): async-gen pairs E8 (birth seat) and E9 (latest in a derive) E8 re-creates the tree's readers in the flush that applies each write and compares screens where nothing is in flight; E9 reads each latest(x) the tree makes through a memo of it. Both from the carve audit's seat pairs. First run (1,000 cases): #4377, #4378. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_0117BoWKJDzwaZPgaGWE181b * test(web): async-gen refetch oracle counts only answers the payload carried A first ask of a memo the server never streamed is not a refetch (#4313, #4414): the emitted app reports each memo's hydration id before creating it, and the hydration campaign counts an ask only when the shell or a chunk carries that id's answer. Re-run on next: the one remaining hydration finding (m2 after an errored root) goes away; 251 of 561 memos are served and still checked. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_0117BoWKJDzwaZPgaGWE181b * test(web): quote the no-refetch rule in the hydration oracle Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_0117BoWKJDzwaZPgaGWE181b --------- Co-authored-by: Claude <[email protected]>
1 parent e56a4a0 commit 5add969

2 files changed

Lines changed: 25 additions & 30 deletions

File tree

‎packages/web/test/async-gen/emit.ts‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,9 @@ import type { Node, Read, Scenario, SrcRef } from "./scenario.js";
1414
export interface AppContext {
1515
values: number[];
1616
ask(name: string, v: number, poison: number | null): unknown;
17+
/** Called just before memo `name` is created (the client half reads the
18+
* hydration id it will take). */
19+
id?(name: string): void;
1720
}
1821

1922
const ref = (r: SrcRef) => (r.k === "s" ? `S[${r.i}][0]` : `M[${r.i}]`);
@@ -56,8 +59,8 @@ export function emitApp(sc: Scenario): string {
5659
.join(" + ");
5760
const poison = m.poison === null ? "null" : String(m.poison);
5861
return m.async
59-
? ` M[${i}] = createMemo(() => { const v = 1 + ${input}; return ctx.ask("m${i}", v, ${poison}); });`
60-
: ` M[${i}] = createMemo(() => { const v = 1 + ${input}; if (${poison} !== null && v % 4 === ${poison}) throw new Error("m${i}:" + v); return v; });`;
62+
? ` ctx.id?.("m${i}");\n M[${i}] = createMemo(() => { const v = 1 + ${input}; return ctx.ask("m${i}", v, ${poison}); });`
63+
: ` ctx.id?.("m${i}");\n M[${i}] = createMemo(() => { const v = 1 + ${input}; if (${poison} !== null && v % 4 === ${poison}) throw new Error("m${i}:" + v); return v; });`;
6164
})
6265
.join("\n");
6366
return `/**

‎packages/web/test/consistency/async-gen/hydrate.spec.tsx‎

Lines changed: 20 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,12 @@
1010
*
1111
* Per case, the shell is hydrated and every chunk applied, then:
1212
*
13-
* - NO REFETCH: no memo asks again before the first client write (an ask
14-
* made under hydration's dry run, with its fake Promise, is not one).
13+
* - NO REFETCH (Ryan, 2026-10-30): "a refetch is a re-ask of a read the
14+
* server answered (value or error); a first ask of a read the server
15+
* never reached is the client's ordinary ask". Hydration is CSR with the
16+
* server's answers claimed, so an ask counts only when the payload carries
17+
* that memo's answer and it comes before the first client write. An ask
18+
* made under hydration's dry run, with its fake Promise, is not one.
1519
* - CLEAN: no console.error, no hydration warning.
1620
* - HYDRATED = CSR: the hydrated screen equals a fresh client render of the
1721
* same inputs with every memo synchronous.
@@ -24,11 +28,12 @@
2428
import { readFileSync, existsSync, writeFileSync } from "node:fs";
2529
import { resolve } from "node:path";
2630
import { expect, test } from "vitest";
27-
import { action, flush, resetErrorHalt } from "solid-js";
31+
import { action, flush, getOwner, resetErrorHalt } from "solid-js";
32+
import { peekNextChildId } from "@solidjs/signals";
2833
import { hydrate, render } from "@solidjs/web";
2934
import { sharedConfig } from "solid-js/internal";
3035
import type { AppContext } from "../../async-gen/emit.js";
31-
import type { Node, Scenario, SrcRef } from "../../async-gen/scenario.js";
36+
import type { Scenario } from "../../async-gen/scenario.js";
3237
import { describeScenario } from "../../async-gen/scenario.js";
3338
import { recordOrDiff, type Trace } from "../../async-gen/differential.js";
3439
import { serialize as serializeDom } from "../../async-gen/run.js";
@@ -87,26 +92,6 @@ async function freshScreen(createApp: (ctx: AppContext) => any, values: number[]
8792
}
8893
}
8994

90-
/** The memos the tree reads, directly or through other memos. */
91-
function readMemos(sc: Scenario): Set<number> {
92-
const out = new Set<number>();
93-
const add = (r: SrcRef) => {
94-
if (r.k !== "m" || out.has(r.i)) return;
95-
out.add(r.i);
96-
sc.memos[r.i].deps.forEach(d => add(d.src));
97-
};
98-
const walk = (n: Node) => {
99-
if (n.t === "leaf") [...n.text, ...n.attrs].forEach(r => add(r.src));
100-
else if (n.t === "group") n.kids.forEach(walk);
101-
else {
102-
if (n.t === "show") add(n.when.src);
103-
walk(n.child);
104-
}
105-
};
106-
walk(sc.tree);
107-
return out;
108-
}
109-
11095
async function runCase(
11196
i: number
11297
): Promise<{ sc: Scenario; findings: Finding[]; screens: string[]; final: string }> {
@@ -170,8 +155,14 @@ async function runCase(
170155
guard("throw", flush);
171156
};
172157

158+
// Each memo's hydration id, taken just before it is created.
159+
const ids = new Map<string, string>();
173160
const ctx: AppContext = {
174161
values,
162+
id(name) {
163+
const owner = getOwner();
164+
if (owner && !ids.has(name)) ids.set(name, String(peekNextChildId(owner)));
165+
},
175166
ask(name, v, poison) {
176167
// Hydration's dry run swaps the global Promise for a mock: that run's
177168
// flight is never consumed, so it is not an ask.
@@ -206,10 +197,11 @@ async function runCase(
206197
}
207198
await drain();
208199
hydrating = false;
209-
// A memo the page never reads is never streamed (the server answers
210-
// what the render waited on), so its client ask is not a refetch.
211-
const read = readMemos(sc);
212-
const refetched = asks.filter(a => read.has(Number(a.slice(1, a.indexOf(":")))));
200+
// Only an answer the payload carried can be re-asked.
201+
const payload = shell + chunks.join("");
202+
const served = (name: string) =>
203+
ids.has(name) && payload.includes(`_$HY.r["${ids.get(name)}"]`);
204+
const refetched = asks.filter(a => served(a.slice(0, a.indexOf(":"))));
213205
if (refetched.length) findings.push({ law: "refetch", step, detail: refetched.join(",") });
214206
const hydrated = serialize(container);
215207
screens.push(hydrated);

0 commit comments

Comments
 (0)