Skip to content

Commit 9e4135c

Browse files
test(lint): ban per-test timeouts and unreasoned skips
Adds t3code/no-test-timeout-or-unreasoned-skip: per-test timeout/retry arguments, hook timeouts, and vi.setConfig timeouts are errors, and .skip/.todo/.only/xit/fit need a `// quarantine: #issue` comment. Opt-in *.live.test.* provider suites keep their timeouts. Removes every existing override, pins retry: 0, allowOnly outside CI, and the full fake-timer set in the root test config, and spells out the no-timing rules in AGENTS.md. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent fcb20e4 commit 9e4135c

22 files changed

Lines changed: 1072 additions & 841 deletions

‎AGENTS.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -99,7 +99,15 @@ An empty database is a bad test. Seed your worktree's `.t3` with a copy of real
9999
- Test meaningful logic or observable behavior. Do not render components to static markup to assert props or attributes, or add tests that merely assert callback wiring or mirror the implementation.
100100
- **Do not run repo-wide checks.** No `vp check`, no `vp run -r test`, no `vp run -r typecheck` unless I ask. CI owns the full suite.
101101
- Backend behavior changes ship with focused tests for that behavior.
102-
- The server is event-sourced, and side effects run after the command commits. In tests, drain the effect worker (`OrchestrationEffectWorkerV2.drain`) or await the specific persisted event or `Deferred` that marks the milestone. Never wait on sleeps or polling. A test that needs a timeout to pass is wrong.
102+
- The server is event-sourced, and side effects run after the command commits. In tests, drain the effect worker (`OrchestrationEffectWorkerV2.drain`) or await the specific persisted event or `Deferred` that marks the milestone. Never wait on sleeps or polling. A test that needs a timeout to pass is wrong. Concretely:
103+
- `it.effect` runs on a `TestClock`; `Effect.sleep` only advances through `TestClock.adjust`. `it.live` is wall-clock and only for subprocess or network fixtures.
104+
- No `yieldNow` or `sleep` polling loops. Await the `Deferred`, take from the `Queue`, or drain the worker.
105+
- No per-test timeout argument or `retry`. The project timeout is the ceiling, and flaky tests are fixed, not retried.
106+
- No `performance.now()` or `Date.now()` deltas. Assert on events and counts, not elapsed time.
107+
- Browser tests assert through `expect.element` or `vi.waitFor`, never `requestAnimationFrame` or `transitionend`.
108+
- `vi.mock` factories must not spread a real module that pulls in Shiki or the diff workers.
109+
- `it.skip` and friends need a tracking issue on the line above: `// quarantine: #1234 <why>`.
110+
- Lint enforces this: `t3code/no-test-timeout-or-unreasoned-skip`, `t3code/no-frame-waits-in-browser-tests`, `t3code/no-wall-clock-in-tests`, `t3code/no-real-time-wait-in-tests`, `t3code/no-polling-loop-in-tests`.
103111
- Web browser tests are `*.browser.test.tsx` or `*.browser.test.ts` and render with `await using screen = await render(...)` from `apps/web/test/render.tsx`. `t3code/require-disposable-render` errors on the old patterns.
104112
- Acquire what the test drives after mounting (fake timers it advances, spies it sets up after mounting) after the render, so it disposes before the unmount. Anything the first render reads comes before it.
105113
- A helper that renders and then waits or clicks puts the screen in an `AsyncDisposableStack` and `move()`s it out only on success, so a failed wait still unmounts.

‎apps/mobile/src/lib/uniwind-platform-variants.test.ts‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ describe("uniwind platform variants compile per platform", () => {
8282

8383
beforeAll(() => {
8484
output = runFixture('@import "tailwindcss";\n@import "uniwind";\n', FIXTURE_SOURCE);
85-
}, 60_000);
85+
});
8686

8787
it("compiles the fixture utilities into shared platform blocks", () => {
8888
expect(output.tailwindChecks).toEqual({
@@ -169,7 +169,7 @@ describe("uniwind keeps media rules nested inside class rules attached to the cl
169169
].join("\n"),
170170
'export const Probe = () => <div className="container-x foo-x" />;\n',
171171
);
172-
}, 60_000);
172+
});
173173

174174
it("keeps the nested breakpoint entry of a nested media rule", () => {
175175
for (const platform of ["ios", "android"]) {
@@ -199,7 +199,7 @@ describe("uniwind keeps selector and transformer behavior", () => {
199199
'@import "tailwindcss";\n@import "uniwind";\n',
200200
'export const Probe = () => <div className="active:opacity-50 disabled:opacity-50 data-x:underline aria-disabled:text-red-500" />;\n',
201201
);
202-
}, 60_000);
202+
});
203203

204204
it("keeps state and data variants conditioned, and rejects unsupported compounds", () => {
205205
for (const platform of ["ios", "android"]) {

‎apps/mobile/src/reactCompilerCoverage.test.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,4 +145,4 @@ describe("React Compiler coverage", () => {
145145
it("lists only files that exist", () => {
146146
expect(Object.keys(KNOWN_BAILOUTS).filter((file) => !sourceFiles.includes(file))).toEqual([]);
147147
});
148-
}, 300_000);
148+
});

‎apps/server/integration/transferBudgetV2.integration.test.ts‎

Lines changed: 139 additions & 145 deletions
Original file line numberDiff line numberDiff line change
@@ -219,150 +219,144 @@ const synchronized = (queue: Queue.Queue<StreamItem>) =>
219219
),
220220
);
221221

222-
it.live(
223-
"reports V2 thread HTTP and WebSocket transfer budgets",
224-
() =>
225-
Effect.gen(function* () {
226-
const runs: TransferBudgetRun[] = [];
227-
for (const provider of [
228-
ProviderDriverKind.make("codex"),
229-
ProviderDriverKind.make("claudeAgent"),
230-
]) {
231-
const counter = makeSqlStatementCounter();
232-
const run = yield* Effect.scoped(
233-
Effect.gen(function* () {
234-
const sink = yield* EventSink.EventSinkV2;
235-
const projection = yield* ProjectionStore.ProjectionStoreV2;
236-
yield* sink.write({ events: [threadCreated(provider)] });
237-
for (let index = 0; index < TRANSFER_HISTORY_TURN_COUNT; index++)
238-
yield* sink.write({ events: turnEvents(provider, index, false) });
239-
const context = yield* Effect.context<Layer.Success<typeof layerServices>>();
240-
const server = yield* Layer.build(
241-
HttpRouter.serve(layerRoutes, { disableListenLog: true }).pipe(
242-
Layer.provide(Layer.succeedContext(context)),
243-
Layer.provideMerge(
244-
NodeHttpServer.layer(NodeHttp.createServer, {
245-
host: "127.0.0.1",
246-
port: 0,
247-
websocket: { perMessageDeflate: true },
248-
}),
249-
),
250-
),
251-
);
252-
const address = Context.get(server, HttpServer.HttpServer).address;
253-
if (!("port" in address)) return yield* Effect.die("Expected TCP server");
254-
const base = `http://127.0.0.1:${address.port}`;
255-
const threadSnapshot = yield* measureHttpGet({
256-
url: `${base}/api/orchestration/threads/${THREAD_ID}`,
257-
headers: { [ORCHESTRATION_PROTOCOL_HEADER]: String(ORCHESTRATION_PROTOCOL_VERSION) },
258-
});
259-
assert.equal(
260-
threadSnapshot.status,
261-
200,
262-
Buffer.from(threadSnapshot.decodedBody).toString(),
263-
);
264-
assert.equal(threadSnapshot.contentEncoding, "gzip");
265-
const decodedThread = yield* decodeThreadSnapshot(
266-
Buffer.from(threadSnapshot.decodedBody).toString(),
267-
);
268-
assert.equal(decodedThread.projection.messages.length, TRANSFER_HISTORY_TURN_COUNT * 2);
269-
assert.equal(
270-
decodedThread.projection.turnItems.length,
271-
TRANSFER_HISTORY_TURN_COUNT * 6,
272-
);
273-
assert.notInclude(Buffer.from(threadSnapshot.decodedBody).toString(), "digest=");
274-
const shellSnapshot = yield* measureHttpGet({
275-
url: `${base}/api/orchestration/shell`,
276-
headers: { [ORCHESTRATION_PROTOCOL_HEADER]: String(ORCHESTRATION_PROTOCOL_VERSION) },
277-
});
278-
assert.equal(shellSnapshot.status, 200);
279-
const decodedShell = yield* decodeShellSnapshot(
280-
Buffer.from(shellSnapshot.decodedBody).toString(),
281-
);
282-
assert.equal(decodedShell.threads.length, 1);
283-
const sequence = decodedThread.snapshotSequence;
284-
assert.equal(decodedShell.snapshotSequence, sequence);
285-
const url = `${base.replace("http:", "ws:")}/ws`;
286-
const thread = yield* openMeasuredWsClient({ url, cookie: "" });
287-
const shell = yield* openMeasuredWsClient({ url, cookie: "" });
288-
const second = yield* openMeasuredWsClient({ url, cookie: "" });
289-
assert.include(thread.recorder.negotiatedExtensions(), "permessage-deflate");
290-
const tq = yield* subscribe(thread, "thread", sequence);
291-
const sq = yield* subscribe(shell, "shell", sequence);
292-
const ttq = yield* subscribe(second, "thread", sequence);
293-
const ssq = yield* subscribe(second, "shell", sequence);
294-
for (const queue of [tq, sq, ttq, ssq])
295-
assert.equal(yield* synchronized(queue), "replay");
296-
const threadBefore = thread.recorder.totals();
297-
const shellBefore = shell.recorder.totals();
298-
const secondBefore = second.recorder.totals();
299-
const sqlBefore = counter.count();
300-
const events = turnEvents(provider, TRANSFER_HISTORY_TURN_COUNT, true);
301-
const stored = yield* sink.write({ events });
302-
const last = stored.at(-1)!;
303-
const threadTerminal = (item: StreamItem) =>
304-
"sequence" in item && item.sequence >= last.sequence;
305-
// Shell items carry the shell read when their batch is projected, and unchanged
306-
// shells are skipped, so a turn split across batches may never yield an item at
307-
// the last sequence. Wait for the final shell instead.
308-
const finalShell = yield* projection.getThreadShell(THREAD_ID);
309-
const shellTerminal = (item: StreamItem) =>
310-
item.kind === "thread.updated" && NodeUtil.isDeepStrictEqual(item.thread, finalShell);
311-
for (const queue of [tq, ttq]) yield* collectUntil(queue, threadTerminal);
312-
for (const queue of [sq, ssq]) yield* collectUntil(queue, shellTerminal);
313-
const measuredTurnWebSocket = difference(thread.recorder.totals(), threadBefore);
314-
const measuredTurnShellWebSocket = difference(shell.recorder.totals(), shellBefore);
315-
const measuredTurnSecondClientWebSocket = difference(
316-
second.recorder.totals(),
317-
secondBefore,
318-
);
319-
const measuredTurnSqlStatements = counter.count() - sqlBefore;
320-
const durable = yield* projection.getThreadProjection(THREAD_ID);
321-
assert.isTrue(
322-
durable.turnItems.some(
323-
(item) =>
324-
item.type === "dynamic_tool" && JSON.stringify(item.output).length > 1_000_000,
222+
it.live("reports V2 thread HTTP and WebSocket transfer budgets", () =>
223+
Effect.gen(function* () {
224+
const runs: TransferBudgetRun[] = [];
225+
for (const provider of [
226+
ProviderDriverKind.make("codex"),
227+
ProviderDriverKind.make("claudeAgent"),
228+
]) {
229+
const counter = makeSqlStatementCounter();
230+
const run = yield* Effect.scoped(
231+
Effect.gen(function* () {
232+
const sink = yield* EventSink.EventSinkV2;
233+
const projection = yield* ProjectionStore.ProjectionStoreV2;
234+
yield* sink.write({ events: [threadCreated(provider)] });
235+
for (let index = 0; index < TRANSFER_HISTORY_TURN_COUNT; index++)
236+
yield* sink.write({ events: turnEvents(provider, index, false) });
237+
const context = yield* Effect.context<Layer.Success<typeof layerServices>>();
238+
const server = yield* Layer.build(
239+
HttpRouter.serve(layerRoutes, { disableListenLog: true }).pipe(
240+
Layer.provide(Layer.succeedContext(context)),
241+
Layer.provideMerge(
242+
NodeHttpServer.layer(NodeHttp.createServer, {
243+
host: "127.0.0.1",
244+
port: 0,
245+
websocket: { perMessageDeflate: true },
246+
}),
325247
),
326-
);
327-
yield* second.close;
328-
const reconnectSqlStart = counter.count();
329-
const resumedThread = yield* openMeasuredWsClient({ url, cookie: "" });
330-
const resumedShell = yield* openMeasuredWsClient({ url, cookie: "" });
331-
const rq = yield* subscribe(resumedThread, "thread", sequence);
332-
const rsq = yield* subscribe(resumedShell, "shell", sequence);
333-
const threadMode = yield* synchronized(rq);
334-
const shellMode = yield* synchronized(rsq);
335-
const reconnectThread = { ...resumedThread.recorder.totals(), mode: threadMode };
336-
const reconnectShell = { ...resumedShell.recorder.totals(), mode: shellMode };
337-
assert.equal(threadMode, "snapshot");
338-
return {
339-
provider,
340-
threadSnapshot,
341-
shellSnapshot,
342-
measuredTurnWebSocket,
343-
measuredTurnShellWebSocket,
344-
measuredTurnSecondClientWebSocket,
345-
measuredTurnSqlStatements,
346-
reconnectThread,
347-
reconnectShell,
348-
reconnectSqlStatements: counter.count() - reconnectSqlStart,
349-
} satisfies TransferBudgetRun;
350-
}).pipe(
351-
Effect.provide(Layer.fresh(layerServices).pipe(Layer.provideMerge(NodeServices.layer))),
352-
Effect.withTracer(counter.tracer),
353-
),
354-
);
355-
runs.push(run);
356-
}
357-
const fs = yield* FileSystem.FileSystem;
358-
const report = formatTransferBudgetReport(runs);
359-
for (const [path, contents] of [
360-
[process.env.T3CODE_TRANSFER_BUDGET_REPORT_PATH, report],
361-
[process.env.T3CODE_TRANSFER_BUDGET_RESULT_PATH, formatTransferBudgetResult(runs)],
362-
])
363-
if (path && contents) yield* fs.writeFileString(path, contents);
364-
yield* Effect.log(report);
365-
assert.deepEqual(transferBudgetViolations(runs), []);
366-
}).pipe(Effect.provide(NodeServices.layer)),
367-
30_000,
248+
),
249+
);
250+
const address = Context.get(server, HttpServer.HttpServer).address;
251+
if (!("port" in address)) return yield* Effect.die("Expected TCP server");
252+
const base = `http://127.0.0.1:${address.port}`;
253+
const threadSnapshot = yield* measureHttpGet({
254+
url: `${base}/api/orchestration/threads/${THREAD_ID}`,
255+
headers: { [ORCHESTRATION_PROTOCOL_HEADER]: String(ORCHESTRATION_PROTOCOL_VERSION) },
256+
});
257+
assert.equal(
258+
threadSnapshot.status,
259+
200,
260+
Buffer.from(threadSnapshot.decodedBody).toString(),
261+
);
262+
assert.equal(threadSnapshot.contentEncoding, "gzip");
263+
const decodedThread = yield* decodeThreadSnapshot(
264+
Buffer.from(threadSnapshot.decodedBody).toString(),
265+
);
266+
assert.equal(decodedThread.projection.messages.length, TRANSFER_HISTORY_TURN_COUNT * 2);
267+
assert.equal(decodedThread.projection.turnItems.length, TRANSFER_HISTORY_TURN_COUNT * 6);
268+
assert.notInclude(Buffer.from(threadSnapshot.decodedBody).toString(), "digest=");
269+
const shellSnapshot = yield* measureHttpGet({
270+
url: `${base}/api/orchestration/shell`,
271+
headers: { [ORCHESTRATION_PROTOCOL_HEADER]: String(ORCHESTRATION_PROTOCOL_VERSION) },
272+
});
273+
assert.equal(shellSnapshot.status, 200);
274+
const decodedShell = yield* decodeShellSnapshot(
275+
Buffer.from(shellSnapshot.decodedBody).toString(),
276+
);
277+
assert.equal(decodedShell.threads.length, 1);
278+
const sequence = decodedThread.snapshotSequence;
279+
assert.equal(decodedShell.snapshotSequence, sequence);
280+
const url = `${base.replace("http:", "ws:")}/ws`;
281+
const thread = yield* openMeasuredWsClient({ url, cookie: "" });
282+
const shell = yield* openMeasuredWsClient({ url, cookie: "" });
283+
const second = yield* openMeasuredWsClient({ url, cookie: "" });
284+
assert.include(thread.recorder.negotiatedExtensions(), "permessage-deflate");
285+
const tq = yield* subscribe(thread, "thread", sequence);
286+
const sq = yield* subscribe(shell, "shell", sequence);
287+
const ttq = yield* subscribe(second, "thread", sequence);
288+
const ssq = yield* subscribe(second, "shell", sequence);
289+
for (const queue of [tq, sq, ttq, ssq])
290+
assert.equal(yield* synchronized(queue), "replay");
291+
const threadBefore = thread.recorder.totals();
292+
const shellBefore = shell.recorder.totals();
293+
const secondBefore = second.recorder.totals();
294+
const sqlBefore = counter.count();
295+
const events = turnEvents(provider, TRANSFER_HISTORY_TURN_COUNT, true);
296+
const stored = yield* sink.write({ events });
297+
const last = stored.at(-1)!;
298+
const threadTerminal = (item: StreamItem) =>
299+
"sequence" in item && item.sequence >= last.sequence;
300+
// Shell items carry the shell read when their batch is projected, and unchanged
301+
// shells are skipped, so a turn split across batches may never yield an item at
302+
// the last sequence. Wait for the final shell instead.
303+
const finalShell = yield* projection.getThreadShell(THREAD_ID);
304+
const shellTerminal = (item: StreamItem) =>
305+
item.kind === "thread.updated" && NodeUtil.isDeepStrictEqual(item.thread, finalShell);
306+
for (const queue of [tq, ttq]) yield* collectUntil(queue, threadTerminal);
307+
for (const queue of [sq, ssq]) yield* collectUntil(queue, shellTerminal);
308+
const measuredTurnWebSocket = difference(thread.recorder.totals(), threadBefore);
309+
const measuredTurnShellWebSocket = difference(shell.recorder.totals(), shellBefore);
310+
const measuredTurnSecondClientWebSocket = difference(
311+
second.recorder.totals(),
312+
secondBefore,
313+
);
314+
const measuredTurnSqlStatements = counter.count() - sqlBefore;
315+
const durable = yield* projection.getThreadProjection(THREAD_ID);
316+
assert.isTrue(
317+
durable.turnItems.some(
318+
(item) =>
319+
item.type === "dynamic_tool" && JSON.stringify(item.output).length > 1_000_000,
320+
),
321+
);
322+
yield* second.close;
323+
const reconnectSqlStart = counter.count();
324+
const resumedThread = yield* openMeasuredWsClient({ url, cookie: "" });
325+
const resumedShell = yield* openMeasuredWsClient({ url, cookie: "" });
326+
const rq = yield* subscribe(resumedThread, "thread", sequence);
327+
const rsq = yield* subscribe(resumedShell, "shell", sequence);
328+
const threadMode = yield* synchronized(rq);
329+
const shellMode = yield* synchronized(rsq);
330+
const reconnectThread = { ...resumedThread.recorder.totals(), mode: threadMode };
331+
const reconnectShell = { ...resumedShell.recorder.totals(), mode: shellMode };
332+
assert.equal(threadMode, "snapshot");
333+
return {
334+
provider,
335+
threadSnapshot,
336+
shellSnapshot,
337+
measuredTurnWebSocket,
338+
measuredTurnShellWebSocket,
339+
measuredTurnSecondClientWebSocket,
340+
measuredTurnSqlStatements,
341+
reconnectThread,
342+
reconnectShell,
343+
reconnectSqlStatements: counter.count() - reconnectSqlStart,
344+
} satisfies TransferBudgetRun;
345+
}).pipe(
346+
Effect.provide(Layer.fresh(layerServices).pipe(Layer.provideMerge(NodeServices.layer))),
347+
Effect.withTracer(counter.tracer),
348+
),
349+
);
350+
runs.push(run);
351+
}
352+
const fs = yield* FileSystem.FileSystem;
353+
const report = formatTransferBudgetReport(runs);
354+
for (const [path, contents] of [
355+
[process.env.T3CODE_TRANSFER_BUDGET_REPORT_PATH, report],
356+
[process.env.T3CODE_TRANSFER_BUDGET_RESULT_PATH, formatTransferBudgetResult(runs)],
357+
])
358+
if (path && contents) yield* fs.writeFileString(path, contents);
359+
yield* Effect.log(report);
360+
assert.deepEqual(transferBudgetViolations(runs), []);
361+
}).pipe(Effect.provide(NodeServices.layer)),
368362
);

0 commit comments

Comments
 (0)