Skip to content

Commit ebb8a8e

Browse files
t3dotggclaude
andcommitted
docs(testing): new tests extend existing cases and stay at one layer
Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 8f0b729 commit ebb8a8e

4 files changed

Lines changed: 47 additions & 6 deletions

File tree

‎.coderabbit.config.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ export default defineConfig((ctx) => ({
2727
instructions: "Hold changed code to the rules in docs/internals/web-ui.md.",
2828
},
2929
{
30-
path: "**/*.test.{ts,tsx}",
30+
path: "**/*.test.*",
3131
instructions:
3232
"Hold changed tests to the rules in docs/internals/testing.md. Request tests only as those rules allow.",
3333
},
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
title: Test Signal
3+
model: gpt-6-sol
4+
effort: max
5+
input: incremental
6+
tools:
7+
- browse_code
8+
- modify_pr
9+
# An explicit include overrides ignore.md, which skips test files for every other check.
10+
include:
11+
- "**/*.test.*"
12+
exclude:
13+
- ".repos/**"
14+
requires:
15+
- Check
16+
maxBudgetPerRun: 2
17+
maxBudgetPerPR: 10
18+
conclusion: neutral
19+
---
20+
21+
# Test signal review
22+
23+
Review only the tests this pull request adds or grows. Tests it moves, renames, or deletes are not findings. The rules are in `docs/internals/testing.md` under `## What to test`; read that section first and hold new tests to it.
24+
25+
## Checklist
26+
27+
Flag a new or grown test when one of these is true:
28+
29+
- It tests what types already guarantee, trivial logic, copy text, a smoke run, mock calls that mirror the implementation, callback wiring, or props rendered to static markup.
30+
- An existing test or table in the same file already covers the behavior. The case belongs there as a row or an assertion.
31+
- The same case is also tested at another layer, for example in a service test and its wrapper test, at each call site of one helper, or in web and mobile copies of the same logic. Keep one copy at the lowest layer that fails on `main`.
32+
33+
## Reporting
34+
35+
Post each finding as a brief inline comment on the new test. Name the rule and give one fix: fold the case into the named test or table, delete it, or keep the copy at the named layer and delete the others. Ask only for fewer or smaller tests, never for more coverage.
36+
37+
When there are no findings, make the entire final response exactly `All clear` on one line with nothing else.

‎AGENTS.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -102,11 +102,13 @@ An empty database is a bad test. Seed your worktree's `.t3` with a copy of real
102102
- While you iterate, run `vp test run <files>`. For big files, add `-t "<name>"`.
103103
- Use `vp run typecheck`, not `npx tsc`; it caches per package. Both run tsc-rs (`typescript` is the `@tsc-rs/typescript` drop-in).
104104
- Run `node -e` snippets from `apps/server` or the owning package. `effect` does not resolve from the repo root.
105-
- Test meaningful logic or observable behavior. Do not render components to static markup to assert props or attributes, or add tests that merely assert callback wiring or mirror the implementation.
105+
- Test behavior that can break, such as orchestration and event-sourcing, provider protocol translation, auth and permission guards, persistence and migrations, external-format parsing, race fixes, and the logic a bug fix changed. Leave out tests for what types already guarantee, trivial logic, copy text, smoke runs, mock-call mirrors, callback wiring, and props rendered to static markup.
106+
- Extend before you add. When an existing test or table covers the behavior, add a row or an assertion there. Write a new `it` only when no case fits.
107+
- Test at one layer: the lowest one that fails on `main`. Do not repeat the case in a wrapper test, at each call site, or in web and mobile copies of the same logic.
106108
- [Testing](docs/internals/testing.md) carries these same test rules, plus review guidance, for review bots. Change both together.
107109
- **Do not run repo-wide checks.** No `vp check`, no `vp run -r test`, no `vp run -r typecheck` unless I ask. CI owns the full suite.
108110
- Backend behavior changes ship with focused tests for that behavior.
109-
- Proof for a bug fix depends on the bug. Logic, data, state, and protocol bugs get a failing test first (the fewest that fail on `main` for that reason, in the module's existing test file), then the fix. Purely visual bugs (spacing, colour, alignment, a layout shift) get before/after captures as their proof and no test: do not add UI tests for nitpicks, and do not ask for one in review when a capture already shows it. A server-only fix with no visible surface ships the test plus a short runtime transcript.
111+
- Proof for a bug fix depends on the bug. Logic, data, state, and protocol bugs get a failing test first (the fewest that fail on `main` for that reason, in an existing case of the module's test file where one fits), then the fix. Purely visual bugs (spacing, colour, alignment, a layout shift) get before/after captures as their proof and no test: do not add UI tests for nitpicks, and do not ask for one in review when a capture already shows it. A server-only fix with no visible surface ships the test plus a short runtime transcript.
110112
- The server is event-sourced, and side effects run after the command commits. In tests, drain the effect worker (`OrchestrationEffectWorkerV2.drain`) or await the specific persisted event or `Deferred` that marks the milestone. Never wait on sleeps or polling. A test that needs a timeout to pass is wrong. Concretely:
111113
- `it.effect` runs on a `TestClock`; `Effect.sleep` only advances through `TestClock.adjust`. `it.live` is wall-clock and only for subprocess or network fixtures.
112114
- No `yieldNow` or `sleep` polling loops. Await the `Deferred`, take from the `Queue`, or drain the worker.

‎docs/internals/testing.md‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,12 @@ Rules for writing tests, and for asking for them in review. AGENTS.md carries th
44

55
## What to test
66

7-
- Test meaningful logic or observable behavior. Do not render components to static markup to assert props or attributes, or add tests that merely assert callback wiring or mirror the implementation.
8-
- In review, ask for a missing test only when it would catch a real regression. Do not ask for one assertion per constant, label, or `switch` case.
7+
- Test behavior that can break, such as orchestration and event-sourcing, provider protocol translation, auth and permission guards, persistence and migrations, external-format parsing, race fixes, and the logic a bug fix changed. Leave out tests for what types already guarantee, trivial logic, copy text, smoke runs, mock-call mirrors, callback wiring, and props rendered to static markup.
8+
- Extend before you add. When an existing test or table covers the behavior, add a row or an assertion there. Write a new `it` only when no case fits.
9+
- Test at one layer: the lowest one that fails on `main`. Do not repeat the case in a wrapper test, at each call site, or in web and mobile copies of the same logic.
10+
- In review, ask for a missing test only when it would catch a real regression. Do not ask for one assertion per constant, label, or `switch` case. Flag a new test that breaks the rules above, and name the fix: fold it into the existing case or table, delete it, or keep one layer.
911
- Backend behavior changes ship with focused tests for that behavior.
10-
- Proof for a bug fix depends on the bug. Logic, data, state, and protocol bugs get a failing test first (the fewest that fail on `main` for that reason, in the module's existing test file), then the fix. Purely visual bugs (spacing, colour, alignment, a layout shift) get before/after captures as their proof and no test: do not add UI tests for nitpicks, and do not ask for one in review when a capture already shows it. A server-only fix with no visible surface ships the test plus a short runtime transcript.
12+
- Proof for a bug fix depends on the bug. Logic, data, state, and protocol bugs get a failing test first (the fewest that fail on `main` for that reason, in an existing case of the module's test file where one fits), then the fix. Purely visual bugs (spacing, colour, alignment, a layout shift) get before/after captures as their proof and no test: do not add UI tests for nitpicks, and do not ask for one in review when a capture already shows it. A server-only fix with no visible surface ships the test plus a short runtime transcript.
1113

1214
## Waiting for work
1315

0 commit comments

Comments
 (0)