Skip to content

Commit 0645252

Browse files
cliffhallclaude
andcommitted
docs(testing): the KNOWN BUG marker for tests that pin a bug
The characterization suites pin current behavior, bugs included. Document the one marker every bug-pinning test carries, so `git grep "KNOWN BUG"` separates them from tests of intended behavior and regression guards, and say what a fix PR and a newly found bug do with it. Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: cliffhall <[email protected]>
1 parent f227824 commit 0645252

1 file changed

Lines changed: 25 additions & 0 deletions

File tree

‎.claude/skills/testing/SKILL.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -201,6 +201,31 @@ do: `fetch` has `pytest-asyncio` with `asyncio_mode = "auto"`, so an
201201
coroutine in `anyio.run(...)` as above (`git`'s `serve()` test does), rather
202202
than adding a dependency for one test.
203203

204+
## Tests that pin a known bug
205+
206+
The suites are characterization tests: each asserts what the server does
207+
today, so a migration that changes behavior fails a test. Most of what they pin
208+
is intended. Some of it is a bug, pinned as it is so that the fix has to change
209+
the assertion visibly. Every test of that kind carries one marker, directly
210+
above the test (or above its `describe`/class when the whole block pins the
211+
same bug):
212+
213+
```ts
214+
// KNOWN BUG #4808: pins current (wrong) behavior; the fix changes this assertion.
215+
```
216+
217+
In Python the same text follows `#`. A bug with no issue yet reads
218+
`KNOWN BUG (no issue): <what is wrong>; …`, which also marks it for filing.
219+
220+
- `git grep "KNOWN BUG"` lists every test that pins a bug. A test without the
221+
marker pins intended behavior, or guards a fixed bug against regression.
222+
- **Fixing the bug** means changing that test's assertion to the correct
223+
behavior and removing the marker, in the fix PR.
224+
- **Pinning a newly found bug**: write the test against what the code does,
225+
add the marker, and file the issue (`/issue-create`), then cite it.
226+
- A pin of current design that a feature request wants changed is not a bug
227+
and gets no marker.
228+
204229
## Running them
205230

206231
| Scope | Command |

0 commit comments

Comments
 (0)