Skip to content

Commit df0493a

Browse files
committed
test(ratchet): hold giant test files to their merge-base length so pin edits cannot admit growth
Review (P1): the equality pin compared measured lengths only against the pin map in the same checkout, so growing a file and raising its pin, or adding a new >1,000-line file with a pin, stayed green. The gate is now history-backed: every test file over the tripwire may be no longer than at the merge-base with origin/main (renames followed; new files may not cross the line), and no pin may exceed its file's base length — one git cat-file --batch spawn, parsed by bytes because the sizes are bytes. Both bypasses planted red against real git on a pinned file and on a fresh 1,001-line file with a pin added.
1 parent 0e557c3 commit df0493a

2 files changed

Lines changed: 189 additions & 4 deletions

File tree

‎docs/agents/testing.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -549,9 +549,12 @@ Measured on the full unit suite (340 files, 3,210 tests, 48s wall at ~7x paralle
549549
shape for the other resource a giant test file consumes — a reader's context. Every test file
550550
over the 1,000-line tripwire is pinned at its exact length, R9-style: growth fails ("split it
551551
along the source module it mirrors"), shrinking fails until the pin is lowered in the same PR,
552-
a file that drops under the line leaves the list, and a new file may not cross it. Adding a
553-
test to a pinned file means moving that family out first — the failure names the file and the
554-
fix; never raise a pin.
552+
a file that drops under the line leaves the list, and a new file may not cross it. The map is
553+
not the authority — git is: every file over the line is also held to its length at the
554+
merge-base with `origin/main` (renames followed), and no pin may exceed its file's base length,
555+
so growing a file and raising its pin, or adding a giant file with a pin, are red against
556+
history. Adding a test to a pinned file means moving that family out first — the failure names
557+
the file and the fix; never raise a pin. Needs `origin/main` fetched (CI's Coverage job does).
555558
- **Isolation stays ON; pool stays forks — both measured.** `--no-isolate`: 205s wall vs 48s
556559
(module state — timers, memos, singletons — thrashes across files sharing a worker).
557560
`--pool=threads`: no change (50.4s). The ~100s aggregate import overhead is the price of

‎src/__tests__/test-file-size-ratchet.test.ts‎

Lines changed: 183 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ import fs from 'node:fs';
22
import path from 'node:path';
33
import { expect, test } from 'vitest';
44
import { walkFiles } from '../../scripts/lib/walk-files.ts';
5+
import { runCmdSync } from '../utils/exec.ts';
56

67
/**
78
* Test-file size ratchet (AGENTS.md "Scope & shape": past 1,000 lines is architecture debt,
@@ -14,7 +15,14 @@ import { walkFiles } from '../../scripts/lib/walk-files.ts';
1415
* shrinking one fails until the pin is lowered, so the list only ever ratchets down. A file
1516
* that drops under the tripwire leaves the list; a new file may not cross it.
1617
*
17-
* Catches: a >1,000-line test file growing, or a new one appearing.
18+
* The pin map alone could be edited alongside the file (raise a pin and grow into it; add a
19+
* pin with a new giant file), so the gate is history-backed as well: every test file over the
20+
* tripwire may be no longer than it was at the merge-base with origin/main (or no longer than
21+
* the tripwire if it did not exist there), and no pin may exceed its file's base length. Both
22+
* pin-edit bypasses go red against git, not against the map.
23+
*
24+
* Catches: a >1,000-line test file growing (with or without a matching pin edit), or a new one
25+
* appearing (with or without a pin).
1826
* Evidence: 26 test files were over the line when this landed (2026-08-18); the largest,
1927
* `snapshot-handler.test.ts`, gained 55 lines in the PR before, under a rule with no gate.
2028
* Cost: one directory walk and a line count per test file — well under a second.
@@ -110,6 +118,108 @@ function ratchetFindings(
110118
return findings;
111119
}
112120

121+
/**
122+
* Line counts of the given repo paths at the merge-base with origin/main, following renames, in
123+
* one `git cat-file --batch` spawn. `undefined` = the file did not exist there.
124+
*/
125+
function baseLineCounts(paths: readonly string[]): ReadonlyMap<string, number | undefined> {
126+
const mergeBase = runCmdSync('git', ['merge-base', 'origin/main', 'HEAD'], {
127+
cwd: REPO_ROOT,
128+
allowFailure: true,
129+
});
130+
if (mergeBase.exitCode !== 0) {
131+
throw new Error(
132+
'test-file size ratchet needs origin/main to read base lengths (git merge-base origin/main HEAD failed): ' +
133+
`${mergeBase.stderr.trim()}. Fetch origin/main; the gate does not skip.`,
134+
);
135+
}
136+
const base = mergeBase.stdout.trim();
137+
const renamedFrom = new Map<string, string>();
138+
const renames = runCmdSync(
139+
'git',
140+
['diff', '--name-status', '--find-renames', '--diff-filter=R', base, 'HEAD', '--', '*.test.*'],
141+
{ cwd: REPO_ROOT },
142+
);
143+
for (const line of renames.stdout.split('\n')) {
144+
const [, from, to] = line.split('\t');
145+
if (from && to) renamedFrom.set(to, from);
146+
}
147+
const requests = paths.map((file) => `${base}:${renamedFrom.get(file) ?? file}`);
148+
const batch = runCmdSync('git', ['cat-file', '--batch'], {
149+
cwd: REPO_ROOT,
150+
stdin: `${requests.join('\n')}\n`,
151+
binaryStdout: true,
152+
maxBuffer: 256 * 1024 * 1024,
153+
});
154+
return parseCatFileBatch(batch.stdoutBuffer ?? Buffer.alloc(0), paths);
155+
}
156+
157+
/**
158+
* `<sha> blob <size>\n<size bytes>\n` per hit, `<request> missing\n` per miss, in request order.
159+
* Sizes are bytes, so this walks the raw buffer: a string offset drifts after the first file with
160+
* a multi-byte character (every test file with an em dash).
161+
*/
162+
function parseCatFileBatch(
163+
output: Buffer,
164+
paths: readonly string[],
165+
): ReadonlyMap<string, number | undefined> {
166+
const counts = new Map<string, number | undefined>();
167+
let offset = 0;
168+
for (const file of paths) {
169+
const headerEnd = output.indexOf(0x0a, offset);
170+
const header = output.subarray(offset, headerEnd).toString('utf8');
171+
offset = headerEnd + 1;
172+
const blob = /^\S+ blob (\d+)$/.exec(header);
173+
if (!blob) {
174+
counts.set(file, undefined); // "<request> missing"
175+
continue;
176+
}
177+
const size = Number(blob[1]);
178+
let lines = 0;
179+
for (let index = offset; index < offset + size; index += 1) {
180+
if (output[index] === 0x0a) lines += 1;
181+
}
182+
offset += size + 1;
183+
counts.set(file, lines);
184+
}
185+
return counts;
186+
}
187+
188+
/**
189+
* The history-backed half: measured against the merge-base, not against the pin map, so
190+
* editing the map alongside the file cannot admit growth.
191+
*/
192+
function historyFindings(
193+
measured: ReadonlyMap<string, number>,
194+
pinned: Readonly<Record<string, number>>,
195+
baseLines: ReadonlyMap<string, number | undefined>,
196+
tripwire: number,
197+
): string[] {
198+
const findings: string[] = [];
199+
for (const [file, lines] of [...measured].sort()) {
200+
if (lines <= tripwire) continue;
201+
const base = baseLines.get(file);
202+
if (base === undefined) {
203+
findings.push(
204+
`${file} is ${lines} lines and did not exist at the merge-base: a new test file may not cross the ${tripwire}-line tripwire, pinned or not.`,
205+
);
206+
} else if (lines > Math.max(base, tripwire)) {
207+
findings.push(
208+
`${file} is ${lines} lines, ${base} at the merge-base: a test file over the tripwire may not grow, whatever its pin says.`,
209+
);
210+
}
211+
}
212+
for (const [file, pin] of Object.entries(pinned).sort()) {
213+
const base = baseLines.get(file);
214+
if (base !== undefined && pin > base) {
215+
findings.push(
216+
`${file} is pinned at ${pin} but was ${base} lines at the merge-base: a pin may not be raised above its file's base length.`,
217+
);
218+
}
219+
}
220+
return findings;
221+
}
222+
113223
test('no test file over the tripwire grows, and every pin matches its file exactly', () => {
114224
const measured = new Map<string, number>();
115225
for (const root of TEST_ROOTS) {
@@ -119,6 +229,15 @@ test('no test file over the tripwire grows, and every pin matches its file exact
119229
}
120230
expect(measured.size).toBeGreaterThan(500);
121231
expect(ratchetFindings(measured, PINNED_TEST_FILE_LINES, TRIPWIRE_LINES)).toEqual([]);
232+
233+
const ofInterest = [
234+
...new Set([
235+
...[...measured].filter(([, lines]) => lines > TRIPWIRE_LINES).map(([file]) => file),
236+
...Object.keys(PINNED_TEST_FILE_LINES),
237+
]),
238+
];
239+
const baseLines = baseLineCounts(ofInterest);
240+
expect(historyFindings(measured, PINNED_TEST_FILE_LINES, baseLines, TRIPWIRE_LINES)).toEqual([]);
122241
});
123242

124243
test('planted reds: growth, shrink, unpinned crossing, and a stale pin each name their fix', () => {
@@ -137,3 +256,66 @@ test('planted reds: growth, shrink, unpinned crossing, and a stale pin each name
137256
]);
138257
expect(ratchetFindings(new Map([['a.test.ts', 1200]]), { 'a.test.ts': 1200 }, 1000)).toEqual([]);
139258
});
259+
260+
test('planted reds against history: raising a pin, growing into it, and pinning a new giant file are all red', () => {
261+
const baseLines = new Map<string, number | undefined>([
262+
['a.test.ts', 1200], // existed, 1200 at base
263+
['b.test.ts', 1500],
264+
['fresh.test.ts', undefined], // did not exist at base
265+
['small.test.ts', 900], // existed, under the tripwire at base
266+
]);
267+
// Bypass 1: grow a pinned file and raise its pin so the equality pin stays green.
268+
expect(
269+
historyFindings(new Map([['a.test.ts', 1230]]), { 'a.test.ts': 1230 }, baseLines, 1000),
270+
).toEqual([
271+
'a.test.ts is 1230 lines, 1200 at the merge-base: a test file over the tripwire may not grow, whatever its pin says.',
272+
"a.test.ts is pinned at 1230 but was 1200 lines at the merge-base: a pin may not be raised above its file's base length.",
273+
]);
274+
// Raising the pin alone (before growing into it) is already red.
275+
expect(
276+
historyFindings(new Map([['a.test.ts', 1200]]), { 'a.test.ts': 1230 }, baseLines, 1000),
277+
).toEqual([
278+
"a.test.ts is pinned at 1230 but was 1200 lines at the merge-base: a pin may not be raised above its file's base length.",
279+
]);
280+
// Bypass 2: add a new >1,000-line file together with a pin for it.
281+
expect(
282+
historyFindings(new Map([['fresh.test.ts', 1400]]), { 'fresh.test.ts': 1400 }, baseLines, 1000),
283+
).toEqual([
284+
'fresh.test.ts is 1400 lines and did not exist at the merge-base: a new test file may not cross the 1000-line tripwire, pinned or not.',
285+
]);
286+
// Same for a file that existed but was under the tripwire at base.
287+
expect(
288+
historyFindings(new Map([['small.test.ts', 1001]]), { 'small.test.ts': 1001 }, baseLines, 1000),
289+
).toEqual([
290+
'small.test.ts is 1001 lines, 900 at the merge-base: a test file over the tripwire may not grow, whatever its pin says.',
291+
"small.test.ts is pinned at 1001 but was 900 lines at the merge-base: a pin may not be raised above its file's base length.",
292+
]);
293+
// Allowed: shrink with a lowered pin, a pin that disappears, an unchanged file.
294+
expect(
295+
historyFindings(
296+
new Map([
297+
['a.test.ts', 1100],
298+
['b.test.ts', 1500],
299+
]),
300+
{ 'a.test.ts': 1100, 'b.test.ts': 1500 },
301+
baseLines,
302+
1000,
303+
),
304+
).toEqual([]);
305+
expect(historyFindings(new Map([['a.test.ts', 950]]), {}, baseLines, 1000)).toEqual([]);
306+
});
307+
308+
test('cat-file --batch output is parsed per request, in order, with misses as undefined', () => {
309+
// The dash is 3 bytes in UTF-8: the parser must count by bytes, not characters.
310+
const dashed = Buffer.from('a — b\nc\n', 'utf8');
311+
const output = Buffer.concat([
312+
Buffer.from(`abc blob ${dashed.length}\n`),
313+
dashed,
314+
Buffer.from('\nHEAD:missing.ts missing\ndef blob 6\nx\ny\nz\n\n'),
315+
]);
316+
expect([...parseCatFileBatch(output, ['dashed.ts', 'missing.ts', 'xyz.ts'])]).toEqual([
317+
['dashed.ts', 2],
318+
['missing.ts', undefined],
319+
['xyz.ts', 3],
320+
]);
321+
});

0 commit comments

Comments
 (0)