Skip to content

Commit 092f465

Browse files
committed
🤖 Verify the whole battery at once, and prove it changed nothing
The checks were always independent; what stopped them running together was shared mutable state, and #304 removed it. This is the command that uses that, and the rule that keeps it true. `deno task verify` starts every applicable check concurrently after one setup, reports them in a fixed order however they finish, prints the first failure's output complete and names the rest, and fails if the battery moved any tracked file's content, mode, symlink target, or presence — including when a command failed, which is when a dirtied tree would otherwise go unnoticed. The split is enforced rather than intended. `scripts/lib/verify.ts` decides what runs and what the report says and is listed in `tsconfig.node.json`, so importing the Deno-only `prepared-state.ts` fails the Node typecheck; `scripts/verify.ts` installs the host behaviour it needs. The harness guarantees complete, byte-exact, non-interleaved output and observed cleanup. It does not slow a child down: `@effectionx/node`'s `fromReadable` subscribes with `on("data")`, so a pipe drains regardless of the consumer. What the guarantee rests on is a sink that cannot fall behind — one that suspends is halted at teardown with its backlog queued and those bytes gone — so the spool writes synchronously, and a test fails if it ever yields. On an unprepared worktree the preflight refuses: a fixture adapter's sentinel is never written, no battery command starts, `deno.lock` is byte-identical, and the exit is non-zero naming `deno task setup`. What the task runner creates while resolving the workspace beforehand is neither required nor asserted. Running the battery concurrently exposed a defect it also fails on serially: a document under test writes its eval blocks to `.xmd-eval/*.ts`, and the workspace `deno check` type-checked them, so `check` broke whenever `docs` had a file in flight. That directory joins the `exclude` list. `verify:clean` now ends by running the battery once, and holds it to a different claim than a build. A build is cache-pure: `build:web`, `build`, and the release compile must leave `node_modules`, the cache's dependency content, and `deno.lock` byte-identical, and that comparison is unchanged. Verification is not cache-pure and does not claim to be — it resolves modules no build walks, the dnt graph behind `scripts/build-npm.ts` among them, so it adds to the Deno cache the runtime owns. What it may never move is what this repository owns, so its phase compares `node_modules` and the lockfile. Once, and `--no-site`, because cost is a contract too: a second site-enabled run took the `composability` job from 1m38s to over an hour. The battery wants four cores and a CI runner has four, so two of them thrash rather than overlap, and the two extra commands are what the `site` job already runs. The cold-refusal test lost its scratch `DENO_DIR` for the same reason — unprepared means the worktree, not the cache, so inheriting a warm one costs six seconds instead of twenty-eight and proves exactly as much. Their scratch repositories also pin identity *and signing* on the command line: inheriting a global config that signs through an external agent made `git commit` exit 128 whenever that agent was busy. `verify:clean` now ends in `verify --no-site` and a site-enabled run, each followed by a prepared-state comparison, so the command that runs everything is no longer the one phase nobody checked. Closes #279
1 parent f0bf961 commit 092f465

15 files changed

Lines changed: 1847 additions & 21 deletions

‎AGENTS.md‎

Lines changed: 42 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -48,13 +48,46 @@ representative release target and proves the host tree and lock survived it,
4848
then runs every build phase offline — including a release compile for
4949
`x86_64-unknown-linux-gnu`, the same shape `release.yml` uses — fingerprinting
5050
the content and modes of `node_modules`, the cache's dependency roots, and
51-
`deno.lock` after each one. It verifies the commit, so commit before running it.
52-
CI runs the same harness in its `composability` job.
51+
`deno.lock` after each one. It then runs the battery itself, once.
52+
53+
Builds and checks are held to different claims there. A build is cache-pure:
54+
nothing it does may move `node_modules`, the cache's dependency content, or the
55+
lockfile. Verification is not, and does not pretend to be — the battery resolves
56+
modules no build walks, so it adds to the Deno cache, which the runtime owns.
57+
What verification may never move is what this repository owns: tracked files,
58+
`node_modules`, `deno.lock`, and another invocation's temporary state.
59+
60+
It verifies the commit, so commit before running it. CI runs the same harness in
61+
its `composability` job.
5362

5463
## Verification
5564

56-
After making any changes to source files (`src/`) or test files (`tests/`),
57-
always run all four checks before committing:
65+
One command runs everything that applies, concurrently:
66+
67+
```bash
68+
deno task verify # add --no-site to skip the site pair
69+
```
70+
71+
**The whole applicable battery is designed to run at once, after one setup.**
72+
That is a repository rule, not a convenience: no check may install, relink,
73+
patch, chmod, or clean a path another check reads, and any temporary state a
74+
check needs belongs to that invocation alone. A helper that reaches for
75+
repository-wide mutable state breaks every other check running beside it, which
76+
is how a `deno task build` came to break the Node typecheck (#279).
77+
78+
`verify` reports every command in a fixed order however they finish, prints the
79+
first failure's output complete and names the rest, and fails if the battery
80+
moved any tracked file's content, mode, symlink target, or presence — including
81+
when a command failed, because that is when a dirtied tree would otherwise go
82+
unnoticed. **Capture a failure's first output before re-running anything**: the
83+
report is what you paste, and a second run can hide the first.
84+
85+
The site pair applies when `site/` changed, judged from the branch and the
86+
worktree, both sides of a rename included.
87+
88+
Running the checks individually is still fine, and is what `verify` does for
89+
you. After making any changes to source files (`src/`) or test files (`tests/`),
90+
always run all four before committing:
5891

5992
1. **Lint + Format**: `deno task lint` (runs `oxlint` + `oxfmt --check`) — must
6093
produce 0 errors. Run `deno task fmt` to auto-fix formatting.
@@ -71,7 +104,11 @@ here:
71104

72105
- `check`, `test`, and `check:jsr` follow the `packages/*` workspace glob in the
73106
root `deno.json`. Its `exclude` list holds the paths that must stay
74-
unchecked — currently the deliberately-malformed `scripts/tests/fixtures`.
107+
unchecked: the deliberately-malformed `scripts/tests/fixtures`, and
108+
`.xmd-eval`, where a running document writes the `.ts` files its eval blocks
109+
compile to. Those are transient and belong to whichever command is executing a
110+
document — type-checking them fails on a half-written file, and fails the
111+
whole workspace check for a file nobody committed.
75112
- `test:node` and `test:bun` derive the same corpus through
76113
`scripts/lib/test-files.ts`, which walks `tests/` beneath each workspace
77114
member plus `scripts/tests/` — that boundary, and nothing else. A new

‎README.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -242,12 +242,14 @@ deno task build # compile the standalone xmd bi
242242
deno task lint # oxlint + oxfmt
243243
deno task check # typecheck
244244
deno task test # run the test suite
245+
deno task verify # the whole applicable battery, concurrently
245246
```
246247

247248
`deno task setup` is the only thing that installs. Builds and checks read what
248249
it prepared and write nothing back, so they compose instead of undoing one
249-
another — `deno task verify:clean` runs that claim end to end against a clean
250-
clone (AGENTS.md).
250+
another: `deno task verify` starts the entire applicable battery at once and
251+
fails if any of it leaves a tracked file changed, and `deno task verify:clean`
252+
runs that claim end to end against a clean clone (AGENTS.md).
251253

252254
## Status
253255

‎deno.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"workspace": ["packages/*", "site"],
3-
"exclude": ["scripts/tests/fixtures"],
3+
"exclude": ["scripts/tests/fixtures", ".xmd-eval"],
44
"nodeModulesDir": "auto",
55
"lock": {
66
"frozen": true
@@ -51,6 +51,7 @@
5151
"gen:publish-workflow": "deno run --allow-all packages/cli/src/deno.ts run scripts/gen-publish-workflow.md",
5252
"bump": "deno run -A scripts/bump-version.ts",
5353
"test": "deno test --allow-all --frozen",
54+
"verify": "deno run --allow-all --node-modules-dir=none --cached-only --frozen scripts/preflight.ts scripts/verify.ts",
5455
"check": "deno check --frozen",
5556
"check:jsr": "deno publish --dry-run --allow-dirty",
5657
"review": "deno run --allow-all packages/cli/src/deno.ts run .reviews/ReviewPR.md --component-dir .reviews/components --component-dir .reviews/policies --component-dir packages/core/components -j .reviews/journal.jsonl",

‎scripts/lib/changed-paths.ts‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
/**
2+
* Which paths this branch and worktree touched, from git's two answers.
3+
*
4+
* Verify asks one question of these: did anything under `site/` change? The
5+
* answer has to survive a rename, which is where the two sources stop agreeing.
6+
* Measured on this repository, renaming `site/page.md` to `moved-out.md`:
7+
*
8+
* ```text
9+
* git status --porcelain=v1 -z "R moved-out.md"\0"site/page.md"\0
10+
* ^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^
11+
* status and NEW path share one field, then old
12+
*
13+
* git diff --name-status -z -M -C "R100"\0"site/page.md"\0"moved-out.md"\0
14+
* ^^^^^ ^^^^^^^^^^^^^ ^^^^^^^^^^^^
15+
* status OLD path new path
16+
* ```
17+
*
18+
* The path order is reversed between them, so one parser cannot read both.
19+
* Both paths are always yielded: a rename out of `site/` changes `site/` just
20+
* as much as a rename into it does.
21+
*
22+
* Two more shapes came out of the same measurements. An *unstaged* rename is
23+
* not reported as a rename at all — git emits ` D site/page.md` and
24+
* `?? moved-out.md` — so nothing here may assume an `R` exists. And a copy does
25+
* reach porcelain when `status.renames=copies` is configured
26+
* (`C site/copy2.md\0keep.md\0`), so `C` is read, not guessed at.
27+
*
28+
* A status neither parser recognises raises. Reading one path where git meant
29+
* two would silently drop the side that mattered.
30+
*/
31+
32+
export class ChangedPathsError extends Error {}
33+
34+
/** Every status letter git may put in a porcelain field or a name-status record. */
35+
const STATUS_LETTERS = new Set([" ", "M", "A", "D", "R", "C", "U", "T", "X", "B", "?", "!"]);
36+
37+
function fields(output: string): string[] {
38+
return output.split("\0").filter((field) => field.length > 0);
39+
}
40+
41+
function known(letter: string): boolean {
42+
return STATUS_LETTERS.has(letter);
43+
}
44+
45+
/**
46+
* Paths from `git status --porcelain=v1 -z`.
47+
*
48+
* The status is a fixed two-character prefix, not a split: `slice(0, 2)` and
49+
* `slice(3)`, because a path may itself contain spaces.
50+
*/
51+
export function parsePorcelain(output: string): string[] {
52+
const parts = fields(output);
53+
const paths: string[] = [];
54+
let index = 0;
55+
while (index < parts.length) {
56+
const record = parts[index]!;
57+
index += 1;
58+
const status = record.slice(0, 2);
59+
const [staged, worktree] = [status[0]!, status[1]!];
60+
if (!known(staged) || !known(worktree)) {
61+
throw new ChangedPathsError(
62+
`git status reported \`${status}\`, which this parser cannot read`,
63+
);
64+
}
65+
paths.push(record.slice(3));
66+
if (staged === "R" || staged === "C" || worktree === "R" || worktree === "C") {
67+
const origin = parts[index];
68+
if (origin === undefined) {
69+
throw new ChangedPathsError(`git status reported \`${status}\` without its second path`);
70+
}
71+
index += 1;
72+
paths.push(origin);
73+
}
74+
}
75+
return paths;
76+
}
77+
78+
/** Paths from `git diff --name-status -z -M -C`. */
79+
export function parseNameStatus(output: string): string[] {
80+
const parts = fields(output);
81+
const paths: string[] = [];
82+
let index = 0;
83+
while (index < parts.length) {
84+
const status = parts[index]!;
85+
index += 1;
86+
const letter = status[0]!;
87+
if (!known(letter)) {
88+
throw new ChangedPathsError(`git diff reported \`${status}\`, which this parser cannot read`);
89+
}
90+
const wanted = letter === "R" || letter === "C" ? 2 : 1;
91+
for (let taken = 0; taken < wanted; taken += 1) {
92+
const path = parts[index];
93+
if (path === undefined) {
94+
throw new ChangedPathsError(`git diff reported \`${status}\` without ${wanted} paths`);
95+
}
96+
index += 1;
97+
paths.push(path);
98+
}
99+
}
100+
return paths;
101+
}
102+
103+
/** Whether any changed path lies under `directory`. */
104+
export function touches(paths: string[], directory: string): boolean {
105+
const prefix = directory.endsWith("/") ? directory : `${directory}/`;
106+
return paths.some((path) => path.startsWith(prefix));
107+
}

‎scripts/lib/tracked.ts‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
/**
2+
* What every tracked path looked like, so a battery can prove it changed none.
3+
*
4+
* A successful battery leaves tracked files byte-for-byte and mode-for-mode
5+
* unchanged (#279). Proving that needs a record per path and a comparison of
6+
* two records — not a "is the tree clean" question, which a worktree that was
7+
* already dirty when verify started would answer wrongly.
8+
*
9+
* ## Reading the index
10+
*
11+
* `git ls-files --stage -z` emits `<mode> <sha> <stage>\t<path>\0`. Two details
12+
* decide the parser, both measured:
13+
*
14+
* - **Only the first tab delimits.** A tracked filename may contain a tab —
15+
* `tab\tname.txt` arrives as `100644 <sha> 0\ttab\tname.txt` — so splitting
16+
* on every tab truncates the path. NUL delimits records, so a filename
17+
* containing a newline needs nothing special.
18+
* - **Mode `160000` is rejected before anything touches the filesystem.** A
19+
* gitlink whose working copy was emptied `lstat`s as a *directory*; one whose
20+
* directory was removed raises `ENOENT` and would be recorded as an absent
21+
* file, so an uninitialized submodule would compare equal to a deleted one.
22+
* Refusing on the recorded mode is what makes that impossible.
23+
*
24+
* A stage other than 0 is an unmerged path. A conflicted worktree is not a
25+
* state in which "nothing moved" means anything, so it is reported rather than
26+
* fingerprinted.
27+
*/
28+
29+
export class UnsupportedEntryError extends Error {}
30+
31+
export const GITLINK_MODE = "160000";
32+
33+
export interface TrackedRecord {
34+
mode: string;
35+
oid: string;
36+
path: string;
37+
}
38+
39+
export type TrackedEntry =
40+
| { kind: "file"; digest: string; executable: boolean }
41+
| { kind: "symlink"; target: string }
42+
| { kind: "absent" };
43+
44+
export type TrackedState = ReadonlyMap<string, TrackedEntry>;
45+
46+
/**
47+
* Records from `git ls-files --stage -z`, with the entries no fingerprint can
48+
* describe refused rather than skipped.
49+
*/
50+
export function parseStageRecords(output: string): TrackedRecord[] {
51+
const records: TrackedRecord[] = [];
52+
for (const field of output.split("\0")) {
53+
if (field.length === 0) {
54+
continue;
55+
}
56+
const tab = field.indexOf("\t");
57+
if (tab < 0) {
58+
throw new UnsupportedEntryError(`git ls-files emitted a record with no path: ${field}`);
59+
}
60+
const [mode, oid, stage] = field.slice(0, tab).split(" ");
61+
const path = field.slice(tab + 1);
62+
if (mode === undefined || oid === undefined || stage === undefined) {
63+
throw new UnsupportedEntryError(`git ls-files emitted incomplete metadata for ${path}`);
64+
}
65+
if (stage !== "0") {
66+
throw new UnsupportedEntryError(
67+
`${path} is unmerged (stage ${stage}); resolve the conflict before verifying`,
68+
);
69+
}
70+
if (mode === GITLINK_MODE) {
71+
throw new UnsupportedEntryError(
72+
`${path} is a submodule, which this fingerprint cannot describe`,
73+
);
74+
}
75+
records.push({ mode, oid, path });
76+
}
77+
return records;
78+
}
79+
80+
function describe(entry: TrackedEntry): string {
81+
if (entry.kind === "file") {
82+
return `${entry.digest.slice(0, 12)}${entry.executable ? " +x" : ""}`;
83+
}
84+
if (entry.kind === "symlink") {
85+
return `-> ${entry.target}`;
86+
}
87+
return "absent";
88+
}
89+
90+
/**
91+
* Every tracked path whose content, mode, symlink target, or presence moved.
92+
*
93+
* Both states cover the same paths by construction — the enumeration is the
94+
* index, not the filesystem — so a path missing from either is itself a change
95+
* worth naming.
96+
*/
97+
export function compareTracked(before: TrackedState, after: TrackedState): string[] {
98+
const moved: string[] = [];
99+
for (const [path, entry] of [...before].sort(([left], [right]) => left.localeCompare(right))) {
100+
const found = after.get(path);
101+
if (!found) {
102+
moved.push(`${path}: no longer tracked`);
103+
continue;
104+
}
105+
if (!same(entry, found)) {
106+
moved.push(`${path}: ${describe(entry)} -> ${describe(found)}`);
107+
}
108+
}
109+
for (const path of [...after.keys()].sort()) {
110+
if (!before.has(path)) {
111+
moved.push(`${path}: newly tracked`);
112+
}
113+
}
114+
return moved;
115+
}
116+
117+
function same(left: TrackedEntry, right: TrackedEntry): boolean {
118+
if (left.kind === "file" && right.kind === "file") {
119+
return left.digest === right.digest && left.executable === right.executable;
120+
}
121+
if (left.kind === "symlink" && right.kind === "symlink") {
122+
return left.target === right.target;
123+
}
124+
return left.kind === "absent" && right.kind === "absent";
125+
}

0 commit comments

Comments
 (0)