Skip to content

Commit d1a5085

Browse files
committed
fix(gate-manifest): only PR and scheduled lanes own a suite; fail closed on dynamic rules
Addresses the two findings from the re-review at f91453c. P1 — suite ownership accepted any lane. #1429 accepts "a PR workflow job, a scheduled workflow, or an explicit local-only declaration", but unownedTerminals credited every lane, so a test:*/check:* script reachable only from a release, push-to-main, or manually-dispatched workflow read as owned while gating nothing on the way in. OWNING_LANE_KINDS now restricts owners to pull-request and scheduled lanes, and the filter lives inside unownedTerminals so a caller cannot re-introduce the hole by passing every lane it happens to hold. laneKind needed fixing for that to mean anything: it mapped workflow_dispatch to 'scheduled', and nearly every release workflow carries dispatch as a manual escape hatch, so the release lanes would have been promoted straight back into owners. Dispatch is now its own 'manual' kind, checked last. No suite in the tree was relying on a non-qualifying lane, so this tightens the rule without changing today's verdict. Planted red: moving check:mcp-metadata to a release-only lane now reports exec:scripts/sync-mcp-metadata.mjs unowned. P2 — selectorRuleIds silently skipped anything it could not read, so a future `reason(..., someExpression, ...)` or computed BUILD_OWNERSHIP.rule would have slipped past the derived universe and therefore past both the representative- sample and reachability checks. Unreadable shapes now throw, naming the line. Two forwarding shapes are exempt because they provably cannot hide a category: the `reason` factory's own `{ check, path: file, rule, detail }` (a shorthand property forwarding a parameter, whose literal is at the call site), and `reason(entry.check, file, entry.rule, entry.detail)` in the build-ownership loop (whose literal is in the table this reader already collects). Anything else — a template string, a variable, a call — fails. Planted red: making one live `reason(...)` rule dynamic fails with its line number. Tests 33 → 40, including the push/release-only ownership case and five selector-rules cases, in a new selector-rules.test.ts mirroring its source. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ
1 parent f91453c commit d1a5085

7 files changed

Lines changed: 236 additions & 21 deletions

File tree

‎package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -129,7 +129,7 @@
129129
"check:affected": "node --experimental-strip-types scripts/check-affected/run.ts",
130130
"check:affected:test": "node --experimental-strip-types scripts/node-test-tmpdir.ts --experimental-strip-types --test scripts/check-affected/model.test.ts scripts/check-affected/platform-packages.test.ts scripts/check-affected/run.test.ts",
131131
"check:gate-manifest": "node --experimental-strip-types scripts/gate-manifest/check.ts",
132-
"check:gate-manifest:test": "node --experimental-strip-types scripts/node-test-tmpdir.ts --experimental-strip-types --test scripts/gate-manifest/shell-commands.test.ts scripts/gate-manifest/execution-terminals.test.ts scripts/gate-manifest/workflow-lanes.test.ts scripts/gate-manifest/suite-ownership.test.ts scripts/gate-manifest/catalog-wiring.test.ts scripts/gate-manifest/path-categories.test.ts scripts/gate-manifest/vitest-projects.test.ts",
132+
"check:gate-manifest:test": "node --experimental-strip-types scripts/node-test-tmpdir.ts --experimental-strip-types --test scripts/gate-manifest/shell-commands.test.ts scripts/gate-manifest/execution-terminals.test.ts scripts/gate-manifest/workflow-lanes.test.ts scripts/gate-manifest/suite-ownership.test.ts scripts/gate-manifest/catalog-wiring.test.ts scripts/gate-manifest/path-categories.test.ts scripts/gate-manifest/selector-rules.test.ts scripts/gate-manifest/vitest-projects.test.ts",
133133
"check:coverage-changed": "node --experimental-strip-types scripts/coverage-changed/run.ts",
134134
"check:coverage-changed:test": "node --experimental-strip-types scripts/node-test-tmpdir.ts --experimental-strip-types --test scripts/coverage-changed/model.test.ts scripts/coverage-changed/run.test.ts",
135135
"check:layering": "node --experimental-strip-types scripts/node-test-tmpdir.ts --experimental-strip-types --test scripts/layering/model.test.ts scripts/layering/zone-policy.test.ts scripts/layering/daemon-modularity.test.ts scripts/layering/package-boundaries.test.ts scripts/layering/platform-package-policy.test.ts scripts/layering/platform-package-repository.test.ts scripts/layering/platform-package-source-policy.test.ts scripts/layering/device-inventory-cutover-policy.test.ts scripts/layering/facade-exports.test.ts scripts/layering/bin-alias-fast-path.test.ts && node --experimental-strip-types scripts/layering/check.ts",
Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
// Deriving the affected-selector's category universe from its source.
2+
3+
import assert from 'node:assert/strict';
4+
import fs from 'node:fs';
5+
import path from 'node:path';
6+
import test from 'node:test';
7+
import { selectorRuleIds } from './selector-rules.ts';
8+
9+
const repoRoot = path.resolve(import.meta.dirname, '../..');
10+
const SELECTOR_SOURCE = 'scripts/check-affected/model.ts';
11+
12+
test('literal rules are collected from both reason() calls and the ownership table', () => {
13+
const source = `
14+
function reason(check, file, rule, detail) { return { check, path: file, rule, detail }; }
15+
const BUILD_OWNERSHIP = [{ check: 'swift-runner', rule: 'own:swift', detail: 'd', owns: () => true }];
16+
const a = reason('lint', file, 'gate:lint', 'oxlint covers the source tree');
17+
const b = BUILD_OWNERSHIP.map((entry) => reason(entry.check, file, entry.rule, entry.detail));
18+
`;
19+
assert.deepEqual(selectorRuleIds('m.ts', source), ['gate:lint', 'own:swift']);
20+
});
21+
22+
test('a fail-open literal is not a category, because it pairs rule: with path:', () => {
23+
const source = `
24+
const a = reason('lint', file, 'gate:lint', 'd');
25+
const open = { path: file, rule: 'workflow-tooling', detail: 'anything can change' };
26+
`;
27+
assert.deepEqual(selectorRuleIds('m.ts', source), ['gate:lint']);
28+
});
29+
30+
test('a computed rule argument fails closed rather than being skipped', () => {
31+
// A category this reader cannot see would bypass the representative-sample and reachability
32+
// checks entirely — the exact hole the derived universe exists to close.
33+
assert.throws(
34+
() => selectorRuleIds('m.ts', "const a = reason('lint', file, `own:${platform}`, 'd');"),
35+
/not statically readable[\s\S]*rule argument is not a string literal/,
36+
);
37+
assert.throws(
38+
() => selectorRuleIds('m.ts', "const a = reason('lint', file, RULE_ID, 'd');"),
39+
/rule argument is not a string literal/,
40+
);
41+
});
42+
43+
test('a computed ownership-table rule fails closed too', () => {
44+
assert.throws(
45+
() =>
46+
selectorRuleIds(
47+
'm.ts',
48+
"const T = [{ check: 'swift-runner', rule: buildRule('swift'), detail: 'd' }];",
49+
),
50+
/build-ownership `rule:` is not a string literal/,
51+
);
52+
});
53+
54+
test('the failure names the line so the selector can be fixed', () => {
55+
try {
56+
selectorRuleIds(
57+
'm.ts',
58+
"const ok = reason('lint', f, 'gate:lint', 'd');\nconst bad = reason('lint', f, X, 'd');",
59+
);
60+
assert.fail('expected a throw');
61+
} catch (error) {
62+
assert.match(String(error), /line 2/);
63+
}
64+
});
65+
66+
test("the repo's real selector still derives without hitting a fail-closed shape", () => {
67+
const rules = selectorRuleIds(
68+
SELECTOR_SOURCE,
69+
fs.readFileSync(path.join(repoRoot, SELECTOR_SOURCE), 'utf8'),
70+
);
71+
assert.ok(
72+
rules.length >= 20,
73+
`expected the live selector to yield its categories, got ${rules.length}`,
74+
);
75+
});

‎scripts/gate-manifest/selector-rules.ts‎

Lines changed: 68 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,8 @@ import {
1818
calleeName,
1919
children,
2020
collect,
21+
isNode,
22+
lineOf,
2123
literalText,
2224
parseProgram,
2325
propertyKey,
@@ -39,12 +41,38 @@ const RULE_ARGUMENT_INDEX = 2;
3941
* exclude them. Requiring a sibling `check:` separates the two by shape rather than by a
4042
* hand-maintained exclusion list, which is the thing this module exists to avoid.
4143
*/
42-
function buildOwnershipRule(node: Node): string | null {
43-
if (node.type !== 'ObjectExpression') return null;
44+
function buildOwnershipRule(node: Node): { rule: string | null; declared: boolean } {
45+
if (node.type !== 'ObjectExpression') return { rule: null, declared: false };
4446
const properties = children(node);
45-
if (!properties.some((property) => propertyKey(property) === 'check')) return null;
47+
if (!properties.some((property) => propertyKey(property) === 'check')) {
48+
return { rule: null, declared: false };
49+
}
4650
const rule = properties.find((property) => propertyKey(property) === 'rule');
47-
return rule ? stringValue(rule) : null;
51+
if (!rule) return { rule: null, declared: false };
52+
// `{ check, path: file, rule, detail }` — the `reason` factory forwarding its own parameter.
53+
// A shorthand property cannot name a category; the literal is at the call site, which this
54+
// reader visits separately. Declaring it "not declared" keeps it out of the fail-closed set
55+
// without weakening the rule for a genuinely dynamic value.
56+
if (rule['shorthand'] === true) return { rule: null, declared: false };
57+
return { rule: stringValue(rule), declared: true };
58+
}
59+
60+
/**
61+
* Whether a `reason(...)` rule argument merely forwards a rule already read from elsewhere.
62+
*
63+
* `reason(entry.check, file, entry.rule, entry.detail)` in the build-ownership loop is the one
64+
* live case: the literal lives in the BUILD_OWNERSHIP table, which this reader already
65+
* collects, so the forwarding call adds no category. Anything else non-literal — a template
66+
* string, a variable, a call — could name a category this reader would never see, and must
67+
* fail closed.
68+
*/
69+
function forwardsAKnownRule(argument: unknown): boolean {
70+
if (!isNode(argument)) return false;
71+
const isMemberAccess =
72+
argument.type === 'MemberExpression' || argument.type === 'StaticMemberExpression';
73+
if (!isMemberAccess) return false;
74+
const property = argument['property'];
75+
return isNode(property) && property['name'] === 'rule';
4876
}
4977

5078
/**
@@ -56,18 +84,51 @@ export function selectorRuleIds(file: string, source: string): string[] {
5684
const program = parseProgram(file, source);
5785
const rules = new Set<string>();
5886

87+
// Shapes this reader cannot turn into a category id. Silently skipping them would let a
88+
// future live category slip past the derived universe — and therefore past the
89+
// representative-sample and reachability checks — which is the whole hole this module
90+
// closes. So they fail closed, pointing at the line to fix.
91+
const unsupported: string[] = [];
92+
5993
collect(program, (node) => {
6094
if (calleeName(node) === REASON_FACTORY) {
6195
const args = node['arguments'];
62-
const literal = Array.isArray(args) ? literalText(args[RULE_ARGUMENT_INDEX]) : null;
63-
if (literal !== null) rules.add(literal);
96+
const argument = Array.isArray(args) ? args[RULE_ARGUMENT_INDEX] : undefined;
97+
const literal = literalText(argument);
98+
if (literal !== null) {
99+
rules.add(literal);
100+
return;
101+
}
102+
if (forwardsAKnownRule(argument)) return;
103+
unsupported.push(
104+
`line ${lineOf(source, isNode(argument) ? argument : node)}: \`${REASON_FACTORY}(...)\` ` +
105+
`rule argument is not a string literal`,
106+
);
64107
return;
65108
}
66109
// The build-ownership table states its rule as a property instead of a call argument.
67110
const owned = buildOwnershipRule(node);
68-
if (owned !== null) rules.add(owned);
111+
if (!owned.declared) return;
112+
if (owned.rule !== null) {
113+
rules.add(owned.rule);
114+
return;
115+
}
116+
unsupported.push(
117+
`line ${lineOf(source, node)}: build-ownership \`rule:\` is not a string literal`,
118+
);
69119
});
70120

121+
if (unsupported.length > 0) {
122+
throw new Error(
123+
`Cannot derive the path-category universe from ${file} — ` +
124+
`${unsupported.length} rule(s) are not statically readable:\n` +
125+
unsupported.map((entry) => ` - ${entry}`).join('\n') +
126+
`\nThe gate manifest needs every category id as a literal so it can check that a ` +
127+
`representative path exercises it. Write the rule as a string literal, or teach ` +
128+
`scripts/gate-manifest/selector-rules.ts the new shape.`,
129+
);
130+
}
131+
71132
if (rules.size === 0) {
72133
throw new Error(
73134
`Found no selection rule ids in ${file}. The gate manifest derives the path-category ` +

‎scripts/gate-manifest/suite-ownership.test.ts‎

Lines changed: 43 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,8 @@ test('a new vitest project no lane runs is unowned until it is wired or waived',
1010
const lanes = lanesFor(
1111
{
1212
'ci.yml':
13-
'jobs:\n cov:\n steps:\n - run: pnpm exec vitest run --project unit-core\n',
13+
'on:\n pull_request:\njobs:\n cov:\n' +
14+
' steps:\n - run: pnpm exec vitest run --project unit-core\n',
1415
},
1516
ctx,
1617
);
@@ -33,7 +34,7 @@ test('ownership is per unit of work, so a split aggregate is not reported as uno
3334
const lanes = lanesFor(
3435
{
3536
'ci.yml':
36-
'jobs:\n cov:\n steps:\n - run: pnpm test:unit\n' +
37+
'on:\n pull_request:\njobs:\n cov:\n steps:\n - run: pnpm test:unit\n' +
3738
' int:\n steps:\n - run: pnpm test:smoke\n',
3839
},
3940
ctx,
@@ -55,3 +56,43 @@ test('the suite universe is every vitest project plus every test:/check: script'
5556
['check:one', 'test:two', 'vitest:unit-core'],
5657
);
5758
});
59+
60+
test('a suite reachable only from a push or release lane is not owned', () => {
61+
// #1429 accepts a PR job or a scheduled workflow as an owner. A script that runs only on a
62+
// push to main, at release time, or on manual dispatch gates nothing on the way in, so
63+
// crediting those lanes would report a hole as covered.
64+
const ctx = context({
65+
vitestProjects: [],
66+
packageScripts: new Map([['check:published', 'node scripts/published.ts']]),
67+
});
68+
const nonOwning = lanesFor(
69+
{
70+
'release.yml':
71+
'name: R\non:\n release:\n types: [published]\n workflow_dispatch:\n' +
72+
'jobs:\n publish:\n steps:\n - run: pnpm check:published\n',
73+
'deploy.yml':
74+
'name: D\non:\n push:\n branches: [main]\n' +
75+
'jobs:\n deploy:\n steps:\n - run: pnpm check:published\n',
76+
},
77+
ctx,
78+
);
79+
assert.deepEqual(
80+
nonOwning.map((lane) => lane.kind).sort(),
81+
['push', 'release'],
82+
'workflow_dispatch beside release must not promote the lane to scheduled',
83+
);
84+
assert.deepEqual(unownedTerminals(suiteUniverse(ctx), nonOwning, new Set()), [
85+
{ terminal: 'exec:scripts/published.ts', suites: ['check:published'] },
86+
]);
87+
88+
// The same script reached from a scheduled lane is owned.
89+
const scheduled = lanesFor(
90+
{
91+
'nightly.yml':
92+
'name: N\non:\n schedule:\n - cron: "0 0 * * *"\n' +
93+
'jobs:\n nightly:\n steps:\n - run: pnpm check:published\n',
94+
},
95+
ctx,
96+
);
97+
assert.deepEqual(unownedTerminals(suiteUniverse(ctx), scheduled, new Set()), []);
98+
});

‎scripts/gate-manifest/suite-ownership.ts‎

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ import {
1212
type Terminal,
1313
type UnresolvedEdge,
1414
} from './execution-terminals.ts';
15-
import type { Lane } from './workflow-lanes.ts';
15+
import { OWNING_LANE_KINDS, type Lane } from './workflow-lanes.ts';
1616

1717
export type Suite = {
1818
/** `vitest:<project>` or the package script name. */
@@ -66,17 +66,28 @@ export type UnownedTerminal = {
6666
};
6767

6868
/**
69-
* Terminals some suite needs that no lane reaches. Ownership is per terminal rather than per
70-
* suite so an aggregate script (`check:tooling`, whose parts are spread across five jobs)
71-
* cannot report as unowned merely because no single job runs all of it.
69+
* Terminals some suite needs that no QUALIFYING lane reaches.
70+
*
71+
* Two rules are doing work here. Ownership is per terminal rather than per suite, so an
72+
* aggregate script (`check:tooling`, whose parts are spread across five jobs) is not reported
73+
* as unowned merely because no single job runs all of it. And only PR-triggered and scheduled
74+
* lanes count (OWNING_LANE_KINDS): a suite reachable only from a release, push-to-main, or
75+
* manually-dispatched workflow is NOT gating anything on the way in, so #1429 requires it to
76+
* be wired or classified local-only rather than quietly credited to that lane.
77+
*
78+
* The filter lives here rather than at the call site so a caller cannot re-introduce the hole
79+
* by passing every lane it happens to have.
7280
*/
7381
export function unownedTerminals(
7482
suites: readonly Suite[],
7583
lanes: readonly Lane[],
7684
waived: ReadonlySet<Terminal>,
7785
): UnownedTerminal[] {
7886
const owned = new Set<Terminal>();
79-
for (const lane of lanes) for (const terminal of lane.terminals) owned.add(terminal);
87+
for (const lane of lanes) {
88+
if (!OWNING_LANE_KINDS.has(lane.kind)) continue;
89+
for (const terminal of lane.terminals) owned.add(terminal);
90+
}
8091

8192
const needed = new Map<Terminal, string[]>();
8293
for (const suite of suites) {

‎scripts/gate-manifest/ts-ast.ts‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,16 @@ export function collect(node: Node, visit: (candidate: Node) => void): void {
6363
for (const child of children(node)) collect(child, visit);
6464
}
6565

66+
/** 1-based line of a node's start offset, for pointing a failure at real source. */
67+
export function lineOf(source: string, node: unknown): number {
68+
const start = isNode(node) && typeof node['start'] === 'number' ? node['start'] : 0;
69+
let line = 1;
70+
for (let index = 0; index < start && index < source.length; index++) {
71+
if (source[index] === '\n') line++;
72+
}
73+
return line;
74+
}
75+
6676
/** The name of a call expression's callee, when it is a plain identifier. */
6777
export function calleeName(node: Node): string | null {
6878
if (node.type !== 'CallExpression') return null;

‎scripts/gate-manifest/workflow-lanes.ts‎

Lines changed: 23 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -115,21 +115,38 @@ export type Lane = {
115115
readonly workflowName: string;
116116
readonly job: string;
117117
readonly checkNames: readonly string[];
118-
/** How the lane is triggered — a suite owned only by a scheduled lane is not a PR gate. */
119-
readonly kind: 'pull-request' | 'scheduled' | 'release' | 'push';
118+
/**
119+
* How the lane is triggered. #1429 accepts a PR workflow job or a scheduled workflow as an
120+
* owner; `release`, `push` and `manual` lanes do not qualify (see OWNING_LANE_KINDS) because
121+
* a suite that only runs at release time, on a push to main, or when somebody clicks the
122+
* button is not gating anything on the way in.
123+
*/
124+
readonly kind: 'pull-request' | 'scheduled' | 'release' | 'push' | 'manual';
120125
readonly terminals: ReadonlySet<Terminal>;
121126
readonly unresolved: readonly UnresolvedEdge[];
122127
};
123128

129+
/**
130+
* Ordered most to least gating. `workflow_dispatch` is checked LAST and maps to `manual`, not
131+
* `scheduled`: nearly every scheduled and release workflow also carries it as a manual escape
132+
* hatch, so treating it as a schedule would silently promote the release lanes into owners.
133+
*/
124134
function laneKind(triggers: WorkflowTriggers): Lane['kind'] {
125135
if (triggers.events.includes('pull_request')) return 'pull-request';
126-
if (triggers.events.includes('schedule') || triggers.events.includes('workflow_dispatch')) {
127-
return 'scheduled';
128-
}
136+
if (triggers.events.includes('schedule')) return 'scheduled';
129137
if (triggers.events.includes('release')) return 'release';
130-
return 'push';
138+
if (triggers.events.includes('push')) return 'push';
139+
return triggers.events.includes('workflow_dispatch') ? 'manual' : 'push';
131140
}
132141

142+
/**
143+
* Lane kinds that can own a suite, per #1429: "owned by a PR workflow job, a scheduled
144+
* workflow, or an explicit local-only declaration with a reason". A release/push/manual lane
145+
* runs too late or too rarely to be the thing keeping a check honest, so a suite reachable
146+
* only from one of those is unowned and must be wired or classified.
147+
*/
148+
export const OWNING_LANE_KINDS: ReadonlySet<Lane['kind']> = new Set(['pull-request', 'scheduled']);
149+
133150
/** Substitutes `${{ inputs.x }}` in a composite action step from the caller's `with:` block. */
134151
function substituteInputs(
135152
run: string,

0 commit comments

Comments
 (0)