Skip to content

Commit 517262c

Browse files
committed
test(ci): replace the gate manifest with a canonical CheckId runner (#1429)
Rebuilds #1429's guarantee on a deeper interface instead of a wider one. The manifest no longer interprets workflows, shell commands, Vitest configuration, TypeScript expressions and waiver provenance to work out what CI runs. Instead: CI may invoke a gate only through `pnpm gate <check-id>`. That single constraint turns the workflow→check mapping into a token scan, and makes an unregistered gate impossible to wire up rather than invisible until someone declares it. `GATE_RUNNERS` and its staleness tests are gone with it — fuzz, mutation and the Maestro differential are ordinary registered checks. One universe: `CHECK_CATALOG` is both the affected-selector's vocabulary and the set of gates CI runs (47 checks). `ciJobs` is derived from the workflows rather than declared, so the "GitHub-authoritative" pointer cannot go stale. One primitive: `covered(check, path)` — some pull_request/schedule lane runs every unit of the check, and a change to that path starts the lane. Units are Vitest projects and `node --test` files, so a lane running one file of a project cannot stand in for the project, and command-docs (now a real CheckId) holds the #1420 docs lane with no docs-specific code. Preserves the CI wiring fixes #1714 found: check:mcp-metadata in Typecheck, check:tmpdir-leaks in Coverage, its model tests in Layering Guard, test:fixture-cache in Integration Tests, command-docs-gate as a checked docs lane. Tests are mutations of the real model, not fixtures over home-grown parsers: the two holes that mattered in the previous design were green in all 75 of those. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ
1 parent aec9df2 commit 517262c

54 files changed

Lines changed: 1777 additions & 4025 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/actions/setup-android-replay-host/action.yml‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -68,10 +68,7 @@ runs:
6868
- name: Package npm-bundled Android helpers
6969
if: inputs.package-helpers == 'true' && steps.android-helpers-cache.outputs.cache-hit != 'true'
7070
run: |
71-
VERSION="${{ steps.android-helper-source.outputs.version }}"
72-
rm -rf android/snapshot-helper/dist android/ime-helper/dist
73-
sh ./scripts/package-android-helper.sh snapshot "$VERSION" "v$VERSION" android/snapshot-helper/dist
74-
sh ./scripts/package-android-helper.sh ime "$VERSION" android/ime-helper/dist
71+
pnpm gate android-helpers
7572
shell: bash
7673

7774
- name: Verify packaged Android helpers

‎.github/workflows/android.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ jobs:
6666
set -eu
6767
ANDROID_SERIAL="$(adb devices | awk 'NR > 1 && $2 == "device" { print $1; exit }')"
6868
test -n "$ANDROID_SERIAL"
69-
pnpm build
69+
pnpm gate build
7070
pnpm clean:daemon
7171
AGENT_DEVICE_ANDROID_E2E=1 AGENT_DEVICE_ANDROID_E2E_TIER=smoke AGENT_DEVICE_ANDROID_SERIAL="$ANDROID_SERIAL" AGENT_DEVICE_FIXTURE_APP_PATH="${{ steps.fixture-app.outputs.apk-path }}" AGENT_DEVICE_FIXTURE_APP_ID="${{ steps.fixture-app.outputs.app-id }}" node --experimental-strip-types scripts/node-test-tmpdir.ts --test test/integration/smoke-android-emulator.test.ts
7272
node --experimental-strip-types src/bin.ts test test/integration/replays/android/01-settings.ad --retries 2 --report-junit test/artifacts/replays-android-smoke.junit.xml

‎.github/workflows/ci.yml‎

Lines changed: 31 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ jobs:
6565
with:
6666
derived-path: ${{ github.workspace }}/.tmp/swift-runner-unit-derived
6767
cache-key-prefix: swift-runner-unit
68-
build-command: AGENT_DEVICE_XCUITEST_INCLUDE_UNIT_TESTS=1 pnpm build:xcuitest:macos
68+
build-command: AGENT_DEVICE_XCUITEST_INCLUDE_UNIT_TESTS=1 pnpm gate swift-runner-macos
6969
xcuitest-platform: macos
7070
xcuitest-destination: platform=macOS,arch=arm64
7171

@@ -81,10 +81,10 @@ jobs:
8181
uses: ./.github/actions/setup-node-pnpm
8282

8383
- name: Run oxlint
84-
run: pnpm lint
84+
run: pnpm gate lint
8585

8686
- name: Check formatting
87-
run: pnpm format:check
87+
run: pnpm gate format
8888

8989
layering-guard:
9090
name: Layering Guard
@@ -107,14 +107,14 @@ jobs:
107107
# Generalizes the former inline commands/-import grep into a structured
108108
# import-direction lint over the resolved graph. See scripts/layering/check.ts
109109
# and CONTEXT.md (Architecture: folder DAG + layering lint).
110-
run: pnpm check:layering
110+
run: pnpm gate layering
111111

112112
- name: Check the depgraph report agrees with the gate
113113
# scripts/depgraph reads the same model as the gate, so its inversion count must
114114
# reproduce TYPE_INVERSION_BASELINE. Free two-sources check: if the tree changes
115115
# and only one side is updated, this fails and names the difference. Runs here
116116
# rather than in its own job so the two can never be green independently.
117-
run: pnpm depgraph:test
117+
run: pnpm gate depgraph
118118

119119
# The TMPDIR redirection both test lanes rely on (#1593/#1595). Its own tests ran
120120
# in no CI job at all until the #1429 gate manifest found them unowned.
@@ -128,7 +128,7 @@ jobs:
128128
# with nothing to gain — the manifest proves ownership per unit of work, so any
129129
# lane satisfies it.
130130
- name: Check the tmpdir redirection model
131-
run: pnpm check:tmpdir-leaks:test
131+
run: pnpm gate tmpdir-leaks-model
132132

133133
affected-selector:
134134
name: Affected-check Selector
@@ -149,18 +149,18 @@ jobs:
149149
# The selector is fail-open and advisory (GitHub CI stays authoritative),
150150
# so the gate only guards the derivation model.
151151
- name: Check affected-selector model
152-
run: pnpm check:affected:test
152+
run: pnpm gate affected-selector
153153

154154
# The gate-of-gates (#1429). It shares this job because it validates the
155155
# same artifact the selector is built on — CHECK_CATALOG's `ciJobs` — and
156156
# because a gate that proves the other gates are wired must not be the one
157157
# gate sitting in its own job, green on its own. Deterministic and
158158
# network-free: every input is a file in the checkout.
159159
- name: Check the gate manifest model
160-
run: pnpm check:gate-manifest:test
160+
run: pnpm gate gate-manifest-model
161161

162162
- name: Check every gate is owned, wired, and reachable
163-
run: pnpm check:gate-manifest
163+
run: pnpm gate gate-manifest
164164

165165
maestro-conformance:
166166
name: Maestro Conformance Oracle
@@ -182,7 +182,7 @@ jobs:
182182
# device-backed layer 3 runs on the scheduled conformance-differential
183183
# workflow. See scripts/maestro-conformance/README.md.
184184
- name: Verify Maestro conformance fixtures
185-
run: pnpm maestro:conformance
185+
run: pnpm gate maestro-conformance
186186

187187
packaged-cli-node-22-12:
188188
name: Packaged CLI Node 22.12
@@ -197,8 +197,8 @@ jobs:
197197

198198
- name: Build CLI
199199
run: |
200-
pnpm build
201-
pnpm check:bundle-owner-files
200+
pnpm gate build
201+
pnpm gate bundle-owner-files
202202
203203
# The build runs on the default toolchain Node and the package is verified on the minimum
204204
# supported Node, so this job covers what a user on `engines.node` floor actually installs.
@@ -232,10 +232,10 @@ jobs:
232232
- name: Run Fallow audit
233233
env:
234234
FALLOW_BASE: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
235-
run: pnpm check:fallow --base "$FALLOW_BASE"
235+
run: pnpm gate fallow --base "$FALLOW_BASE"
236236

237237
- name: Check for production-unused exports
238-
run: pnpm check:production-exports
238+
run: pnpm gate production-exports
239239

240240
replay-compat-provenance:
241241
# The frozen replay-compat corpus (#1417) claims each entry was published by
@@ -255,7 +255,7 @@ jobs:
255255
uses: ./.github/actions/setup-node-pnpm
256256

257257
- name: Verify corpus entries against their released blobs
258-
run: pnpm check:replay-compat
258+
run: pnpm gate replay-compat
259259

260260
released-surface-compat:
261261
# The daemon RPC wire ledger (#1432) is compared against the ledger as it
@@ -276,10 +276,10 @@ jobs:
276276
uses: ./.github/actions/setup-node-pnpm
277277

278278
- name: Verify the wire-compat rules
279-
run: pnpm check:daemon-wire-compat:test
279+
run: pnpm gate wire-compat-model
280280

281281
- name: Compare the daemon RPC wire surface against the last released tag
282-
run: pnpm check:daemon-wire-compat
282+
run: pnpm gate daemon-wire-compat
283283

284284
coverage:
285285
# Runs the full unit + provider-integration suites under coverage with
@@ -297,20 +297,20 @@ jobs:
297297
uses: ./.github/actions/setup-node-pnpm
298298

299299
- name: Test changed-line coverage gate
300-
run: pnpm check:coverage-changed:test
300+
run: pnpm gate coverage-model
301301

302302
# The retry list is an enumerated set of owned waivers, so an expired entry
303303
# must fail before the suite runs rather than quietly keeping its retry.
304304
- name: Check contention retry policy
305-
run: pnpm check:contention-retry
305+
run: pnpm gate contention-retry
306306

307307
# Wrapped in the single-retry policy (#1419): a timeout-shaped failure in
308308
# an enumerated contention-flaky file reruns that file once and reports it
309309
# in the job summary. Assertion failures fail here on the first run.
310310
- name: Run coverage
311311
env:
312312
OUTPUT_ECONOMY_BASE: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
313-
run: pnpm test:coverage:ci
313+
run: pnpm gate unit-ci
314314

315315
# A run whose globalTeardown never fired (crash, OOM, timeout kill) leaves
316316
# its tmpdir behind. `check:unit` runs this locally after the suite; until
@@ -319,7 +319,7 @@ jobs:
319319
# worth reporting.
320320
- name: Check for leaked test tmpdirs
321321
if: always()
322-
run: pnpm check:tmpdir-leaks
322+
run: pnpm gate tmpdir-leaks
323323

324324
- name: Upload contention-retry envelope
325325
if: always()
@@ -337,7 +337,7 @@ jobs:
337337
if: always() && github.event_name == 'pull_request'
338338
env:
339339
AGENT_DEVICE_COVERAGE_WAIVER: ${{ contains(github.event.pull_request.labels.*.name, 'coverage-waiver') }}
340-
run: pnpm check:coverage-changed --base "${{ github.event.pull_request.base.sha }}"
340+
run: pnpm gate coverage --base "${{ github.event.pull_request.base.sha }}"
341341

342342
typecheck:
343343
name: Typecheck
@@ -351,14 +351,14 @@ jobs:
351351
uses: ./.github/actions/setup-node-pnpm
352352

353353
- name: Run typecheck
354-
run: pnpm typecheck
354+
run: pnpm gate typecheck
355355

356356
# CHECK_CATALOG has always claimed this job mirrors the `mcp-metadata`
357357
# check; until #1429 checked the claim, no PR job ran it at all, so
358358
# server.json/smithery.yaml drift only surfaced at publish time (where
359359
# publish-mcp-registry.yml duplicates the same command). Parse-only.
360360
- name: Check MCP registry metadata is in sync
361-
run: pnpm check:mcp-metadata
361+
run: pnpm gate mcp-metadata
362362

363363
freerange:
364364
name: FreeRange
@@ -375,7 +375,7 @@ jobs:
375375
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
376376

377377
- name: Check numeric ranges
378-
run: pnpm check:freerange
378+
run: pnpm gate freerange
379379

380380
integration:
381381
name: Integration Tests
@@ -391,21 +391,21 @@ jobs:
391391
- name: Run integration tests
392392
run: |
393393
pnpm clean:daemon
394-
pnpm test:integration:node
394+
pnpm gate integration-node
395395
396396
- name: Run seeded concurrency torture lane (fast PR sweep)
397397
# #1416's nightly torture lane lives under test/integration/nightly/, out
398398
# of the test:integration:node glob, so this is a *deliberate* fast PR
399399
# sweep (TORTURE_RUNS default 128 seeds, ~sub-second) — not an accidental
400400
# glob inclusion. The Concurrency Torture Nightly workflow sweeps a much
401401
# larger seed range on schedule.
402-
run: pnpm test:concurrency-torture
402+
run: pnpm gate concurrency-torture
403403

404404
- name: Run provider-backed integration tests
405-
run: pnpm test:integration:provider
405+
run: pnpm gate provider-integration
406406

407407
- name: Check Provider-backed integration architecture progress
408-
run: pnpm test:integration:progress:check
408+
run: pnpm gate integration-progress
409409

410410
# A build-cache lookup outage must degrade setup-fixture-app to an inline
411411
# build, not fail the caller. This drives that step's real shell against a
@@ -417,7 +417,7 @@ jobs:
417417
# reused at all — same subject as the fallback smoke above, and parse-only.
418418
# Ran in no CI job until the #1429 gate manifest found it unowned.
419419
- name: Check trusted fixture-artifact selection
420-
run: pnpm test:fixture-cache
420+
run: pnpm gate fixture-cache
421421

422422
web-smoke:
423423
name: Web Platform Smoke
@@ -437,7 +437,7 @@ jobs:
437437
- name: Run live web smoke
438438
run: |
439439
pnpm clean:daemon
440-
pnpm test:smoke:web
440+
pnpm gate web-smoke
441441
442442
- name: Upload web smoke artifacts
443443
if: always()

‎.github/workflows/concurrency-torture-nightly.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ name: Concurrency Torture Nightly
1010
# Scheduled + manual only: the PR gate already runs a fast default sweep via the
1111
# Node integration lane; this nightly sweeps a much larger seed range to keep
1212
# mining for ordering bugs. A failure prints the seed and the exact
13-
# `TORTURE_SEED=<n> pnpm test:concurrency-torture` replay command.
13+
# `TORTURE_SEED=<n> pnpm gate concurrency-torture` replay command.
1414

1515
on:
1616
schedule:
@@ -54,7 +54,7 @@ jobs:
5454
uses: ./.github/actions/setup-node-pnpm
5555

5656
- name: Run concurrency torture sweep
57-
run: pnpm test:concurrency-torture
57+
run: pnpm gate concurrency-torture
5858

5959
- name: Upload torture lane envelope
6060
if: always()

‎.github/workflows/conformance-differential.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ jobs:
6060
derived-path: ${{ env.AGENT_DEVICE_IOS_RUNNER_DERIVED_PATH }}
6161
cache-key-prefix: ios-runner-prebuilt
6262
cache-key-suffix: -ios-${{ env.IOS_RUNTIME_VERSION }}
63-
build-command: sh ./scripts/build-xcuitest-apple.sh
63+
build-command: pnpm gate swift-runner-ios
6464
xcuitest-platform: ios
6565
xcuitest-destination: generic/platform=iOS Simulator
6666

@@ -126,7 +126,7 @@ jobs:
126126
# burning its budget) — outcome parity alone cannot see that.
127127
- name: Run differential
128128
run: |
129-
pnpm maestro:conformance:differential \
129+
pnpm gate maestro-differential \
130130
--platform ios \
131131
--out-dir "${{ github.workspace }}/.tmp/conformance-differential" \
132132
--trace-root "${{ github.workspace }}/.agent-device" \

‎.github/workflows/conformance-regenerate.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,13 +49,13 @@ jobs:
4949
# Verifies the pinned jar SHA-256s against pinned-upstream.json, runs the
5050
# harness over the corpus, and rewrites fixtures/ + corpus/manifest.json.
5151
- name: Regenerate fixtures from pinned upstream
52-
run: pnpm maestro:conformance:regenerate
52+
run: pnpm gate maestro-regenerate
5353

5454
- name: Fail if regeneration changed anything
5555
run: |
5656
if ! git diff --exit-code -- scripts/maestro-conformance/fixtures scripts/maestro-conformance/corpus/manifest.json; then
5757
echo "::error::Checked-in conformance fixtures do not match a fresh regeneration from the pinned upstream artifacts."
58-
echo "The fixtures are generated: run 'pnpm maestro:conformance:regenerate' and commit the result."
58+
echo "The fixtures are generated: run 'pnpm gate maestro-regenerate' and commit the result."
5959
exit 1
6060
fi
6161
echo "Fixtures are byte-identical to a fresh regeneration from Maestro ${{ github.sha }}."

‎.github/workflows/ios.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ jobs:
7373
derived-path: ${{ env.AGENT_DEVICE_IOS_RUNNER_DERIVED_PATH }}
7474
cache-key-prefix: ios-runner-prebuilt
7575
cache-key-suffix: -ios-${{ env.IOS_RUNTIME_VERSION }}
76-
build-command: sh ./scripts/build-xcuitest-apple.sh
76+
build-command: pnpm gate swift-runner-ios
7777
xcuitest-platform: ios
7878
xcuitest-destination: generic/platform=iOS Simulator
7979

@@ -158,7 +158,7 @@ jobs:
158158
AGENT_DEVICE_IOS_E2E_TIER: smoke
159159
AGENT_DEVICE_IOS_UDID: ${{ steps.ios-simulator.outputs.simulator-udid }}
160160
run: |
161-
pnpm build
161+
pnpm gate build
162162
node --experimental-strip-types scripts/node-test-tmpdir.ts --test test/integration/smoke-ios-simulator-coverage.test.ts test/integration/smoke-ios-simulator.test.ts
163163
164164
- name: Assert simulator automation preserved host focus

‎.github/workflows/linux.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@ jobs:
104104
- name: Run Linux replay smoke test
105105
run: |
106106
pnpm clean:daemon
107-
pnpm test:replay:linux \
107+
pnpm gate replay-linux \
108108
--retries 2 \
109109
--report-junit test/artifacts/replays-linux.junit.xml
110110

‎.github/workflows/macos.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -41,15 +41,15 @@ jobs:
4141
with:
4242
derived-path: ${{ env.AGENT_DEVICE_IOS_RUNNER_DERIVED_PATH }}
4343
cache-key-prefix: macos-runner-prebuilt
44-
build-command: pnpm build:xcuitest:macos
44+
build-command: pnpm gate swift-runner-macos
4545
xcuitest-platform: macos
4646
xcuitest-destination: platform=macOS,arch=arm64
4747

4848
- name: Build macOS helper
49-
run: pnpm build:macos-helper
49+
run: pnpm gate macos-helper
5050

5151
- name: Run macOS integration test
52-
run: pnpm test:replay:macos --retries 2 --report-junit test/artifacts/replays-macos.junit.xml
52+
run: pnpm gate replay-macos --retries 2 --report-junit test/artifacts/replays-macos.junit.xml
5353

5454
- name: Upload macOS artifacts
5555
if: always()

0 commit comments

Comments
 (0)