Skip to content

Commit 0d8be2c

Browse files
committed
ci: verify published CLI with an owned fixture
1 parent 47dd3e0 commit 0d8be2c

5 files changed

Lines changed: 522 additions & 0 deletions

File tree

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
name: Released aictrl consumer-contract smoke
2+
3+
# Maintainer-owned branches only; never execute a pull request checkout here.
4+
on:
5+
push:
6+
branches:
7+
- main
8+
- ci/released-consumer-smoke-20261021
9+
paths:
10+
- .github/workflows/released-smoke.yml
11+
- examples/released-smoke/**
12+
workflow_dispatch:
13+
14+
permissions:
15+
contents: read
16+
17+
jobs:
18+
smoke:
19+
runs-on: ubuntu-24.04
20+
timeout-minutes: 5
21+
env:
22+
RECIPE_DIR: examples/released-smoke
23+
steps:
24+
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
25+
with:
26+
persist-credentials: false
27+
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
28+
with:
29+
node-version: 20.20.2
30+
- name: Install pinned public release without package scripts
31+
shell: bash
32+
run: |
33+
npm install --prefix "$RUNNER_TEMP/aictrl-owned-cli" \
34+
--ignore-scripts --no-audit --no-fund \
35+
36+
node -e 'const fs=require("node:fs"); const root=process.env.RUNNER_TEMP+"/aictrl-owned-cli/node_modules/@aictrl/"; for(const name of ["cli","cli-linux-x64-baseline"]){const p=JSON.parse(fs.readFileSync(root+name+"/package.json","utf8")); if(p.version!=="0.5.4") throw new Error("Unexpected package version"); console.log(p.name+"@"+p.version)}'
37+
- name: Run owned fixture against released native artifact
38+
shell: bash
39+
run: |
40+
node "$RECIPE_DIR/smoke.mjs" \
41+
--binary "$RUNNER_TEMP/aictrl-owned-cli/node_modules/@aictrl/cli-linux-x64-baseline/bin/aictrl" \
42+
--expected-sha256 f12a54259ae0d77d85fd6d0cfefe1dcc742b95a617ca71e48f4fdfb2575182ee \
43+
--evidence-dir "$RUNNER_TEMP/aictrl-owned-evidence"
44+
- name: Retain only inert fixture evidence
45+
if: always()
46+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
47+
with:
48+
name: aictrl-owned-fixture-evidence
49+
path: ${{ runner.temp }}/aictrl-owned-evidence
50+
if-no-files-found: error
51+
retention-days: 7

‎examples/released-smoke/README.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Released CLI consumer-contract smoke
2+
3+
A credential-free companion to the [real-provider PR review recipe](../../README.md#2-drop-in-github-actions-pr-review-split-secret-pattern). This runs published **0.5.4**, rather than building repository source. A scripted loopback provider requests a real file read and checks the returned bytes. It tests CLI integration, JSON events and failure handling; it does not measure model coding quality or establish safety for untrusted checkouts.
4+
5+
## Run locally
6+
7+
Use Node **20.20.2** and the published Linux x64 baseline native package. Package setup needs network access; package scripts are disabled:
8+
9+
```bash
10+
CLI_DIR=$(mktemp -d)
11+
npm install --prefix "$CLI_DIR" --ignore-scripts --no-audit --no-fund \
12+
13+
node examples/released-smoke/smoke.mjs \
14+
--binary "$CLI_DIR/node_modules/@aictrl/cli-linux-x64-baseline/bin/aictrl" \
15+
--expected-sha256 f12a54259ae0d77d85fd6d0cfefe1dcc742b95a617ca71e48f4fdfb2575182ee \
16+
--evidence-dir "$CLI_DIR/evidence"
17+
```
18+
19+
The runner verifies the native hash/version, two provider requests, an effective wire catalog containing only `read`, the completed read result, schema-version-1 NDJSON, successful terminal events and exit 0. A second invocation supplies invalid root schema `[]`: it must exit 2 with `OUTPUT_SCHEMA_CONFIG` before any additional model request. This is schema rejection, not successful structured output.
20+
21+
The CLI gets closed stdin, fresh HOME/XDG/workspace directories and an explicit environment allowlist without host credentials. Each subprocess has a 30-second deadline; the whole run has 60 seconds. Cleanup terminates POSIX process groups, escalates to SIGKILL after five seconds, and removes temporary state. A successful run prints `"status": "pass"`; a nonzero exit is failure. Evidence contains only the fixed inert fixture, public prompts and a placeholder key. Remove `CLI_DIR` after inspecting its evidence.
22+
23+
## GitHub Actions
24+
25+
The [active workflow](../../.github/workflows/released-smoke.yml) runs on changes to this directory or itself on `main` and the dedicated maintainer branch, plus manual dispatch. It has a five-minute job timeout, immutable action revisions, pinned Node/npm package versions and no user-managed secrets. `contents: read` follows the pinned checkout action's [recommended permissions](https://github.com/actions/checkout/blob/d23441a48e516b6c34aea4fa41551a30e30af803/README.md#recommended-permissions); credentials are not persisted or forwarded to the CLI. The [manual-only example](workflow.example.yml) can be copied to another trusted maintained branch with the recipe path preserved.
26+
27+
Neither workflow runs on `pull_request` or `pull_request_target`. The npm setup step uses network access; the CI job does not enforce no-egress isolation. The local transport binds only an ephemeral loopback port. Environment and tool restrictions are not an operating-system sandbox.
28+
29+
Released instruction discovery is imperfect ([#250](https://github.com/aictrl-dev/cli/issues/250)), so the runner rejects ancestor `AGENTS.md` and `.opencode/AGENTS.md` files before launching the CLI. Closed stdin avoids [#210](https://github.com/aictrl-dev/cli/issues/210). Ambient shell/config execution concerns remain [#245](https://github.com/aictrl-dev/cli/issues/245); use only this owned fixture, not an unknown PR checkout.
30+
31+
The runner passed local acceptance against the pinned released artifact. GitHub Actions execution must be checked separately; adding the workflow alone is not evidence of a successful CI run.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
CI_FIXTURE_BYTES_20261021=owned-and-inert

0 commit comments

Comments
 (0)