docs(readme): modernize README for headless agent automation and benchmark positioning - #249

Fix this code review finding (aictrl-dev/cli PR #249, README.md:102-107):

Problem: PR-authored .opencode/AGENTS.md bypasses isolation claim
Detail: The workflow's Review step comment claims "no config or plugins from the pull request's .aictrl directory" via AICTRL_DISABLE_PROJECT_CONFIG, but that flag only gates project config discovery (packages/cli/src/session/instruction.ts:27,71 — the findUp of project files). globalFiles() unconditionally pushes path.join(Instance.directory, ".opencode", "AGENTS.md") (instruction.ts:22) and is consumed unconditionally at instruction.ts:83 — and the working directory of this job is the PR's own checkout. A same-repo PR can therefore add .opencode/AGENTS.md and have it loaded as agent instructions during the review run, steering the review that is auto-posted to the PR via `gh pr comment`. This contradicts the isolation claim the README pattern sells, and this README is the product's copy-paste security template. (External skill dirs are also scanned, but the skill tool is denied by the "*": "deny" rule, so it is inert here.)
Suggested fix: Either gate the Instance.directory/.opencode/AGENTS.md entry in globalFiles() behind Flag.AICTRL_DISABLE_PROJECT_CONFIG (CLI fix), or harden the embedded workflow: add `- run: rm -rf .opencode .claude .agents .aictrl` between checkout and the Review step, and soften the README comment to match what the flag actually covers.

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.

      - run: gh pr diff "$PR" > pr.diff
        env:
          GH_TOKEN: ${{ github.token }}

      - run: npm install -g @aictrl/[email protected]

      # 2. Review with the model key only: no GitHub token, no shell, read-only tools,
      #    and no config or plugins from the pull request's .aictrl directory.
      - name: Review
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          MODEL: ${{ vars.AICTRL_MODEL }} # e.g. anthropic/claude-sonnet-4-5
          AICTRL_DISABLE_PROJECT_CONFIG: "true"

Fix this code review finding (aictrl-dev/cli PR #249, README.md:71):

Problem: Workflow never runs for reopened PRs (missing trigger)
Detail: The copy-paste PR-review workflow triggers only on [opened, synchronize, ready_for_review]; `reopened` is missing and no other type fires on reopen, so a closed-then-reopened PR with no new pushes gets no aictrl review at all — a silent coverage gap for every user who pastes this template. Repro: copy the workflow from the README; open a PR (review comment posted); close it; reopen it without pushing — no workflow run triggers and the reopened PR carries no review.
Suggested fix: Add `reopened` to the types list: `types: [opened, reopened, synchronize, ready_for_review]`.

Suggested patch:
--- a/README.md
+++ b/README.md
@@ -69,7 +69,7 @@
 
 on:
   pull_request:
-    types: [opened, synchronize, ready_for_review]
+    types: [opened, reopened, synchronize, ready_for_review]
 
 permissions:

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.

name: aictrl review

on:
  pull_request:
    types: [opened, synchronize, ready_for_review]

permissions:
  contents: read
  pull-requests: write

Fix this code review finding (aictrl-dev/cli PR #249, README.md:116-117):

Problem: jq `last` posts only final text block, can drop review
Detail: Per EVENTS.md, the CLI emits one `text` event per completed text block, and a multi-tool review run interleaves text blocks with tool calls. `[.[] | select(.type == "text") | .part.text] | last` keeps only the final block: if the model writes the review, makes one more read/grep call, then emits a short closing block ("Done — see above."), review.md contains only that fragment while `test -s` still passes, so the fragment is what gets posted to the PR — the full review is silently dropped. Repro: run the workflow on a PR where the model emits text → grep call → closing text; the posted comment is just the closing line.
Suggested fix: Collect text blocks emitted after the last tool_use event, e.g. `jq -rs '[.[] | select(.type=="text" or .type=="tool_use")] | (map(select(.type=="tool_use")) | length) as $n | .[$n:] | map(select(.type=="text") | .part.text) | join("\n\n") // empty'`, or join all text blocks and instruct the model to put the entire review in its final message.

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.

            \"Reply with the review only, in Markdown.\" > aictrl-events.jsonl\n          jq -rs '[.[] | select(.type == \"text\") | .part.text] | last // empty' aictrl-events.jsonl > review.md\n          test -s review.md\n\n      # 3. Post with the GitHub token. The model key is not in this step.\n      - run: gh pr comment \"$PR\" --body-file review.md --edit-last --create-if-none

Fix this code review finding (aictrl-dev/cli PR #249, README.md:106):

Problem: Unset AICTRL_MODEL var silently changes the model used
Detail: MODEL comes only from `${{ vars.AICTRL_MODEL }}` with no fallback. If the repository variable is not configured, the step runs `aictrl run --model ""`; in packages/cli/src/cli/cmd/run.ts the model is resolved via a falsy check (`args.model ? Provider.parseModel(...) : undefined`), so the empty string is treated as unset and the run silently uses the default agent model rather than the commented example. If that default targets a provider whose key the repo didn't configure, the run fails mid-flight with a confusing auth error (exit 1) after minutes of setup. Also, the step hardcodes ANTHROPIC_API_KEY while the variable may select another provider. Repro: add the workflow, set only ANTHROPIC_API_KEY, leave AICTRL_MODEL unset, open a PR — the Review step silently runs the default model, not claude-sonnet-4-5.
Suggested fix: Default at the expression level — `MODEL: ${{ vars.AICTRL_MODEL || 'anthropic/claude-sonnet-4-5' }}` — or fail fast in the script: `test -n "$MODEL" || { echo "Set the AICTRL_MODEL repository variable" >&2; exit 2; }`.

Suggested patch:
--- a/README.md
+++ b/README.md
@@ -104,7 +104,7 @@
       - name: Review
         env:
           ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
-          MODEL: ${{ vars.AICTRL_MODEL }} # e.g. anthropic/claude-sonnet-4-5
+          MODEL: ${{ vars.AICTRL_MODEL || 'anthropic/claude-sonnet-4-5' }}
           AICTRL_DISABLE_PROJECT_CONFIG: "true"

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.

      # 2. Review with the model key only: no GitHub token, no shell, read-only tools,\n      #    and no config or plugins from the pull request's .aictrl directory.\n      - name: Review\n        env:\n          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}\n          MODEL: ${{ vars.AICTRL_MODEL }} # e.g. anthropic/claude-sonnet-4-5\n          AICTRL_DISABLE_PROJECT_CONFIG: \"true\"\n          AICTRL_MODEL_STREAM_IDLE_TIMEOUT_MS: \"300000\"

Fix this code review finding (aictrl-dev/cli PR #249, README.md:252):

Problem: Fork attribution dropped; LICENSE still credits OpenCode
Detail: The PR deletes the "Attribution" section ("Aictrl is a fork of the OpenCode project") and the new License section says only "MIT License — see LICENSE", yet the LICENSE file at HEAD still carries "Copyright (c) 2025 opencode" alongside the Aictrl notice. After this change no user-facing doc mentions the fork (packages/cli/README.md has no attribution either), which reads as inconsistent with the LICENSE's upstream copyright and drops the fork disclosure the project previously displayed.
Suggested fix: Keep a one-line note under ## License, e.g. "MIT License — see [LICENSE](LICENSE). aictrl is a fork of the [OpenCode](https://opencode.ai) project."

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.

- **[CLI Package & Configuration Reference (`packages/cli/README.md`)](packages/cli/README.md)** — detailed `--output-schema`, `--skill`, and permission contracts\n\n## License\n\nMIT License — see [LICENSE](LICENSE).\n

Fix this code review finding (aictrl-dev/cli PR #249, README.md:99):

Problem: Quickstart pins @0.5.4; will rot without a hint
Detail: The copy-paste-ready workflow hard-pins @aictrl/[email protected] while the README's own Install section says `npm i -g @aictrl/cli` (unpinned). The pin matches today's benchmark numbers, but nothing in the example says the version is intentional, so copy-pasters stay on 0.5.4 indefinitely and silently miss fixes.
Suggested fix: Keep the pin for benchmark parity but add a comment (e.g. `# pinned to the benchmarked version; bump with releases`), or use `@^0.5` if moving tags are preferred.

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.

      # 1. Fetch the diff with the GitHub token. The model key is not in this step.\n      - run: gh pr diff \"$PR\" > pr.diff\n        env:\n          GH_TOKEN: ${{ github.token }}\n\n      - run: npm install -g @aictrl/[email protected]\n\n      # 2. Review with the model key only: no GitHub token, no shell, read-only tools,

Fix this code review finding (aictrl-dev/cli PR #249, README.md:179-182):

Problem: Event list omits 8 types EVENTS.md documents
Detail: The "covering:" bullets in the `--format json` section read as an exhaustive list, but EVENTS.md documents additional event types they omit: `session_complete` (lifecycle), `user_message`, `text`, `reasoning`, `step_start`/`step_finish`, `skill_resource_loaded`, and `error`. The omission is also self-contradictory: the README's own GitHub Actions example (line 122) filters the stream with `select(.type == "text")`, yet `text` is absent from this list.
Suggested fix: Either add the missing types (`session_complete` under Lifecycle; `user_message`, `text`, `reasoning` under a model-output bullet; `step_start`/`step_finish`; `skill_resource_loaded`; `error`), or reword "covering:" to "key events include:" so the list does not read as exhaustive.

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.

Passing `--format json` writes one JSON event per line to stdout (`schemaVersion: "1"`), covering:

- **Lifecycle:** `invocation_start`, `session_start`, `retry`, `session_error`, `invocation_error`, `invocation_complete`
- **Model & cost accounting:** `message_complete` and `subagent_message_complete` with input/output/reasoning/cache tokens, cost, and context-window utilization
- **Tools & permissions:** `tool_catalog`, `permission_granted`, `permission_rejected`, `tool_use` (with start/end timestamps), `skill_discovered`, `skill_loaded`
- **Subagents & results:** `subagent_start`, `subagent_complete`, `structured_output`, `structured_output_rejected`

Fix this code review finding (aictrl-dev/cli PR #249, README.md:126):

Problem: Injection egress via auto-posted comment undocumented
Detail: The model's input is PR-controlled (pr.diff plus files in the PR-merged checkout readable via read/glob/grep) and its verbatim text output is auto-posted as the bot's PR comment. A prompt injection planted in the diff or a reviewed file can therefore publish arbitrary markdown/links under the repo's identity (e.g. phishing or "maintainer approved" text), and `--edit-last` lets later runs rewrite it. The split-secret pattern blocks token abuse but not this output channel, and the README presents the example as safe without mentioning the residual risk.
Suggested fix: Add one sentence next to the example noting the residual prompt-injection risk (model output is PR-influenced and posted verbatim; review comments before trusting them), and/or constrain the read surface with pattern denies (e.g. deny read on `**/.env*`, `**/*.key`, `**/*.pem`).

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.

      # 3. Post with the GitHub token. The model key is not in this step.
      - run: gh pr comment "$PR" --body-file review.md --edit-last --create-if-none
        env:
          GH_TOKEN: ${{ github.token }}

      - uses: actions/upload-artifact@v4

Fix this code review finding (aictrl-dev/cli PR #249, README.md:117-123):

Problem: Flagship example skips the advertised --output-schema
Detail: The "Why a dedicated headless harness" table sells enforced JSON Schema with corrective turns and "No prose fallback" as the differentiator, but the flagship example runs free-form prose output guarded only by `test -s`. Prose is arguably the right format for a PR review, yet the example never acknowledges the tension, and using `--output-schema` here would also structurally bound what reaches the PR comment.
Suggested fix: Either use the structured path in the example (`--output-schema review.schema.json --output-result review.json`, render the comment from review.json) or add a sentence explaining when free-form prose output is appropriate vs. schema-validated results.

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.

          AICTRL_CONFIG_CONTENT: >-
            {"permission": {"*": "deny", "read": "allow", "glob": "allow", "grep": "allow"}}
        run: |
          aictrl run --format json --model "$MODEL" \
            "Review the change in pr.diff. Read files in this repository for context.
          Report bugs, security and reliability problems with file and line; skip style.
          Do not narrate your steps. Reply with the review only, in Markdown." > aictrl-events.jsonl

Fix this code review finding (aictrl-dev/cli PR #249, README.md:130-135):

Problem: Events artifact duplicates all file contents the agent read
Detail: Per EVENTS.md, each `tool_use` event carries `state.input` and `state.metadata.output`, so aictrl-events.jsonl contains the full contents of every file the reviewer read plus reasoning traces, and it is uploaded `if: always()` (including failed runs) for 7 days. Anyone with repo read access can download it — for public repos that is everyone. The contents only mirror the already-checkoutable repo, but the example should say so or redact tool outputs, since adopters may not realize the artifact duplicates their source tree.
Suggested fix: Strip tool outputs before upload (e.g. `jq 'del(.part.state.metadata.output)' aictrl-events.jsonl > upload.jsonl`) or note that the artifact mirrors repository contents and can be dropped for public repos.

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.

      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: aictrl-events
          path: aictrl-events.jsonl
          retention-days: 7

Fix this code review finding (aictrl-dev/cli PR #249, README.md:140):

Problem: "Hermes v0.22.2" vs table's "Hermes 0.22.2"
Detail: The benchmark prose introduces "`Hermes v0.22.2`" while the table header and every other CLI label use bare versions without a "v" prefix (pi 1.3.0, Goose 1.55.0, Codex 0.163.0, OpenCode 2.1.0, Hermes 0.22.2). Label drift in an otherwise strictly formatted comparison.
Suggested fix: Drop the "v": "…and `Hermes 0.22.2`) in fresh 2-CPU Docker containers…"

Implement the fix on the PR head branch and add a regression test that fails before the fix and passes after.

We benchmarked six coding-agent CLIs (`aictrl 0.5.4`, `pi 1.3.0`, `Goose 1.55.0`, `Codex 0.163.0`, `OpenCode 2.1.0`, and `Hermes v0.22.2`) in fresh 2-CPU Docker containers against a deterministic 50 ms local stub model to isolate harness overhead from remote LLM latency.
| Metric (lower is better, except coverage) | **aictrl 0.5.4** | pi 1.3.0 | Goose 1.55.0 | Codex 0.163.0 | OpenCode 2.1.0 | Hermes 0.22.2 |