Repository navigation
Add security-reviewer role and targeted skill preloads (WP5 PR-B) - #376
Conversation
GPT-6 cross-family reviewRead-only
Repair build: |
Replayed onto main 881164e (after #364 and #375) with a 3-way squash: main had changed adoption/bootstrap.md, which merged cleanly; manifests/evidence.json was re-registered and the matrix and grand list regenerated. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
e0bd0d7 to
5f32272
Compare
|
Trading-lane ack (lane:shared), 2026-09-27, at 5f32272. Relabelled from
Merge on green CI. 🤖 Generated with Claude Code |
|
Trading-lane ack confirmed: no impact on the trading lane. Scope: a read-only look at head
|
…asks, metrics, gate, procedure) Freezes the E2E preregistration half of full-save/PLAN.md step 9 (the baseline-receipt half already merged via #369) before any organic run: 77 frozen task definitions (16 reused Claude + 15 reused Codex information needs from #296/#343's receipts, 46 seeded/control tasks), every AA §8.3 metric/threshold/guardrail/outcome rule with its three "proposed" bounds now stated as frozen, the AA §8.1b capability gate, the AA §8.4 procedure, the AA §8.5 overturn conditions, the identity scheme, the Q1 blind-process qualification, the Workflow script that will run arms B/A/A0 (syntax-checked only, never executed), and the measurement runbook. Corrected on independent Claude review (11 findings, 95 turns) and a GPT-6 cross-family review (2 findings, 1 high) plus one combined repair round (13/13 addressed with failing-first evidence): relocated from blueprints/token-practice-e2e/ to evidence/artifacts/token-adoption-e2e-20260926/ per adoption-audit/PLAN.md line 402; moved the Workflow script out of examples/claude-native/workflows/ to avoid that directory's shared PACKET/ROUTING contract test, which this one-off frozen script correctly does not match; corrected PR-A's status (its base branch merged as #369 -- baseline only, tooling-fix scope still open); added the six AA §6 machine-readable policy fields the JSON was missing; fixed a wrong Codex arm-A lifecycle binding; blocked the one information need (an otel-tui receiver task) that no current role may legally run, pending a named permitted role; froze the coordinator's own main-dispatch task at xhigh instead of max; fixed a denylist word-boundary gap that let underscore-qualified MCP names slip past; and sealed the five normative files' SHA256 with an append-only amendment rule. Also refreshes PR #376 and #364's status to merged (both landed while this unit was in review), while recording that #364's field-preservation gap (workflow.run_id/tool_use_id still missing from collector.yaml) persists after its merge. Co-Authored-By: Claude Sonnet 5 <[email protected]>
…asks, (#381) metrics, gate, procedure) Freezes the E2E preregistration half of full-save/PLAN.md step 9 (the baseline-receipt half already merged via #369) before any organic run: 77 frozen task definitions (16 reused Claude + 15 reused Codex information needs from #296/#343's receipts, 46 seeded/control tasks), every AA §8.3 metric/threshold/guardrail/outcome rule with its three "proposed" bounds now stated as frozen, the AA §8.1b capability gate, the AA §8.4 procedure, the AA §8.5 overturn conditions, the identity scheme, the Q1 blind-process qualification, the Workflow script that will run arms B/A/A0 (syntax-checked only, never executed), and the measurement runbook. Corrected on independent Claude review (11 findings, 95 turns) and a GPT-6 cross-family review (2 findings, 1 high) plus one combined repair round (13/13 addressed with failing-first evidence): relocated from blueprints/token-practice-e2e/ to evidence/artifacts/token-adoption-e2e-20260926/ per adoption-audit/PLAN.md line 402; moved the Workflow script out of examples/claude-native/workflows/ to avoid that directory's shared PACKET/ROUTING contract test, which this one-off frozen script correctly does not match; corrected PR-A's status (its base branch merged as #369 -- baseline only, tooling-fix scope still open); added the six AA §6 machine-readable policy fields the JSON was missing; fixed a wrong Codex arm-A lifecycle binding; blocked the one information need (an otel-tui receiver task) that no current role may legally run, pending a named permitted role; froze the coordinator's own main-dispatch task at xhigh instead of max; fixed a denylist word-boundary gap that let underscore-qualified MCP names slip past; and sealed the five normative files' SHA256 with an append-only amendment rule. Also refreshes PR #376 and #364's status to merged (both landed while this unit was in review), while recording that #364's field-preservation gap (workflow.run_id/tool_use_id still missing from collector.yaml) persists after its merge. Co-authored-by: Scout <scout@local> Co-authored-by: Claude Sonnet 5 <[email protected]>
…ed count Skills-trial condition log: isolated-builder's verification-before-completion preload and the body line that names it came from #376 (623d34f, 2026-09-27T03:38:47Z per TZ=UTC git log -S on adoption/agents/claude/isolated-builder.md), a boundary inside C0, not a constant across every condition. The one recorded install time is attributed as measured-gaps.md reports it (the coordinator's local measurement), without naming a host. Token-lanes addendum: the community alternative keeps superpowers as the verifiable example and drops a survey count that is not retained in the repository. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…ed count Skills-trial condition log: isolated-builder's verification-before-completion preload and the body line that names it came from #376 (623d34f, 2026-09-27T03:38:47Z per TZ=UTC git log -S on adoption/agents/claude/isolated-builder.md), a boundary inside C0, not a constant across every condition. The one recorded install time is attributed as measured-gaps.md reports it (the coordinator's local measurement), without naming a host. Token-lanes addendum: the community alternative keeps superpowers as the verifiable example and drops a survey count that is not retained in the repository. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…ed count Skills-trial condition log: isolated-builder's verification-before-completion preload and the body line that names it came from #376 (623d34f, 2026-09-27T03:38:47Z per TZ=UTC git log -S on adoption/agents/claude/isolated-builder.md), a boundary inside C0, not a constant across every condition. The one recorded install time is attributed as measured-gaps.md reports it (the coordinator's local measurement), without naming a host. Token-lanes addendum: the community alternative keeps superpowers as the verifiable example and drops a survey count that is not retained in the repository. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…ed count Skills-trial condition log: isolated-builder's verification-before-completion preload and the body line that names it came from #376 (623d34f, 2026-09-27T03:38:47Z per TZ=UTC git log -S on adoption/agents/claude/isolated-builder.md), a boundary inside C0, not a constant across every condition. The one recorded install time is attributed as measured-gaps.md reports it (the coordinator's local measurement), without naming a host. Token-lanes addendum: the community alternative keeps superpowers as the verifiable example and drops a survey count that is not retained in the repository. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…l line (AN-15, after #447) (#456) * Token-lanes carrier: evidence line replaces the verification-skill instruction Default block line 13 now asks for evidence (the command and what it returned, or the file:line read) and keeps the search-first sentence: 4,094 -> 4,088 bytes against the unchanged 4,100-byte bound (AN-15). The builder block drops its verification sentence (2,883 -> 2,747 bytes): isolated-builder.md:7-9 preloads the skill and :12 names it. Sources: code.claude.com/docs/en/best-practices.md L52 (show evidence); platform.claude.com prompting-claude-opus-5.md L61, L81 (remove explicit verification instructions), prompting-claude-opus-5-5.md L9 (carried forward), claude-prompting-best-practices.md L780 (Opus 5 exception). Handbook mirror: carrier line, its sources paragraph naming only search-first, and the builder table row and variant. Tests: the default key phrase becomes "Show evidence before a success claim", the builder drops the skill phrase, and no block may name the withdrawn skill. SHA256SUMS refreshed for both blocks. No installer run. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Decision records: verification-line addendum and skills-trial carrier conditions Token-lanes decision, addendum 2026-09-28: why the default carrier line now asks only for evidence (Opus 5 guide L61, L81; Opus 5.5 guide L9; prompting best practices L780; Claude Code best practices L52), why the builder line is removed on its own merits (isolated-builder.md L5, L7-9, L12; sub-agents reference L600), sizes, the failing-first control, the non-Opus boundary, alternatives, #381 freeze and overturn. Skills-trial record, addendum 2026-09-28: condition log C0-C3 (#378, 2026-10-25 review rule for verification-before-completion: a with/without comparison on frozen builder tasks through an upstream harness named with its pin (tokens, elapsed time, review-found defects). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Decision records: date the builder preload boundary; drop an unretained count Skills-trial condition log: isolated-builder's verification-before-completion preload and the body line that names it came from #376 (623d34f, 2026-09-27T03:38:47Z per TZ=UTC git log -S on adoption/agents/claude/isolated-builder.md), a boundary inside C0, not a constant across every condition. The one recorded install time is attributed as measured-gaps.md reports it (the coordinator's local measurement), without naming a host. Token-lanes addendum: the community alternative keeps superpowers as the verifiable example and drops a survey count that is not retained in the repository. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Token-lanes carrier: tag the evidence sentence's upstream item; narrow the no-verification claim Repair round for the verification-line change (review findings R-F2, R-F7). - token-lanes-block.md line 13 moves "(code.claude.com best practices)" to follow "the command and what it returned", the item Claude Code best practices L52 names (https://code.claude.com/docs/en/best-practices.md, re-read 2026-09-28). The file:line alternative is local policy, not in L52. Zero-byte reorder: the block stays 4,088 bytes and line 13 295. - SHA256SUMS lists the new block hash; `sha256sum -c SHA256SUMS` passes. - Handbook: the mirrored line follows, and the sources paragraph records the file:line alternative as local policy. It now says the carrier gives no instruction to verify finished work and names no verification skill, since the codebase-memory line still checks caller lists (handbook L264). - Test comment: WITHDRAWN_PHRASES is a literal skill-name check, not a check for every verification wording. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Decision records: keep the manifest prune rule, name confounds and overturn variables Repair round for the verification-line addendum and the skills-trial carrier-condition addendum (findings V-F1, R-F1, R-F3, R-F4, R-F5, R-F6, R-F7, GAP-3, GAP-6, GAP-7). Skills-trial addendum: - The review no longer "replaces" the count-based prune rule. It adds a with/without comparison that varies the builder's preload for the skill's Claude listing; trial.prune_rule (adoption/skills/manifest.json L30) and tools/skill-usage/skill_usage.py L516-538 still apply, and Codex enablement stays under that rule. A disagreement is written into the manifest update the trial window end already requires. - Other sources and role changes: the landscape-sweep templates name both skills from 1b0e459 (templates.json common; build_args.py L59 TEMPLATE_SKILLS), for the Claude workers and the GPT-6 lane; the C0 baseline counts (child-lane baseline README L128-130) match that list without proving cause; #398 (5483412) and #402 (d022295) change the sweep workers' type and model and the builder's model inside C1. - Boundaries: the installer writes no install record (shutil.copy2 at install_claude_profile.py L113, L143 keeps the source mtime), and #381 step 2 freezes hashes, not install times. Verification-line addendum: - The builder runs model: opus, which is Opus 5.5 on this host since September 22 (grand-catalog-handbook.md L242-244), so the Opus 5 advice reaches it through the 5.5 guide's L9; before #402 it ran sonnet. - #381 records hashes, not an install time; a Supersedes note corrects the qmd addendum's L389 wording the same way. - Overturn names what each comparison varies: default-block line 13, a non-Opus child set, the builder line with the preload held on, or a prompting-guide reversal. - Line 13's quote, the local file:line policy, the re-read of every documentation quote at its cited line (2026-09-28 08:55Z), the installed SKILL.md hash check and the control's own returned lines. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Token-lanes record: the no-command-output case needs neither Bash nor ctx_execute security-reviewer has no Bash but runs commands through Context Mode's ctx_execute (Codex review), so the rationale is qualified to children with no command runner. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Evidence manifest: re-register this change's files Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Scout <scout@local> Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Summary
Finishes unit PR-B (WP5: stack agents and role table with skill preloads). The worktree
held a prior builder's 16 uncommitted files (stack-researcher/stack-verifier roles,
isolated-builder's Serena symbol-edit-tool removal, the dispatch-by-role table); this PR
commits that work as-is and has GPT-6 (Codex,
gpt-6-astra, effort max, live web search)finish the remaining goals against the plan and role table:
security-reviewerrole (Opus, max;evidence-reviewer's named readtools, no Bash/Edit/Write/WebFetch/Skill), preloading
security-best-practices.skills:preload onisolated-builder(context-mode:context-modeandverification-before-completion), with a body note thatctx_executecalls need anexplicit
cd <worktree> &&since the context-mode server binds the main checkout.menugrant and an explicit ai-memory static-client workspace/project noteon
stack-researcher.rtk proxyexactness exceptions) and one-lane-per-artifactguidance added to
stack-verifierandsource-scout, matching whatstack-researcherand
isolated-builderalready state.security-reviewerwired into every role table (the workflows README's two tables, theUltracode recipe's qualification table) and both decision docs.
test-envelope.mjs(244 assertions) andtest-contract-mutations.mjs(66 mutationfixtures, 10 new for the security role and builder preload) extended test-first against a
red baseline; one new installer test checks every shipped agent's
skills:preloadagainst the pinned skills table's
Listingcolumn (on/name-onlyonly) and the twonew agents' exact preload lists.
GPT-6 independently verified my task brief against this repository's actual files and
caught a real discrepancy: this branch's fork point (
771f25f8) predates three addendathat origin/main had already merged, including the "listing state and agent preload"
addendum with its Claude Code 2.1.283 preload probe. GPT-6 correctly declined to cite that
probe from a checkout that didn't yet contain it, and disclosed the gap in the decision
docs instead of fabricating the citation. The rebase below (which merges in those 21
commits, including that addendum) let me resolve this properly: I updated both decision
docs' text to cite the now-present probe correctly, rather than leaving the pre-rebase
disclaimers in place.
Verification
Commands run and their real results (
ci-venvis a session-scoped venv, not part of thisrepository):
git diffreview of every changed/new file (20 total), plus independentcmpon allfive adoption/examples agent-file pairs: byte-identical.
bash rec/validate.sh <ci-venv>/bin/python→FAILS=0(after registering the 20 filesand regenerating
component_matrix.py/new_host_grand_list.py, both left unchanged).<ci-venv>/bin/python -m unittest tests.test_install_claude_profile -v→ 48 tests,OK.(cd examples/claude-native/workflows && sha256sum --check --strict SHA256SUMS && node test-envelope.mjs && node test-contract-mutations.mjs)→ 244/244 and 66/66, exit 0.<ci-venv>/bin/python -m unittest discover -s tests -q, run twice (once before therebase, once after — these are not symmetric runs): the pre-rebase run (6,197 tests,
598s) had exactly one failure,
test_landscape_sweep_harness.RunnerTests.test_a_done_job_is_reused_only_for_the_same_inputs,on an unrelated environment condition (
codex_call.sh: work directory ... is inside the git repository /tmp) that has nothing to do with agent-role changes; the post-rebase runon the final, pushed tree (6,369 tests, 638s) had no failures at all:
OK (skipped=643).verdict_review_gate.py --root . --base origin/main→no verdict rows changed(checkedafter the rebase restored proper ancestry to
origin/main; an intermediate content-onlysquash produced a false "4 verdict paths changed" violation because
merge-base(HEAD, origin/main)still resolved to the old fork point until the branch tip was reset ontoorigin/maindirectly).git ls-remote origin refs/heads/claude/stack-agents-dispatch-20260926after push:2e7022e15455c69c07945c998fee8d62a05e7925, matching the pushed local HEAD.These are local integration and synthetic mutation checks (
docs/acceptance-evidence-policy.mdevidence classes), not a native agent run or provider acceptance: no new role has a
recorded native invocation yet, and every preload's first-prompt size is unmeasured. They
join the researcher/verifier rows' preregistered comparison in the decision record.
Residuals
test_landscape_sweep_harnessfailure was a pre-existing,environment-specific flake unrelated to this diff (this host's
/tmpregisters as beinginside a git repository); it did not recur on the post-rebase run, which passed clean.
stack-researcher,stack-verifiernorsecurity-reviewerhas a nativeinvocation yet, and
isolated-builder's two-skill preload has no measured first-promptsize. All four join the preregistered A/B comparison in
docs/decisions/2026-09-26-stack-agents-role-dispatch.md.context-mode:context-mode's preload eligibility rests on its installedSKILL.mdcarrying no
disable-model-invocationfield, not a repeat of the 2.1.283 probe on thatspecific plugin skill (the probe covered a pinned, table-listed skill instead).
GPT-6 cross-family review
Read-only
gpt-6-astra, max effort, live search, run against this branch's pushedworktree (head
2e7022e1). Verdict: DEFECTS FOUND, 3 findings (2 medium, 1 low), allresolved in
e0bd0d72. See the PR comment for the finding table and resolutions.SOTA sources
frontmatter fields (
tools,model,effort,skills,maxTurns,omitClaudeMd,isolation), theskills:preload mechanism and itsdisable-model-invocationexclusion, and MCP tool scoping to the parent session's connection.
docs/decisions/2026-09-25-skills-trial-and-usage.md— the 26 pinned skills'Listingcolumn (source of truth for preload eligibility) and its "listing state and agent
preload" addendum's Claude Code 2.1.283 probe.
security-best-practices(pinned
kept/on).verification-before-completion(pinnedtrial/on).src/jcodemunch_mcp/server.py—the
menu/route/ordertool instructions and schemas behind the new researcher grant.plugins/feature-dev/agents/code-explorer.mdand giancarloerra/SocratiCode@23569f0 —
precedent for a read-only reviewer defined by a tools allowlist without edit tools.
docs/token-session-handbook.md("Full reusable task prompt") — the one-lane-per-artifactguidance now stated in every touched role body.
🤖 Generated with Claude Code