Skip to content

feat(code-reviewer): add boundary review for env/credential/spawn changes - #219

Merged
wpfleger96 merged 3 commits into
mainfrom
duncan/code-reviewer-boundary-patches
Aug 12, 2026
Merged

wpfleger96 merged 3 commits into
mainfrom
duncan/code-reviewer-boundary-patches

Conversation

@wpfleger96

@wpfleger96 wpfleger96 commented Aug 12, 2026 •

Copy link
Copy Markdown
Owner

Problem

The code-reviewer skill classified review effort purely by line and file count and analyzed each diff entirely within the repository. A small change at an env-var, credential, spawn/exec, or network boundary therefore got the lightest path, and the contracts of the external binaries and services that consume those values — the actual blast radius — were never examined. That is the failure mode behind the bad approval on block/buzz#3398: a 68-line diff at the agent-spawn seam passed review with four misses that repo-local, size-routed analysis could not surface.

What the patched skill does

Boundary Relevance scan (SKILL.md). A two-tier diff scan sets boundary_relevant. Tier 1 — hard runtime-boundary constructs (env::, process.env, Command::new/.spawn(, auth-header construction, credential assignment) — sets the flag directly. Tier 2 — broad lexical hits (password, endpoint, https://, bare exec) — sets it only after a one-line semantic confirmation that the value actually crosses a process/auth/network boundary at runtime, so lexical coincidence doesn't gate a verdict. Any boundary_relevant diff routes to the Phase 2B orchestrated path regardless of size.

Boundary & External Contract specialist (references/subagent-template.md). A fifth Phase 2B opus agent whose lens is explicitly outside the repo: which binaries/services consume the touched values, what their env/API contracts require (read from their docs/source), what behavior each value activates downstream, and whether paired values (key + base URL) travel together.

Linked-issue scope-drift check (SKILL.md, PR Mode). Step 1 derives the PR's owner/repo, parses closing references (bare Fixes/Closes/Resolves #N against the target repo; qualified owner/repo#N and full issue URLs keep their explicit repo), and runs gh issue view <N> --repo <owner/repo>. Implementation scope vs. issue scope is a 🟡 finding minimum, 🔴 when the drift widens a security/credential/auth surface. A failed lookup is a named blocker to the check, not silent continuation.

Executed mutation check (references/subagent-template.md, Functionality & Testing lens). For fix PRs, the Functionality specialist reverts only the production hunks in a scratch worktree (preserving added tests when production and test hunks share a file), verifies the reversal, runs the project's full suite, and expects RED. A suite that stays green with the fix removed = 🔴 "untested at its seam". The protocol has one authoritative home; the Phase 2A inline lens references it rather than duplicating it. Inability to execute is a named unmet precondition, never a silent skip.

Ordered verdict tree (SKILL.md). (1) any verified HIGH-confidence 🔴 → REQUEST_CHANGES — a verified blocker is never suppressed by the Comment cap; else (2) any applicable mandatory check not executed (including a needed live probe the reviewer can't run) → Comment naming it, with applicability stated (issue comparison only when a linked issue exists, mutation only for fix PRs); else (3) APPROVE. A single-agent 🔴 at HIGH confidence with orchestrator concurrence can request changes — the old 2-agent bar contradicted the mutually-exclusive lens design. Forward-momentum philosophy stays intact for every non-boundary change.

The skill version is bumped 1.1.1 → 1.2.0. A prompt-contract test (tests/unit/test_code_reviewer_skill_contract.py) asserts the Phase 2B Functionality briefing carries the mutation instruction, the protocol has a single authoritative home, boundary diffs route to Phase 2B, and the verdict tree never caps a blocker — the existing CI asserts nothing about boundary_relevant or mutation behavior.

How this catches the #3398 misses

#3398 miss Now caught by
Issue scoped "goose + provider openai" but PR affects every harness Repo-stable gh issue view + scope-drift check → 🔴 (widens a credential surface)
OPENAI_COMPAT_API_KEY shared across providers with base URL in a separate var the mirror drops *_API_KEY sets boundary_relevant; Boundary specialist asks whether key + base URL are carried together
External binaries (goose/codex) consume OPENAI_API_KEY under contracts invisible to full-file reading Boundary specialist reads external consumers' env contracts, not this repo
The three tests exercise only the private predicate, not the spawn wiring Phase 2B Functionality specialist's executed mutation check → 🔴 untested at seam
A 68-line boundary diff took a lighter path boundary_relevant forces Phase 2B regardless of size; a single-agent boundary 🔴 can request changes

Duncan and others added 3 commits August 12, 2026 11:10
…nges

The skill reviewed diffs entirely within the repo and routed by line/file
count, so a small change at an env-var, credential, spawn, or network
boundary got the lightest path and its cross-repo blast radius went unseen.

Add a Boundary Relevance scan that flags such diffs, forces the orchestrated
path regardless of size, and activates a Boundary & External Contract
specialist that reads external consumers' contracts. Fetch linked issues in
PR Mode to catch scope drift, add an executed mutation check that fails a fix
whose deletion leaves tests green, and repair the verdict rule so a
single-agent boundary finding can request changes and APPROVE is unavailable
until the boundary checks have run.

Co-authored-by: Will Pfleger <[email protected]>
Signed-off-by: Will Pfleger <[email protected]>
Round-1 boundary review had five gaps: the executed mutation check lived
only in the Phase 2A inline path while boundary diffs route to Phase 2B, so
the one deterministic seam check sat on a path boundary PRs never take; the
verdict bullets let the Comment cap suppress a verified blocker and demanded
issue/mutation checks that a boundary PR without a linked issue or a non-fix
PR could not satisfy; issue lookup ran bare `gh issue view` with no --repo;
later routing prose contradicted the boundary override; and the flag tripped
on broad lexical hits with no semantic gate.

Give the mutation protocol one authoritative home in the Functionality lens
(with mixed-hunk mechanics and cleanup), replace the verdict with an ordered
tree that never caps a verified blocker and states check applicability, make
issue lookup derive and pass owner/repo, rename the phase headings so the
override has no contradiction, and split the classifier into hard constructs
plus semantically-confirmed lexical hints. Add a prompt-contract test that
fails if the Phase 2B Functionality briefing loses the mutation instruction.

Co-authored-by: Will Pfleger <[email protected]>
Signed-off-by: Will Pfleger <[email protected]>
…tory

The no-URL issue-lookup path prescribed `gh pr view --json headRepository,baseRepository`, but `baseRepository` does not exist on current gh (exits 1) and `headRepository` is the fork, not the target, for cross-repo PRs — so the mandatory linked-issue scope-drift lookup could not derive its repo in PR-number mode. Add `url` to the step-1 call and parse owner/repo from the canonical PR URL, which resolves identically for URL and bare-number inputs and always names the true target.

Re-anchor the prompt-contract test on semantic invariants instead of vocabulary: assert the production-only revert direction (a swap to test-only reversal now trips a negative check) and drop the capitalization-pinned RED literal so a benign RED->fails reword no longer false-fails.

Co-authored-by: Will Pfleger <[email protected]>
Signed-off-by: Will Pfleger <[email protected]>
@wpfleger96
wpfleger96 merged commit 5608301 into main Aug 12, 2026
13 checks passed
@wpfleger96
wpfleger96 deleted the duncan/code-reviewer-boundary-patches branch August 12, 2026 16:13
wpfleger96 pushed a commit that referenced this pull request Aug 12, 2026
🤖 I have created a release *beep* *boop*
---


##
[0.76.0](v0.75.6...v0.76.0)
(2026-08-12)


### Features

* **code-reviewer:** add boundary review for env/credential/spawn
changes
([#219](#219))
([5608301](5608301))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: wpfleger-release-bot[bot] <282999489+wpfleger-release-bot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant