Skip to content

Commit 99a44f6

Browse files
authored
feat(commands): add santa-loop adversarial review command (anomalyco#1052)
* feat(commands): add santa-loop adversarial review command Adds /santa-loop, a convergence loop command built on the santa-method skill. Two independent reviewers (Claude Opus + external model) must both return NICE before code ships. Supports Codex CLI (GPT-5.4), Gemini CLI (3.1 Pro), or Claude-only fallback. Fixes are committed per round and the loop repeats until convergence or escalation. * fix: address all PR review findings for santa-loop command - Add YAML frontmatter with description (coderabbit) - Add Purpose, Usage, Output sections per CONTRIBUTING.md template (coderabbit) - Fix literal <prompt> placeholder in Gemini CLI invocation (greptile P1) - Use mktemp for unique temp file instead of fixed /tmp path (greptile P1, cubic P1) - Use --sandbox read-only instead of --full-auto to prevent repo mutation (cubic P1) - Use git push -u origin HEAD instead of bare git push (greptile P2, cubic P1) - Clarify verdict protocol: reviewers return PASS/FAIL, gate maps to NICE/NAUGHTY (greptile P2, coderabbit) - Specify parallel execution mechanism via Agent tool (coderabbit nitpick) - Add escalation format for max-iterations case (coderabbit nitpick) - Fix model IDs: gpt-5.4 for Codex, gemini-2.5-pro for Gemini
1 parent 9b611f1 commit 99a44f6

1 file changed

Lines changed: 175 additions & 0 deletions

File tree

‎commands/santa-loop.md‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
---
2+
description: Adversarial dual-review convergence loop — two independent model reviewers must both approve before code ships.
3+
---
4+
5+
# Santa Loop
6+
7+
Adversarial dual-review convergence loop using the santa-method skill. Two independent reviewers — different models, no shared context — must both return NICE before code ships.
8+
9+
## Purpose
10+
11+
Run two independent reviewers (Claude Opus + an external model) against the current task output. Both must return NICE before the code is pushed. If either returns NAUGHTY, fix all flagged issues, commit, and re-run fresh reviewers — up to 3 rounds.
12+
13+
## Usage
14+
15+
```
16+
/santa-loop [file-or-glob | description]
17+
```
18+
19+
## Workflow
20+
21+
### Step 1: Identify What to Review
22+
23+
Determine the scope from `$ARGUMENTS` or fall back to uncommitted changes:
24+
25+
```bash
26+
git diff --name-only HEAD
27+
```
28+
29+
Read all changed files to build the full review context. If `$ARGUMENTS` specifies a path, file, or description, use that as the scope instead.
30+
31+
### Step 2: Build the Rubric
32+
33+
Construct a rubric appropriate to the file types under review. Every criterion must have an objective PASS/FAIL condition. Include at minimum:
34+
35+
| Criterion | Pass Condition |
36+
|-----------|---------------|
37+
| Correctness | Logic is sound, no bugs, handles edge cases |
38+
| Security | No secrets, injection, XSS, or OWASP Top 10 issues |
39+
| Error handling | Errors handled explicitly, no silent swallowing |
40+
| Completeness | All requirements addressed, no missing cases |
41+
| Internal consistency | No contradictions between files or sections |
42+
| No regressions | Changes don't break existing behavior |
43+
44+
Add domain-specific criteria based on file types (e.g., type safety for TS, memory safety for Rust, migration safety for SQL).
45+
46+
### Step 3: Dual Independent Review
47+
48+
Launch two reviewers **in parallel** using the Agent tool (both in a single message for concurrent execution). Both must complete before proceeding to the verdict gate.
49+
50+
Each reviewer evaluates every rubric criterion as PASS or FAIL, then returns structured JSON:
51+
52+
```json
53+
{
54+
"verdict": "PASS" | "FAIL",
55+
"checks": [
56+
{"criterion": "...", "result": "PASS|FAIL", "detail": "..."}
57+
],
58+
"critical_issues": ["..."],
59+
"suggestions": ["..."]
60+
}
61+
```
62+
63+
The verdict gate (Step 4) maps these to NICE/NAUGHTY: both PASS → NICE, either FAIL → NAUGHTY.
64+
65+
#### Reviewer A: Claude Agent (always runs)
66+
67+
Launch an Agent (subagent_type: `code-reviewer`, model: `opus`) with the full rubric + all files under review. The prompt must include:
68+
- The complete rubric
69+
- All file contents under review
70+
- "You are an independent quality reviewer. You have NOT seen any other review. Your job is to find problems, not to approve."
71+
- Return the structured JSON verdict above
72+
73+
#### Reviewer B: External Model (Claude fallback only if no external CLI installed)
74+
75+
First, detect which CLIs are available:
76+
```bash
77+
command -v codex >/dev/null 2>&1 && echo "codex" || true
78+
command -v gemini >/dev/null 2>&1 && echo "gemini" || true
79+
```
80+
81+
Build the reviewer prompt (identical rubric + instructions as Reviewer A) and write it to a unique temp file:
82+
```bash
83+
PROMPT_FILE=$(mktemp /tmp/santa-reviewer-b-XXXXXX.txt)
84+
cat > "$PROMPT_FILE" << 'EOF'
85+
... full rubric + file contents + reviewer instructions ...
86+
EOF
87+
```
88+
89+
Use the first available CLI:
90+
91+
**Codex CLI** (if installed)
92+
```bash
93+
codex exec --sandbox read-only -m gpt-5.4 -C "$(pwd)" - < "$PROMPT_FILE"
94+
rm -f "$PROMPT_FILE"
95+
```
96+
97+
**Gemini CLI** (if installed and codex is not)
98+
```bash
99+
gemini -p "$(cat "$PROMPT_FILE")" -m gemini-2.5-pro
100+
rm -f "$PROMPT_FILE"
101+
```
102+
103+
**Claude Agent fallback** (only if neither `codex` nor `gemini` is installed)
104+
Launch a second Claude Agent (subagent_type: `code-reviewer`, model: `opus`). Log a warning that both reviewers share the same model family — true model diversity was not achieved but context isolation is still enforced.
105+
106+
In all cases, the reviewer must return the same structured JSON verdict as Reviewer A.
107+
108+
### Step 4: Verdict Gate
109+
110+
- **Both PASS** → **NICE** — proceed to Step 6 (push)
111+
- **Either FAIL** → **NAUGHTY** — merge all critical issues from both reviewers, deduplicate, proceed to Step 5
112+
113+
### Step 5: Fix Cycle (NAUGHTY path)
114+
115+
1. Display all critical issues from both reviewers
116+
2. Fix every flagged issue — change only what was flagged, no drive-by refactors
117+
3. Commit all fixes in a single commit:
118+
```
119+
fix: address santa-loop review findings (round N)
120+
```
121+
4. Re-run Step 3 with **fresh reviewers** (no memory of previous rounds)
122+
5. Repeat until both return PASS
123+
124+
**Maximum 3 iterations.** If still NAUGHTY after 3 rounds, stop and present remaining issues:
125+
126+
```
127+
SANTA LOOP ESCALATION (exceeded 3 iterations)
128+
129+
Remaining issues after 3 rounds:
130+
- [list all unresolved critical issues from both reviewers]
131+
132+
Manual review required before proceeding.
133+
```
134+
135+
Do NOT push.
136+
137+
### Step 6: Push (NICE path)
138+
139+
When both reviewers return PASS:
140+
141+
```bash
142+
git push -u origin HEAD
143+
```
144+
145+
### Step 7: Final Report
146+
147+
Print the output report (see Output section below).
148+
149+
## Output
150+
151+
```
152+
SANTA VERDICT: [NICE / NAUGHTY (escalated)]
153+
154+
Reviewer A (Claude Opus): [PASS/FAIL]
155+
Reviewer B ([model used]): [PASS/FAIL]
156+
157+
Agreement:
158+
Both flagged: [issues caught by both]
159+
Reviewer A only: [issues only A caught]
160+
Reviewer B only: [issues only B caught]
161+
162+
Iterations: [N]/3
163+
Result: [PUSHED / ESCALATED TO USER]
164+
```
165+
166+
## Notes
167+
168+
- Reviewer A (Claude Opus) always runs — guarantees at least one strong reviewer regardless of tooling.
169+
- Model diversity is the goal for Reviewer B. GPT-5.4 or Gemini 2.5 Pro gives true independence — different training data, different biases, different blind spots. The Claude-only fallback still provides value via context isolation but loses model diversity.
170+
- Strongest available models are used: Opus for Reviewer A, GPT-5.4 or Gemini 2.5 Pro for Reviewer B.
171+
- External reviewers run with `--sandbox read-only` (Codex) to prevent repo mutation during review.
172+
- Fresh reviewers each round prevents anchoring bias from prior findings.
173+
- The rubric is the most important input. Tighten it if reviewers rubber-stamp or flag subjective style issues.
174+
- Commits happen on NAUGHTY rounds so fixes are preserved even if the loop is interrupted.
175+
- Push only happens after NICE — never mid-loop.

0 commit comments

Comments
 (0)