|
| 1 | +--- |
| 2 | +description: Adversarial dual-review convergence loop — two independent model reviewers must both approve before code ships. |
| 3 | +--- |
| 4 | + |
| 5 | +# Santa Loop |
| 6 | + |
| 7 | +Adversarial dual-review convergence loop using the santa-method skill. Two independent reviewers — different models, no shared context — must both return NICE before code ships. |
| 8 | + |
| 9 | +## Purpose |
| 10 | + |
| 11 | +Run two independent reviewers (Claude Opus + an external model) against the current task output. Both must return NICE before the code is pushed. If either returns NAUGHTY, fix all flagged issues, commit, and re-run fresh reviewers — up to 3 rounds. |
| 12 | + |
| 13 | +## Usage |
| 14 | + |
| 15 | +``` |
| 16 | +/santa-loop [file-or-glob | description] |
| 17 | +``` |
| 18 | + |
| 19 | +## Workflow |
| 20 | + |
| 21 | +### Step 1: Identify What to Review |
| 22 | + |
| 23 | +Determine the scope from `$ARGUMENTS` or fall back to uncommitted changes: |
| 24 | + |
| 25 | +```bash |
| 26 | +git diff --name-only HEAD |
| 27 | +``` |
| 28 | + |
| 29 | +Read all changed files to build the full review context. If `$ARGUMENTS` specifies a path, file, or description, use that as the scope instead. |
| 30 | + |
| 31 | +### Step 2: Build the Rubric |
| 32 | + |
| 33 | +Construct a rubric appropriate to the file types under review. Every criterion must have an objective PASS/FAIL condition. Include at minimum: |
| 34 | + |
| 35 | +| Criterion | Pass Condition | |
| 36 | +|-----------|---------------| |
| 37 | +| Correctness | Logic is sound, no bugs, handles edge cases | |
| 38 | +| Security | No secrets, injection, XSS, or OWASP Top 10 issues | |
| 39 | +| Error handling | Errors handled explicitly, no silent swallowing | |
| 40 | +| Completeness | All requirements addressed, no missing cases | |
| 41 | +| Internal consistency | No contradictions between files or sections | |
| 42 | +| No regressions | Changes don't break existing behavior | |
| 43 | + |
| 44 | +Add domain-specific criteria based on file types (e.g., type safety for TS, memory safety for Rust, migration safety for SQL). |
| 45 | + |
| 46 | +### Step 3: Dual Independent Review |
| 47 | + |
| 48 | +Launch two reviewers **in parallel** using the Agent tool (both in a single message for concurrent execution). Both must complete before proceeding to the verdict gate. |
| 49 | + |
| 50 | +Each reviewer evaluates every rubric criterion as PASS or FAIL, then returns structured JSON: |
| 51 | + |
| 52 | +```json |
| 53 | +{ |
| 54 | + "verdict": "PASS" | "FAIL", |
| 55 | + "checks": [ |
| 56 | + {"criterion": "...", "result": "PASS|FAIL", "detail": "..."} |
| 57 | + ], |
| 58 | + "critical_issues": ["..."], |
| 59 | + "suggestions": ["..."] |
| 60 | +} |
| 61 | +``` |
| 62 | + |
| 63 | +The verdict gate (Step 4) maps these to NICE/NAUGHTY: both PASS → NICE, either FAIL → NAUGHTY. |
| 64 | + |
| 65 | +#### Reviewer A: Claude Agent (always runs) |
| 66 | + |
| 67 | +Launch an Agent (subagent_type: `code-reviewer`, model: `opus`) with the full rubric + all files under review. The prompt must include: |
| 68 | +- The complete rubric |
| 69 | +- All file contents under review |
| 70 | +- "You are an independent quality reviewer. You have NOT seen any other review. Your job is to find problems, not to approve." |
| 71 | +- Return the structured JSON verdict above |
| 72 | + |
| 73 | +#### Reviewer B: External Model (Claude fallback only if no external CLI installed) |
| 74 | + |
| 75 | +First, detect which CLIs are available: |
| 76 | +```bash |
| 77 | +command -v codex >/dev/null 2>&1 && echo "codex" || true |
| 78 | +command -v gemini >/dev/null 2>&1 && echo "gemini" || true |
| 79 | +``` |
| 80 | + |
| 81 | +Build the reviewer prompt (identical rubric + instructions as Reviewer A) and write it to a unique temp file: |
| 82 | +```bash |
| 83 | +PROMPT_FILE=$(mktemp /tmp/santa-reviewer-b-XXXXXX.txt) |
| 84 | +cat > "$PROMPT_FILE" << 'EOF' |
| 85 | +... full rubric + file contents + reviewer instructions ... |
| 86 | +EOF |
| 87 | +``` |
| 88 | + |
| 89 | +Use the first available CLI: |
| 90 | + |
| 91 | +**Codex CLI** (if installed) |
| 92 | +```bash |
| 93 | +codex exec --sandbox read-only -m gpt-5.4 -C "$(pwd)" - < "$PROMPT_FILE" |
| 94 | +rm -f "$PROMPT_FILE" |
| 95 | +``` |
| 96 | + |
| 97 | +**Gemini CLI** (if installed and codex is not) |
| 98 | +```bash |
| 99 | +gemini -p "$(cat "$PROMPT_FILE")" -m gemini-2.5-pro |
| 100 | +rm -f "$PROMPT_FILE" |
| 101 | +``` |
| 102 | + |
| 103 | +**Claude Agent fallback** (only if neither `codex` nor `gemini` is installed) |
| 104 | +Launch a second Claude Agent (subagent_type: `code-reviewer`, model: `opus`). Log a warning that both reviewers share the same model family — true model diversity was not achieved but context isolation is still enforced. |
| 105 | + |
| 106 | +In all cases, the reviewer must return the same structured JSON verdict as Reviewer A. |
| 107 | + |
| 108 | +### Step 4: Verdict Gate |
| 109 | + |
| 110 | +- **Both PASS** → **NICE** — proceed to Step 6 (push) |
| 111 | +- **Either FAIL** → **NAUGHTY** — merge all critical issues from both reviewers, deduplicate, proceed to Step 5 |
| 112 | + |
| 113 | +### Step 5: Fix Cycle (NAUGHTY path) |
| 114 | + |
| 115 | +1. Display all critical issues from both reviewers |
| 116 | +2. Fix every flagged issue — change only what was flagged, no drive-by refactors |
| 117 | +3. Commit all fixes in a single commit: |
| 118 | + ``` |
| 119 | + fix: address santa-loop review findings (round N) |
| 120 | + ``` |
| 121 | +4. Re-run Step 3 with **fresh reviewers** (no memory of previous rounds) |
| 122 | +5. Repeat until both return PASS |
| 123 | + |
| 124 | +**Maximum 3 iterations.** If still NAUGHTY after 3 rounds, stop and present remaining issues: |
| 125 | + |
| 126 | +``` |
| 127 | +SANTA LOOP ESCALATION (exceeded 3 iterations) |
| 128 | +
|
| 129 | +Remaining issues after 3 rounds: |
| 130 | +- [list all unresolved critical issues from both reviewers] |
| 131 | +
|
| 132 | +Manual review required before proceeding. |
| 133 | +``` |
| 134 | + |
| 135 | +Do NOT push. |
| 136 | + |
| 137 | +### Step 6: Push (NICE path) |
| 138 | + |
| 139 | +When both reviewers return PASS: |
| 140 | + |
| 141 | +```bash |
| 142 | +git push -u origin HEAD |
| 143 | +``` |
| 144 | + |
| 145 | +### Step 7: Final Report |
| 146 | + |
| 147 | +Print the output report (see Output section below). |
| 148 | + |
| 149 | +## Output |
| 150 | + |
| 151 | +``` |
| 152 | +SANTA VERDICT: [NICE / NAUGHTY (escalated)] |
| 153 | +
|
| 154 | +Reviewer A (Claude Opus): [PASS/FAIL] |
| 155 | +Reviewer B ([model used]): [PASS/FAIL] |
| 156 | +
|
| 157 | +Agreement: |
| 158 | + Both flagged: [issues caught by both] |
| 159 | + Reviewer A only: [issues only A caught] |
| 160 | + Reviewer B only: [issues only B caught] |
| 161 | +
|
| 162 | +Iterations: [N]/3 |
| 163 | +Result: [PUSHED / ESCALATED TO USER] |
| 164 | +``` |
| 165 | + |
| 166 | +## Notes |
| 167 | + |
| 168 | +- Reviewer A (Claude Opus) always runs — guarantees at least one strong reviewer regardless of tooling. |
| 169 | +- Model diversity is the goal for Reviewer B. GPT-5.4 or Gemini 2.5 Pro gives true independence — different training data, different biases, different blind spots. The Claude-only fallback still provides value via context isolation but loses model diversity. |
| 170 | +- Strongest available models are used: Opus for Reviewer A, GPT-5.4 or Gemini 2.5 Pro for Reviewer B. |
| 171 | +- External reviewers run with `--sandbox read-only` (Codex) to prevent repo mutation during review. |
| 172 | +- Fresh reviewers each round prevents anchoring bias from prior findings. |
| 173 | +- The rubric is the most important input. Tighten it if reviewers rubber-stamp or flag subjective style issues. |
| 174 | +- Commits happen on NAUGHTY rounds so fixes are preserved even if the loop is interrupted. |
| 175 | +- Push only happens after NICE — never mid-loop. |
0 commit comments