|
| 1 | +# 002 — Audit round 1: findings and plan amendments |
| 2 | + |
| 3 | +Reviewer: independent \`explorer\` subagent on \`xai/grok-4.6\` (decorrelated from |
| 4 | +the planning model, REVIEW-DECORRELATE-01). Read-only. 2026-08-27. |
| 5 | + |
| 6 | +**VERDICT: GO-WITH-FIXES (blockers=3)** — all three Medium, all three folded. |
| 7 | +Every finding was re-verified by the main agent before acceptance; none was |
| 8 | +taken on the reviewer's word. |
| 9 | + |
| 10 | +## Blocker 1 — stale comments will misdescribe the guard (folded) |
| 11 | + |
| 12 | +\`src/server/responses/core.ts:3094-3096\`: |
| 13 | + |
| 14 | + // the raw \`text\` controls go too: Kiro's capability guard reads both and would reject |
| 15 | + // the turn outright, and the key-mode openai-responses adapter builds from _rawBody. |
| 16 | + |
| 17 | +And \`tests/server-kiro-completion-e2e.test.ts:237-238\`: |
| 18 | + |
| 19 | + // Routed compaction must strip the structured-output request; before the strip, |
| 20 | + // Kiro's capability guard rejected the whole turn as unsupported text controls. |
| 21 | + |
| 22 | +Both re-read and confirmed by the main agent. After the narrowing, "Kiro's |
| 23 | +capability guard reads both" is false: the guard reads \`_structuredOutput\` |
| 24 | +only. A comment that names a behavior the code no longer has is worse than no |
| 25 | +comment — the next reader trusts it. |
| 26 | + |
| 27 | +**Amendment.** \`010\` gains a comment-only edit to both files. The three |
| 28 | +deletions at \`core.ts:3098-3102\` **stay**: the reviewer confirmed, and the main |
| 29 | +agent verified via \`4d1e9fcb2\` (*fix(responses): strip all structured-output |
| 30 | +traces from routed compaction*), that \`_rawBody.text\` deletion remains |
| 31 | +load-bearing for the key-mode \`openai-responses\` adapter, pinned by |
| 32 | +\`tests/responses-compaction-routing.test.ts:607\`. Only the justification |
| 33 | +changes, not the behavior. |
| 34 | + |
| 35 | +This narrows the \`000\` scope boundary, which listed \`core.ts\` as OUT. The |
| 36 | +boundary was written to protect the Lab-import invariant \`AGENTS.md\` guards; |
| 37 | +a comment edit touches no import and no code path, so the invariant is intact. |
| 38 | +Recorded explicitly rather than silently widened. |
| 39 | + |
| 40 | +## Blocker 2 — the wire-absence assertion was sloppy (folded) |
| 41 | + |
| 42 | +\`010\` proposed \`expect(serialized).not.toContain("verbosity")\` — a substring |
| 43 | +scan of the whole JSON body. Two failure modes, both real: |
| 44 | + |
| 45 | +- **False fail:** any later fixture whose prose happens to contain the word. |
| 46 | +- **False confidence:** a control forwarded under a different key would pass, |
| 47 | + since only top-level \`payload.text\` was otherwise checked. |
| 48 | + |
| 49 | +The sibling test added by \`db040e70f\` already shows the right shape |
| 50 | +(\`tests/kiro-adapter.test.ts:936-941\`): assert the specific key absent at |
| 51 | +\`payload\`, \`conversationState\`, and \`userInputMessageContext\`. |
| 52 | + |
| 53 | +**Amendment.** \`010\`'s test drops the substring scan and mirrors that pattern. |
| 54 | +Better in kind, not merely stricter: it names where the control could appear |
| 55 | +instead of hoping a word does not. |
| 56 | + |
| 57 | +## Blocker 3 — the verifier claim was wrong (folded) |
| 58 | + |
| 59 | +\`000\` claimed \`tsconfig.json\` \`include\` covers \`src/\` and \`tests/\`. It does not: |
| 60 | + |
| 61 | + "include": ["src"] |
| 62 | + |
| 63 | +\`bun x tsc --showConfig\` confirms, and \`noUnusedLocals\` is unset — TypeScript's |
| 64 | +\`strict\` does not imply it. Two consequences the plan got wrong: |
| 65 | + |
| 66 | +1. \`tsc\` does **not** typecheck the new tests. Only \`bun test\` proves them. |
| 67 | +2. Leaving the dead \`raw\` local would **not** fail typecheck. It is still |
| 68 | + deleted — dead code — but not for the stated reason. |
| 69 | + |
| 70 | +This is exactly the failure PLAN-VERIFIER-REAL-01 exists to catch: the command |
| 71 | +was run and its exit code recorded, but the *reads-the-target* claim was |
| 72 | +asserted from the config's reputation rather than its contents. The rule asks |
| 73 | +for the \`include\` entry to be quoted. It was not, and the claim was false. |
| 74 | + |
| 75 | +**Amendment.** \`000\`'s verifier table is corrected below and \`010\`'s rationale |
| 76 | +for deleting \`raw\` is restated as dead-code hygiene. \`docs-site\` gets its own |
| 77 | +build check since \`010\` edits it. |
| 78 | + |
| 79 | +### Corrected verifier table |
| 80 | + |
| 81 | +| Command | Exit | Reads this unit's target? | |
| 82 | +|---|---|---| |
| 83 | +| \`bun install\` | 0 | Prerequisite; without it \`bun test\` dies with \`Cannot find module 'zod/v4'\`. | |
| 84 | +| \`bun test tests/kiro-adapter.test.ts\` | 0 (56 pass) | **Yes** — direct path argument; imports \`../src/adapters/kiro\`. | |
| 85 | +| \`bun x tsc --noEmit\` | to capture | **Partly** — \`tsconfig.json\` \`"include": ["src"]\` covers \`src/adapters/kiro.ts\` but **not** \`tests/\`. | |
| 86 | +| \`bun run test\` | to capture | **Yes** — full \`tests/\` suite; required by \`AGENTS.md\` for shared adapter surfaces. | |
| 87 | +| \`cd docs-site && bun run build\` | to capture | **Yes** — \`010\` edits \`docs-site/src/content/docs/reference/adapters.md\`. | |
| 88 | +| \`gh pr checks <n>\` | wp3 | **Yes** — CI at the pushed head SHA. | |
| 89 | + |
| 90 | +## Reviewer findings accepted as confirmation, not amendment |
| 91 | + |
| 92 | +Independently re-verified by the main agent: |
| 93 | + |
| 94 | +- The throw string appears in exactly two places — \`src/adapters/kiro.ts:326\` |
| 95 | + and \`tests/kiro-adapter.test.ts:895\`. \`010\` already updates both. No miss. |
| 96 | +- \`_structuredOutput\` has exactly one production writer |
| 97 | + (\`src/responses/parser.ts:817\`) and one clearer |
| 98 | + (\`src/server/responses/core.ts:3099\`). Confirmed by \`rg\`. |
| 99 | +- \`buildKiroPayload\` never spreads \`_rawBody\`; \`rg _rawBody src/adapters/kiro.ts\` |
| 100 | + returns only the guard local. The no-stripping-needed claim holds. |
| 101 | +- \`src/web-search/loop.ts:745\` only picks JSON vs markdown for sidecar results. |
| 102 | + Unaffected. |
| 103 | +- \`parseRequest\` is imported at \`tests/kiro-adapter.test.ts:15\`; |
| 104 | + \`kiro/claude-haiku-4.5\` is already exercised at \`:904\`. |
| 105 | +- All three shapes throw against the current guard — the activation evidence |
| 106 | + will be real. |
| 107 | +- \`dev\` is the correct base; the gui-screenshot rule does not apply. |
| 108 | + |
| 109 | +## Why the presence check survived so long |
| 110 | + |
| 111 | +The reviewer traced it, and the main agent confirmed the chain: |
| 112 | + |
| 113 | +- \`df31ca4aba\` (2026-07-22) introduced it alongside the parallel-tool refusal, |
| 114 | + with no rationale. |
| 115 | +- \`ea6ff8fe62\` added the capability test — but pinned only |
| 116 | + \`_structuredOutput: true\`, never the raw-\`text\` disjunct. The over-broad half |
| 117 | + was **never covered by a test**, which is why it survived \`db040e70f\`'s |
| 118 | + cleanup of its neighbour. |
| 119 | +- \`4d1e9fcb2\` then wrote code *around* it: routed compaction deletes |
| 120 | + \`_rawBody.text\` partly to satisfy this guard, which made the disjunct look |
| 121 | + load-bearing. |
| 122 | + |
| 123 | +An untested branch that other code defends against reads as intentional. It was |
| 124 | +not; it was a leftover from before \`parseTextFormat\` existed to tell the two |
| 125 | +concepts apart. |
| 126 | + |
| 127 | +## External contract check |
| 128 | + |
| 129 | +The reviewer verified against upstream rather than from memory: OpenAI's |
| 130 | +\`ResponseTextConfig\` carries exactly \`format\` and \`verbosity\`, with \`format\` one |
| 131 | +of \`text\` | \`json_schema\` | \`json_object\` (openai-python |
| 132 | +\`response_text_config_param.py\`, fetched 2026-08-27). So the tolerated set is |
| 133 | +closed at today's contract, and refusing structured output — rather than |
| 134 | +silently degrading to prose — is the honest disposition. |
| 135 | + |
0 commit comments