Skip to content

Commit 7547c89

Browse files
authored
Merge pull request #2725 from lidge-jun/codex/kiro-text-control-guard
fix(kiro): reject only genuine structured output, not every Responses text control
2 parents 70159d9 + 3727207 commit 7547c89

11 files changed

Lines changed: 1093 additions & 8 deletions

File tree

‎devlog/_fin/260827_kiro_text_control_guard/000_plan.md‎

Lines changed: 316 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
# 001 — How each adapter treats Responses \`text\` controls
2+
3+
Research only. No diffs here (LEXICO-SPLIT-01); the implementation design is
4+
\`010\`.
5+
6+
## The two things \`text\` carries
7+
8+
The Responses \`text\` object mixes two unrelated concerns, which is the root of
9+
the confusion this unit fixes:
10+
11+
- \`text.format\` — **output shape**. \`json_schema\` and \`json_object\` constrain
12+
the model to emit parseable JSON. \`type: "text"\` is the default: ordinary
13+
prose. A provider that cannot constrain output genuinely cannot honour the
14+
first two, and has nothing to honour for the third.
15+
- \`text.verbosity\` — **output length preference**. A hint. A provider that
16+
ignores it produces a slightly longer or shorter answer, and nothing breaks.
17+
18+
Conflating them means treating a length hint as an unsatisfiable contract.
19+
20+
## What the parser already decides
21+
22+
\`src/responses/parser.ts\`:
23+
24+
- \`parseTextFormat\` (\`:829-843\`) returns a value **only** for \`json_schema\` and
25+
\`json_object\`. Its own docstring: *"unknown or malformed formats are ignored,
26+
never rejected, so the native passthrough keeps forwarding whatever the caller
27+
sent via \`_rawBody\`."*
28+
- \`options.textFormat\` is set from that result (\`:801\`).
29+
- \`_structuredOutput: true\` is derived from the same result (\`:817\`).
30+
31+
So the proxy's own parser already draws the line this unit needs. The Kiro guard
32+
simply does not consult it, reaching past \`_structuredOutput\` to test
33+
\`_rawBody.text\` for mere existence.
34+
35+
## Per-adapter comparison
36+
37+
| Adapter | \`json_schema\` / \`json_object\` | \`verbosity\` | \`format:"text"\` |
38+
|---|---|---|---|
39+
| \`openai-responses\` (passthrough) | forwarded verbatim from \`_rawBody\` | **stripped** when the model advertises \`supportsVerbosity: false\` (\`:379-398\`) | forwarded |
40+
| \`openai-chat\` | re-nested as chat \`response_format\` (\`:~\`) | not applicable to the chat wire | not applicable |
41+
| \`anthropic\` | mapped to an output schema via \`normalizeAnthropicOutputSchema\` | not applicable | not applicable |
42+
| \`kiro\` (today) | **rejected** | **rejected** | **rejected** |
43+
| \`kiro\` (this unit) | rejected | tolerated, not forwarded | tolerated, not forwarded |
44+
45+
Kiro is the only adapter that turns a client hint into a 400. Every other
46+
adapter either maps the control, drops it, or ignores it.
47+
48+
## The precedent that matters most
49+
50+
\`src/adapters/openai-responses.ts:379-398\`, \`stripDisabledVerbosity\`:
51+
52+
/**
53+
* Hide a no-op Responses verbosity control from the wire as well as the catalog. This runs at
54+
* final serialization so a stale catalog or direct caller cannot bypass the capability. Other
55+
* \`text\` settings (notably structured-output \`format\`) remain untouched.
56+
*/
57+
58+
Three things worth extracting:
59+
60+
1. **The catalog is not the enforcement point.** The comment explicitly
61+
anticipates "a stale catalog or direct caller". A capability flag is a
62+
declaration; the wire needs its own handling.
63+
2. **The disposition for an unsupported control is to drop it**, not to refuse
64+
the turn.
65+
3. **\`format\` is deliberately exempted** from that dropping, because output
66+
shape is a real contract while verbosity is a preference. Exactly the
67+
distinction this unit draws inside the Kiro guard.
68+
69+
Kiro needs no equivalent stripper, for a structural reason: it does not
70+
serialize from \`_rawBody\` at all. \`buildKiroPayload\` constructs
71+
\`conversationState\` field by field from \`parsed\`. A \`text\` control the guard
72+
stops rejecting is therefore simply never read — dropped by construction. The
73+
fix is subtraction, and \`010\`'s A6 assertion pins that property.
74+
75+
## Why the catalog does not already prevent this
76+
77+
It does its job; the job is just narrower than it looks. On the operator's
78+
machine \`kiro/claude-opus-5\` carries \`support_verbosity: false\`. But:
79+
80+
- \`ensureStrictCatalogFields\` (\`src/codex/catalog/parsing.ts:408\`) defaults
81+
\`support_verbosity\` to \`true\` when a source asserts nothing, so a row without
82+
explicit provider evidence advertises the control.
83+
- Clients cache catalogs. A Codex instance holding an older \`models_cache.json\`
84+
keeps sending \`verbosity\` after the catalog says stop.
85+
- \`text.format.type: "text"\` is **not governed by any capability flag at all**.
86+
No catalog value suppresses it, because it is the default output mode. A
87+
client sending it is behaving correctly. That case alone means catalog
88+
correctness can never fully prevent this 400.
89+
90+
## The sibling defect already fixed
91+
92+
\`db040e70f\`, \`structure/04_transports-and-sidecars.md\` — *Kiro client
93+
parallel-tool hint*:
94+
95+
> That request field is permissive: it allows parallel calls but does not
96+
> require the routed transport to expose a matching flag.
97+
98+
and, from the same decision log:
99+
100+
> Rejection interprets permission as a requirement and blocks valid turns.
101+
102+
Substitute "verbosity preference" for "parallel calls" and the paragraph needs
103+
no other edit. The same function contained both defects; one was removed on
104+
2026-08-21, the other was not noticed because \`_structuredOutput\` was in the
105+
condition and looked like it was doing the discriminating.
106+
107+
## Open questions (settled)
108+
109+
- *Should Kiro map \`verbosity\` onto its emulated thinking budget?* No. Out of
110+
scope, and it would invent a behavior the upstream never promised. Tolerate
111+
and ignore.
112+
- *Should the guard reject unknown \`text\` members defensively?* No. That is the
113+
current defect restated. The parser's stance — ignore what you do not
114+
understand — is the house convention.
115+
- *Does the routed-compaction path still need to delete \`_rawBody.text\`?*
116+
(\`src/server/responses/core.ts:3099\`) Not for correctness after this change,
117+
but its comment names two consumers and the key-mode \`openai-responses\`
118+
adapter is the other. Leave it.
119+
Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
# 002 — Audit round 1: findings and plan amendments
2+
3+
Reviewer: independent \`explorer\` subagent on \`xai/grok-4.6\` (decorrelated from
4+
the planning model, REVIEW-DECORRELATE-01). Read-only. 2026-08-27.
5+
6+
**VERDICT: GO-WITH-FIXES (blockers=3)** — all three Medium, all three folded.
7+
Every finding was re-verified by the main agent before acceptance; none was
8+
taken on the reviewer's word.
9+
10+
## Blocker 1 — stale comments will misdescribe the guard (folded)
11+
12+
\`src/server/responses/core.ts:3094-3096\`:
13+
14+
// the raw \`text\` controls go too: Kiro's capability guard reads both and would reject
15+
// the turn outright, and the key-mode openai-responses adapter builds from _rawBody.
16+
17+
And \`tests/server-kiro-completion-e2e.test.ts:237-238\`:
18+
19+
// Routed compaction must strip the structured-output request; before the strip,
20+
// Kiro's capability guard rejected the whole turn as unsupported text controls.
21+
22+
Both re-read and confirmed by the main agent. After the narrowing, "Kiro's
23+
capability guard reads both" is false: the guard reads \`_structuredOutput\`
24+
only. A comment that names a behavior the code no longer has is worse than no
25+
comment — the next reader trusts it.
26+
27+
**Amendment.** \`010\` gains a comment-only edit to both files. The three
28+
deletions at \`core.ts:3098-3102\` **stay**: the reviewer confirmed, and the main
29+
agent verified via \`4d1e9fcb2\` (*fix(responses): strip all structured-output
30+
traces from routed compaction*), that \`_rawBody.text\` deletion remains
31+
load-bearing for the key-mode \`openai-responses\` adapter, pinned by
32+
\`tests/responses-compaction-routing.test.ts:607\`. Only the justification
33+
changes, not the behavior.
34+
35+
This narrows the \`000\` scope boundary, which listed \`core.ts\` as OUT. The
36+
boundary was written to protect the Lab-import invariant \`AGENTS.md\` guards;
37+
a comment edit touches no import and no code path, so the invariant is intact.
38+
Recorded explicitly rather than silently widened.
39+
40+
## Blocker 2 — the wire-absence assertion was sloppy (folded)
41+
42+
\`010\` proposed \`expect(serialized).not.toContain("verbosity")\` — a substring
43+
scan of the whole JSON body. Two failure modes, both real:
44+
45+
- **False fail:** any later fixture whose prose happens to contain the word.
46+
- **False confidence:** a control forwarded under a different key would pass,
47+
since only top-level \`payload.text\` was otherwise checked.
48+
49+
The sibling test added by \`db040e70f\` already shows the right shape
50+
(\`tests/kiro-adapter.test.ts:936-941\`): assert the specific key absent at
51+
\`payload\`, \`conversationState\`, and \`userInputMessageContext\`.
52+
53+
**Amendment.** \`010\`'s test drops the substring scan and mirrors that pattern.
54+
Better in kind, not merely stricter: it names where the control could appear
55+
instead of hoping a word does not.
56+
57+
## Blocker 3 — the verifier claim was wrong (folded)
58+
59+
\`000\` claimed \`tsconfig.json\` \`include\` covers \`src/\` and \`tests/\`. It does not:
60+
61+
"include": ["src"]
62+
63+
\`bun x tsc --showConfig\` confirms, and \`noUnusedLocals\` is unset — TypeScript's
64+
\`strict\` does not imply it. Two consequences the plan got wrong:
65+
66+
1. \`tsc\` does **not** typecheck the new tests. Only \`bun test\` proves them.
67+
2. Leaving the dead \`raw\` local would **not** fail typecheck. It is still
68+
deleted — dead code — but not for the stated reason.
69+
70+
This is exactly the failure PLAN-VERIFIER-REAL-01 exists to catch: the command
71+
was run and its exit code recorded, but the *reads-the-target* claim was
72+
asserted from the config's reputation rather than its contents. The rule asks
73+
for the \`include\` entry to be quoted. It was not, and the claim was false.
74+
75+
**Amendment.** \`000\`'s verifier table is corrected below and \`010\`'s rationale
76+
for deleting \`raw\` is restated as dead-code hygiene. \`docs-site\` gets its own
77+
build check since \`010\` edits it.
78+
79+
### Corrected verifier table
80+
81+
| Command | Exit | Reads this unit's target? |
82+
|---|---|---|
83+
| \`bun install\` | 0 | Prerequisite; without it \`bun test\` dies with \`Cannot find module 'zod/v4'\`. |
84+
| \`bun test tests/kiro-adapter.test.ts\` | 0 (56 pass) | **Yes** — direct path argument; imports \`../src/adapters/kiro\`. |
85+
| \`bun x tsc --noEmit\` | to capture | **Partly** — \`tsconfig.json\` \`"include": ["src"]\` covers \`src/adapters/kiro.ts\` but **not** \`tests/\`. |
86+
| \`bun run test\` | to capture | **Yes** — full \`tests/\` suite; required by \`AGENTS.md\` for shared adapter surfaces. |
87+
| \`cd docs-site && bun run build\` | to capture | **Yes** — \`010\` edits \`docs-site/src/content/docs/reference/adapters.md\`. |
88+
| \`gh pr checks <n>\` | wp3 | **Yes** — CI at the pushed head SHA. |
89+
90+
## Reviewer findings accepted as confirmation, not amendment
91+
92+
Independently re-verified by the main agent:
93+
94+
- The throw string appears in exactly two places — \`src/adapters/kiro.ts:326\`
95+
and \`tests/kiro-adapter.test.ts:895\`. \`010\` already updates both. No miss.
96+
- \`_structuredOutput\` has exactly one production writer
97+
(\`src/responses/parser.ts:817\`) and one clearer
98+
(\`src/server/responses/core.ts:3099\`). Confirmed by \`rg\`.
99+
- \`buildKiroPayload\` never spreads \`_rawBody\`; \`rg _rawBody src/adapters/kiro.ts\`
100+
returns only the guard local. The no-stripping-needed claim holds.
101+
- \`src/web-search/loop.ts:745\` only picks JSON vs markdown for sidecar results.
102+
Unaffected.
103+
- \`parseRequest\` is imported at \`tests/kiro-adapter.test.ts:15\`;
104+
\`kiro/claude-haiku-4.5\` is already exercised at \`:904\`.
105+
- All three shapes throw against the current guard — the activation evidence
106+
will be real.
107+
- \`dev\` is the correct base; the gui-screenshot rule does not apply.
108+
109+
## Why the presence check survived so long
110+
111+
The reviewer traced it, and the main agent confirmed the chain:
112+
113+
- \`df31ca4aba\` (2026-07-22) introduced it alongside the parallel-tool refusal,
114+
with no rationale.
115+
- \`ea6ff8fe62\` added the capability test — but pinned only
116+
\`_structuredOutput: true\`, never the raw-\`text\` disjunct. The over-broad half
117+
was **never covered by a test**, which is why it survived \`db040e70f\`'s
118+
cleanup of its neighbour.
119+
- \`4d1e9fcb2\` then wrote code *around* it: routed compaction deletes
120+
\`_rawBody.text\` partly to satisfy this guard, which made the disjunct look
121+
load-bearing.
122+
123+
An untested branch that other code defends against reads as intentional. It was
124+
not; it was a leftover from before \`parseTextFormat\` existed to tell the two
125+
concepts apart.
126+
127+
## External contract check
128+
129+
The reviewer verified against upstream rather than from memory: OpenAI's
130+
\`ResponseTextConfig\` carries exactly \`format\` and \`verbosity\`, with \`format\` one
131+
of \`text\` | \`json_schema\` | \`json_object\` (openai-python
132+
\`response_text_config_param.py\`, fetched 2026-08-27). So the tolerated set is
133+
closed at today's contract, and refusing structured output — rather than
134+
silently degrading to prose — is the honest disposition.
135+

0 commit comments

Comments
 (0)