Skip to content

Commit 69031f6

Browse files
authored
fix(kiro): render one answer when prose and the completion tool share an inference (#2835)
Kiro emits answer-shaped prose and then calls the private completion tool in the SAME inference. The adapter released the prose as `phase: "commentary"` and the completion answer as `phase: "final_answer"`, and `src/bridge.ts` closes the commentary message on the phase change, so the client rendered two assistant messages with near-identical text. That is the duplicate answer users reported after #2819. A valid completion answer supersedes prose staged during the same inference, so consume that collection instead of releasing it: drop the redundant `text_delta`, keep every non-text event, and release retention either way. Applied to `deferred` in required mode and `fallbackEvents` in text_fallback, which is already gated on a valid completion answer. The outer drain in `parseKiroAttempt` is deliberately untouched. An independent audit caught that it is also the leftover flush for early terminal returns, so teaching it to discard text would hide the only commentary a failed turn ever produces. Splicing at the inner site leaves that drain empty on the completion path and unchanged on every failure path. Two existing expectations pinned the duplicate as intended behaviour and now state the new contract. A Responses-protocol assertion covers what adapter-event coverage cannot: the split happens in the bridge, so the proof has to be taken at the wire. Both were driven red against the unpatched adapter — two assistant messages before, one after.
1 parent 1a18ace commit 69031f6

5 files changed

Lines changed: 247 additions & 4 deletions

File tree

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# wp2 audit round 3 — the outer drain must not learn about completions
2+
3+
Reviewer: independent explorer lane, read-only, HEAD `a43e4cda`.
4+
Verdict: **FAIL** on the plan as written in `020_wp2_duplicate_answer.md`.
5+
This document records the finding and the corrected design.
6+
7+
## What the reviewer accepted
8+
9+
- Consuming the retained collection on a valid completion IS sufficient to remove
10+
the user-visible duplicate. Required-mode text is held only in `deferred`
11+
(`src/adapters/kiro.ts:1174-1179`, `:1207`), fallback text only in
12+
`fallbackEvents` (`:1202-1205`), and a valid completion never live-flushes that
13+
run because a completion alongside a real tool call is already a protocol error
14+
(`:1260-1261`). With the commentary `text_delta` gone, `src/bridge.ts` never
15+
splits the message.
16+
- There is **no third emitter**. `:1523` builds a new event from
17+
`completionAnswer`; it does not read `deferred`. The early-loop yields at
18+
`:1398` and `:1418` only flush `deferred` when a real tool starts (`:1175`).
19+
- Dropping `text_delta` while keeping non-text retained events loses nothing
20+
load-bearing on this path. Tool events never sit in the collection — they splice
21+
live and forbid a valid completion.
22+
- `releaseEvent` is idempotent via its `eventBytes` map guard (`:808-810`), so a
23+
double release cannot double-credit the budget.
24+
25+
## The blocker
26+
27+
`020`'s option 1 says to consume the collection at BOTH readers — the inner flush
28+
and the outer drain in `parseKiroAttempt` (`:996-1001`). The reviewer showed the
29+
second half is wrong:
30+
31+
> `996-1001` is also the leftover flush for early `terminal` returns that never
32+
> hit `1468` (`1405-1413`). Dropping `text_delta` there without a completion flag
33+
> hides the only commentary.
34+
35+
That outer drain is the release path for stream, protocol, and provider failures —
36+
the row `020`'s own table requires to stay intact. A turn that fails after emitting
37+
progress prose would lose that prose entirely, which is a worse defect than the
38+
duplicate: the user would see an error with no indication of what the model had
39+
been doing.
40+
41+
## Corrected design
42+
43+
The inner site is the only consumer, and it leaves the collection **empty**:
44+
45+
1. `src/adapters/kiro.ts:1468`, `mode === "required"`: when
46+
`completionAnswer !== undefined`, splice the collection and consume it — drop
47+
each `text_delta` after releasing its retention, yield every non-text event and
48+
release it. When there is no completion answer, flush exactly as today.
49+
2. `:1474`, `mode === "text_fallback"`: this branch is **already** gated on
50+
`completionAnswer !== undefined`, so the same consume applies there and nowhere
51+
else in that mode.
52+
3. `:996-1001`, the outer drain: **unchanged**. Because step 1 splices, there is
53+
nothing left for it to emit on the completion path, and it keeps its full
54+
flush behaviour for every early-terminal path.
55+
56+
The correction is that suppression is expressed by emptying the collection at the
57+
one site that knows a completion arrived — not by teaching a second,
58+
failure-serving reader to discard text.
59+
60+
Untouched, per `020`'s release table: the `sawRealTool` flush (`:1483`), the
61+
plain-text promotion (`:1490-1498`), and the empty/reasoning-only fallback
62+
(`:1505`).
63+
64+
## Budget note
65+
66+
The reviewer's condition for a leak is "splice, skip `releaseEvent`, and skip
67+
`releaseAll`". The consume path releases every event it drops, and
68+
`releaseRetained`/`releaseAll` still runs for the `trackReplacement` remainder
69+
(`:801-803`), which is not tracked in `eventBytes`. A test asserts the budget
70+
returns to baseline.
71+
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# wp1 — live measurement: what is stale, what is a real defect
2+
3+
Measured 2026-08-29 by direct probe of three hosts plus the current tree.
4+
This phase changed no production code.
5+
6+
## Host attribution
7+
8+
| host | opencodex | process | age at measurement | verdict |
9+
|------|-----------|---------|--------------------|---------|
10+
| jun's mac (local) | 2.35.0, run from the checkout via `bun src/cli/index.ts start --port 10100` | PID 62773 | started 2026-08-28 22:12:11 | **STALE by 3 commits** |
11+
| `suji` (sujis-MacBook-Pro, 100.65.106.2) | 2.24.2, installed binary `~/.local/bin/ocx` | PID 98048 | uptime 941472s ≈ **10.9 days** | **GROSSLY STALE** — predates every Kiro fix in this unit |
12+
| `macmini-cf` (juniui-Macmini) | checkout `~/opencodex` at `d7a82a8fc` (dev, includes all of #2819) | none | `ocx` not installed, no proxy running | current source, not serving |
13+
14+
Commands: `ps -o lstart=,etime= -p 62773`, `curl -s localhost:10100/healthz` on each
15+
host, `git log --oneline -3` in `~/opencodex` on macmini-cf.
16+
17+
## Finding 1 — part of the non-termination report is a stale process
18+
19+
The local proxy the user was routed through started at **22:12:11**. Three commits
20+
of #2819 landed *after* that:
21+
22+
| commit | time | content |
23+
|--------|------|---------|
24+
| `b0740840d` | 22:14 | mark the physical attempt as locally answered too |
25+
| `d9d26552f` | 22:15 | state the code-mode echo rule before the first call |
26+
| `68eaf45d8` | 22:30 | remember a delivered final answer instead of trusting the client to echo phase |
27+
28+
`68eaf45d8` is the one that matters: it stops the terminal boundary from depending
29+
on the client echoing `phase`. A proxy started before it cannot have the completed
30+
form of the wp1 terminal-boundary fix. So the user's "still doesn't finish" report
31+
is measured against a binary that never contained the finished fix.
32+
33+
`suji` is worse and is worth stating plainly: at 2.24.2 with 10.9 days of uptime it
34+
predates the entire unit, so any Kiro turn routed there reproduces every symptom
35+
regardless of what `dev` contains.
36+
37+
**This does not close the report.** It explains part of it. Finding 2 is a real
38+
defect in current source.
39+
40+
## Finding 2 — the duplicate answer is live in current `dev`
41+
42+
Live probes against the running proxy (`/v1/responses`, streaming,
43+
`kiro/claude-opus-5`):
44+
45+
- plain question, no tools -> 1 visible assistant message, `phase: "final_answer"`, `end_turn: true`
46+
- question with a tool available, answered directly -> 1 visible message
47+
- tool-result round trip -> normal `function_call`
48+
49+
Those turns are clean, which is consistent with finding 1's fix having landed in
50+
source. But the duplicate needs a specific shape: **one inference that emits
51+
ordinary prose AND then calls the private completion tool**. The model does not
52+
take that shape on every turn, so a live probe is not a reliable trigger.
53+
54+
It does not need to be. The shape is pinned deterministically by the suite in the
55+
current tree — `tests/kiro-stream.test.ts` asserts the duplicate as expected
56+
behaviour in three places:
57+
58+
| test | asserted events |
59+
|------|-----------------|
60+
| "tool-enabled commentary can finish only through a fragmented private completion call" (:267) | `"Checking the result."` commentary, then `"Task complete."` final_answer |
61+
| "STOP_SEQUENCE text also enters bounded completion validation" (:659) | `"Done."` commentary, then `"Done."` final_answer — **byte-identical** |
62+
| "END_TURN does not promote a private completion answer's commentary" (:746) | `"Checking the result."` commentary, then `"Task complete."` final_answer |
63+
64+
Verified green at HEAD: `bun test tests/kiro-stream.test.ts -t "STOP_SEQUENCE text
65+
also enters bounded completion validation"` -> 1 pass, 3 expect() calls.
66+
67+
The `:659` case is the user's symptom exactly: the same text delivered twice, once
68+
as commentary and once as the final answer. `src/bridge.ts` splits on the phase
69+
change, so the client renders two assistant messages.
70+
71+
Mechanism in source, unchanged from `000_research.md`:
72+
`src/adapters/kiro.ts:1468` flushes the whole `deferred` collection
73+
unconditionally when `mode === "required"`, immediately before the completion
74+
answer is emitted as `final_answer`. Nothing consumes the retained prose when a
75+
valid completion answer supersedes it.
76+
77+
## Finding 3 — the outer drain is a second, independent emitter
78+
79+
`parseKiroAttempt` drains `deferred` again at `src/adapters/kiro.ts:996-999` after
80+
the inner generator returns. Skipping only the inner flush at `:1468` therefore
81+
does not remove the duplicate — it reverses its order. Any fix must CONSUME the
82+
collection, not skip one of its two readers. This confirms the audit round-2
83+
correction in `020_wp2_duplicate_answer.md` against current source.
84+
85+
## Conclusion
86+
87+
- The non-termination report: **partly stale process** (local proxy predates
88+
`68eaf45d8`; `suji` predates the entire unit). Restart is the remedy, not a code
89+
change.
90+
- The duplicate answer: **a real, currently-pinned defect in `dev`**. It is wp2.
91+
92+
Both hosts need a restart onto current `dev` before any further live judgement of
93+
turn termination is meaningful.
94+

‎src/adapters/kiro.ts‎

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1062,6 +1062,27 @@ async function* parseKiroAttemptEvents(
10621062
try { yield event; } finally { retention.releaseEvent(event); }
10631063
}
10641064
};
1065+
// A valid private completion answer supersedes the progress prose staged during the SAME
1066+
// inference: Kiro emits answer-like text and then calls the completion tool, so releasing both
1067+
// makes the bridge close the commentary message and open a second one with near-identical text
1068+
// (#2819 follow-up). Consume the collection instead — drop the redundant text, keep every
1069+
// non-text event, and release retention either way.
1070+
//
1071+
// This is deliberately the ONLY suppression site. The outer drain in `parseKiroAttempt` is also
1072+
// the leftover flush for early terminal returns (stream, protocol, and provider failures), so
1073+
// teaching it to discard text would hide the only commentary a failed turn ever produced.
1074+
// Splicing here leaves that drain empty on the completion path and untouched everywhere else.
1075+
const consumeSupersededByCompletion = async function* (
1076+
events: AdapterEvent[],
1077+
): AsyncGenerator<AdapterEvent> {
1078+
for (const event of events.splice(0)) {
1079+
try {
1080+
if (event.type !== "text_delta") yield event;
1081+
} finally {
1082+
retention.releaseEvent(event);
1083+
}
1084+
}
1085+
};
10651086

10661087
const providerState = (): { kiro: { conversationId: string } } | undefined =>
10671088
returnedConversationId ? { kiro: { conversationId: returnedConversationId } } : undefined;
@@ -1466,12 +1487,15 @@ async function* parseKiroAttemptEvents(
14661487
});
14671488

14681489
if (mode === "required") {
1469-
yield* emitRetained(deferred.splice(0));
1490+
// A valid completion answer makes this inference's staged prose redundant; anything else
1491+
// still flushes exactly as before (bounded fallback, explicit stops, real tool calls).
1492+
if (completionAnswer !== undefined) yield* consumeSupersededByCompletion(deferred);
1493+
else yield* emitRetained(deferred.splice(0));
14701494
}
14711495

14721496
if (mode === "text_fallback") {
14731497
if (completionAnswer !== undefined) {
1474-
yield* emitRetained(fallbackEvents);
1498+
yield* consumeSupersededByCompletion(fallbackEvents);
14751499
yield { type: "text_delta", text: completionAnswer, phase: "final_answer" };
14761500
return {
14771501
assistantText,

‎tests/kiro-stream.test.ts‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -267,8 +267,9 @@ describe("kiro adapter — parseStream", () => {
267267
...completionFrames("Task complete."),
268268
))));
269269

270+
// The completion answer supersedes prose staged in the SAME inference. Releasing both made the
271+
// bridge split one turn into two near-identical assistant messages, which is what the user saw.
270272
expect(events.filter(event => event.type === "text_delta")).toEqual([
271-
{ type: "text_delta", text: "Checking the result.", phase: "commentary" },
272273
{ type: "text_delta", text: "Task complete.", phase: "final_answer" },
273274
]);
274275
expect(events.some(event => event.type === "tool_call_start" || event.type === "tool_call_delta")).toBe(false);
@@ -742,8 +743,9 @@ describe("kiro adapter — parseStream", () => {
742743
eventFrame({ stopReason: "END_TURN" }, "metadataEvent"),
743744
))));
744745

746+
// END_TURN still does not promote the prose to a final answer — but the prose is now consumed
747+
// rather than released, so the turn renders as one answer instead of two.
745748
expect(events.filter(event => event.type === "text_delta")).toEqual([
746-
{ type: "text_delta", text: "Checking the result.", phase: "commentary" },
747749
{ type: "text_delta", text: "Task complete.", phase: "final_answer" },
748750
]);
749751
expect(events.at(-1)).toMatchObject({ type: "done", endTurn: true });

‎tests/server-kiro-completion-e2e.test.ts‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -218,6 +218,58 @@ describe("Kiro completion through public server endpoints", () => {
218218
}
219219
});
220220

221+
222+
test("answer-like prose plus a completion answer in ONE inference renders exactly one answer", async () => {
223+
// The user-visible defect: Kiro emits answer-shaped prose and then calls the private completion
224+
// tool in the SAME inference. Releasing both made bridge.ts close the commentary message and
225+
// open a second one with near-identical text, so the client rendered the answer twice.
226+
// Adapter-event coverage cannot prove this is gone — the split happens in the bridge.
227+
const upstream = scriptedKiroUpstream([
228+
[textFrame("The workspace is ready."), ...completionFrames("The workspace is ready.")],
229+
]);
230+
saveConfig(kiroConfig(upstream.server.url.toString()));
231+
const proxy = startServer(0);
232+
try {
233+
const response = await originalFetch(new URL("/v1/responses", proxy.url), {
234+
method: "POST",
235+
headers: { "content-type": "application/json" },
236+
body: JSON.stringify({
237+
model: "kiro-test/gpt-5.6-sol",
238+
input: "Inspect the workspace",
239+
stream: true,
240+
tools: [{ type: "function", name: "bash", description: "Run a command", parameters: { type: "object" } }],
241+
}),
242+
});
243+
244+
expect(response.status).toBe(200);
245+
const wire = await response.text();
246+
const events = responseEvents(wire);
247+
248+
// One inference only: the completion answer resolves the turn, so no bounded fallback runs.
249+
expect(upstream.requests).toHaveLength(1);
250+
251+
// Exactly one visible assistant answer, and the prose is not repeated as its own message.
252+
const completed = events.filter(event => event.name === "response.completed");
253+
expect(completed).toHaveLength(1);
254+
const messages = completed[0].data.response.output.filter((item: { type: string }) => item.type === "message");
255+
expect(messages).toHaveLength(1);
256+
expect(messages[0].phase).toBe("final_answer");
257+
expect(messages[0].content.map((part: { text: string }) => part.text).join("")).toBe("The workspace is ready.");
258+
259+
// And the duplicate is gone at the delta level too, not merely coalesced into one message.
260+
const deltas = events
261+
.filter(event => event.name === "response.output_text.delta")
262+
.map(event => event.data.delta);
263+
expect(deltas.join("")).toBe("The workspace is ready.");
264+
265+
expect(events.at(-1)?.name).toBe("response.completed");
266+
expect(wire).not.toContain(KIRO_COMPLETION_TOOL_NAME);
267+
} finally {
268+
await proxy.stop(true);
269+
upstream.server.stop(true);
270+
}
271+
});
272+
221273
test("routed compaction with text.format summarizes instead of tripping the capability guard", async () => {
222274
const upstream = scriptedKiroUpstream([
223275
[textFrame("Compaction summary of the earlier turns.")],

0 commit comments

Comments
 (0)