Skip to content

Commit d7a82a8

Browse files
authored
Merge pull request #2819 from lidge-jun/codex/kiro-exec-empty-output-and-turn-termination
fix(kiro): explain empty exec output and stop reopening a delivered final answer
2 parents f1d819b + 68eaf45 commit d7a82a8

21 files changed

Lines changed: 1416 additions & 47 deletions
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Kiro turn termination — residual defect research
2+
3+
Observed 2026-08-28 21:20 KST by the user in a Codex desktop session routed
4+
`kiro/claude-opus-5` through the local proxy on port 10100.
5+
6+
## Symptom
7+
8+
1. A plain question ("근데 코드 모드가 뭐임") produced the final answer TWICE in
9+
one turn, the second a near-duplicate rewrite of the first.
10+
2. The user reports the "answer finishes, then continues like a goal" loop is
11+
still present after `cf1a5720c`.
12+
13+
## Live-state evidence
14+
15+
- Listener PID 3653, started 2026-08-28 21:17:50, running the checkout at
16+
`/Users/jun/Developer/new/700_projects/opencodex/src/cli/index.ts start --port 10100`.
17+
- `cf1a5720c` committed 21:09:50 — the running process DID load that fix.
18+
Confirmed independently: a probe that produced no stdout in this session
19+
returned the new empty-exec wording added by that commit.
20+
- HEAD advanced to `60537f067` at 21:26:53 (a different session's commit), so
21+
the running proxy is stale relative to HEAD but not relative to `cf1a5720c`.
22+
23+
So the residual behaviour is a real defect, not a stale process.
24+
25+
## Mechanism 1 — duplicate final answer (rendering)
26+
27+
Kiro emits answer-like ordinary text, then calls the private completion tool in
28+
the SAME inference. The adapter releases the prose as `phase: "commentary"` and
29+
the completion `answer` as `phase: "final_answer"`. `src/bridge.ts` closes the
30+
commentary message on the phase change and opens a new assistant message, so the
31+
client renders two assistant messages whose text is nearly identical.
32+
33+
This is pinned by the existing suite, so it is verified behaviour rather than a
34+
hypothesis — `tests/kiro-stream.test.ts` asserts exactly:
35+
36+
```
37+
{ type: "text_delta", text: "Done.", phase: "commentary" },
38+
{ type: "text_delta", text: "Done.", phase: "final_answer" },
39+
```
40+
41+
## Mechanism 2 — non-terminating turn (upstream fetch)
42+
43+
When replayed history ends in a delivered final answer, `buildKiroPayload` still
44+
appends a synthetic trailing user turn carrying `KIRO_ANSWER_DELIVERED_MESSAGE`
45+
and performs a real upstream inference. Neutral wording removes the instruction
46+
to resume but does not remove the prompt: the model is asked again and answers
47+
again. `60537f067` additionally suppresses the completion contract for that
48+
shape, which narrows the loop, but there is still no terminal boundary that
49+
avoids the fetch.
50+
51+
## Hypotheses tested
52+
53+
| id | claim | verdict | evidence |
54+
|----|-------|---------|----------|
55+
| H1 | the completion TOOL CALL never sets the phase flag | refuted | the proxy consumes the completion tool; a replayed history containing it throws at `src/adapters/kiro-wire.ts:88` (reproduced directly) |
56+
| H2 | any synthetic trailing user turn re-invokes the model | confirmed | `src/adapters/kiro.ts` trailing-turn append still yields an upstream inference |
57+
| H3 | another layer replays the answer | partial | the generic Responses guard is not involved; the Kiro-owned bounded completion fallback does perform a second fetch |
58+
| H4 | commentary is the last recorded component | confirmed | ordinary text is forced to commentary; the completion answer is a separate final message, split at `src/bridge.ts:922` |
59+
60+
## Verification baseline
61+
62+
`bun test tests/kiro-adapter.test.ts tests/kiro-stream.test.ts tests/server-kiro-completion-e2e.test.ts`
63+
-> 180 pass, 0 fail at `60537f067`.
Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# wp1 — terminal boundary for a delivered final answer
2+
3+
Consumes: `000_research.md` mechanism 2.
4+
5+
## Problem
6+
7+
`buildKiroPayload` (`src/adapters/kiro.ts`) turns a trailing delivered final
8+
answer into a synthetic user turn carrying `KIRO_ANSWER_DELIVERED_MESSAGE` and
9+
then performs a real upstream inference. Neutral wording is still a prompt, so
10+
the model answers again — the closed task reads as an open goal.
11+
12+
`60537f067` suppresses the completion contract for that shape. Keep that as
13+
defence in depth; it is not the boundary.
14+
15+
## Change (revised after the A-phase audit — audit verdict was FAIL on the
16+
## original placement, see 011_audit_round1.md)
17+
18+
Short-circuit BEFORE the provider fetch, but NOT inside `buildRequest` and NOT
19+
as a bare outputless `done`. Three constraints the audit established, each
20+
verified against current source:
21+
22+
1. **An outputless `done` is retried, not accepted.** `guardEmptyCompletionEventStream`
23+
treats a `done` with no content event as an empty completion, suppresses the
24+
terminal, and re-invokes the identical turn; a second empty terminal becomes
25+
`empty_completion_retry_failed`
26+
(`src/server/responses/empty-completion-guard.ts:246-270`). So the naive
27+
terminal turns one loop into either another inference or a stated error.
28+
The local terminal must therefore bypass the empty-completion guard as well as
29+
the transport.
30+
2. **`buildRequest` cannot emit events.** The adapter contract returns an
31+
`AdapterRequest`; events only exist once a `Response` reaches `parseStream`
32+
(`src/adapters/base.ts`). The server then records and sends the attempt
33+
unconditionally. Manufacturing a fake `Response` inside `fetchResponse` is
34+
also wrong: it records a physical send and still meets the guard.
35+
3. **A phantom estimate must not be logged.** Kiro attaches an estimated input
36+
count during build and the server notes the attempt send before fetching, so
37+
short-circuiting after a build would log a request that never happened.
38+
39+
Placement: an explicit adapter-owned local-terminal decision consulted in
40+
`handleResponsesInner` AFTER adapter resolution and BEFORE the ordinary
41+
build/send path, short-circuiting to a locally constructed terminal response.
42+
Reuse `hasTrailingDeliveredFinalAnswer` as the predicate; do not introduce a
43+
second notion of "delivered". The hook must not intercept the adapter-owned
44+
bounded retry, which builds with a forced `text_fallback` mode.
45+
46+
Usage accounting for the local terminal: no build-time estimate, `sendCount`
47+
zero, response usage explicitly zero for input/output/total, and no estimated
48+
usage in the request log.
49+
50+
## Out of scope
51+
52+
- Any change to how the completion tool is parsed or consumed.
53+
- Any change to the empty-exec normalisation from `cf1a5720c` / `60537f067`.
54+
- The duplicate-rendering defect, which is wp2.
55+
56+
## Criteria
57+
58+
1. Replaying a delivered final answer issues ZERO upstream requests and yields a
59+
completed turn with `endTurn: true`.
60+
2. Criterion 1 holds with `emptyCompletionRetry` BOTH enabled and disabled, for
61+
streaming and non-streaming Responses. This is the criterion the audit added;
62+
without it the fix passes a test and still loops in the user's config.
63+
3. The short-circuited turn logs `sendCount === 0` and no estimated usage.
64+
4. A genuine later user message after a delivered final answer still performs a
65+
normal inference (control).
66+
5. An unfinished trailing assistant turn still gets the continuation prompt and
67+
still performs an inference (control, already covered — must stay green).
68+
6. The adapter-owned bounded `text_fallback` retry is NOT intercepted.
69+
7. `60537f067`'s completion-mode suppression remains asserted.
70+
71+
## Completion language
72+
73+
Closing wp1 fixes the repeated inference ONLY. The user-visible duplicate answer
74+
remains until wp2 lands, and the wp1 report must say so rather than implying the
75+
reported symptom is fully resolved.
76+
77+
## Evidence
78+
79+
`bun test tests/kiro-adapter.test.ts tests/kiro-stream.test.ts tests/server-kiro-completion-e2e.test.ts`
80+
plus new public-server coverage in `tests/server-kiro-completion-e2e.test.ts`,
81+
each new assertion driven red once by reverting the change.
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# A-phase audit round 1 — verdict FAIL
2+
3+
Reviewer: independent subagent (gpt-5.6-sol, medium effort), read-only lane.
4+
Audited: `000_research.md`, `010_wp1_terminal_boundary.md` as first written.
5+
Reviewer's own checks: 180 pass / 0 fail on the three Kiro suites,
6+
`bun x tsc --noEmit` exit 0, live proxy untouched.
7+
8+
The plan was rewritten rather than argued with. Findings and dispositions:
9+
10+
| # | Finding | Disposition |
11+
|---|---------|-------------|
12+
| 1 | An outputless `done` is consumed by the empty-completion guard, which suppresses the terminal and re-invokes the identical turn (`src/server/responses/empty-completion-guard.ts:246-270`). The "safe terminal" becomes another inference or `empty_completion_retry_failed`. | ACCEPTED. Verified independently by reading the guard. wp1 now requires bypassing the guard as well as the transport, and adds a criterion covering `emptyCompletionRetry` both ON and OFF. |
13+
| 2 | `buildRequest` cannot emit events under the adapter contract, and faking a `Response` in `fetchResponse` still records a physical send. | ACCEPTED. Placement moved to an adapter-owned local-terminal decision consulted in `handleResponsesInner` before the build/send path. |
14+
| 3 | Short-circuiting after a build logs a phantom estimated request. | ACCEPTED. Criteria now demand no build-time estimate, `sendCount === 0`, zero response usage, and no estimated usage in the request log. |
15+
| 4 | `hasTrailingDeliveredFinalAnswer` is sound, but the hook must not intercept the forced `text_fallback` build. | ACCEPTED as a criterion. The predicate was re-read directly: role-and-phase based, so a user message merely QUOTING the acknowledgement is unaffected. |
16+
| 5 | wp1/wp2 separation is legitimate, but wp1's completion language must not imply the reported symptom is fully fixed. | ACCEPTED. wp1 now carries an explicit completion-language section. |
17+
| 6 | The broad "buffer commentary" direction is wrong; required-mode commentary is ALREADY deferred, and the real defect is the unconditional flush before the validated answer. Retaining across the bounded fallback would hide progress during a long second inference. | ACCEPTED, and it improves the design: wp2 is now a change to WHEN the existing deferred run is released, not a new buffer. |
18+
| 7 | No user-visible Responses-level regression was specified; adapter-event coverage cannot prove the rendered duplicate is gone. | ACCEPTED. Both work-phases now name `tests/server-kiro-completion-e2e.test.ts`. |
19+
20+
Process note the reviewer raised: it expected a staged diff and found none, and
21+
observed HEAD had moved to `761cb4cfe`. Correct on both counts — the plan unit is
22+
untracked while in P, and two unrelated commits landed from another session
23+
during the audit. Neither invalidates the substance, and the untracked worktree
24+
changes in `src/adapters/cursor/` at dispatch time belonged to that other session
25+
and were left alone.
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# A-phase audit round 2 — verdict FAIL on wp2, wp1 cleared
2+
3+
Same reviewer as round 1 (blocker-closure reuse). Round 1's seven findings were
4+
all accepted; this round re-read the revised plan.
5+
6+
## wp1 — cleared
7+
8+
The reviewer confirms the revised placement, criteria, and completion language
9+
are sufficient: bypassing transport, request building, and the empty-completion
10+
guard, with guard-on/guard-off and streaming/non-streaming coverage, zero sends,
11+
no estimated usage, and the forced `text_fallback` exclusion.
12+
13+
It also rejected `runTurn` as the seam, with specifics worth keeping: adopting
14+
`runTurn` routes EVERY Kiro request into the custom transport branch, which waits
15+
on provider pacing and increments `sendCount` before any local decision, still
16+
meets the empty-completion guard afterwards, and would force Kiro to re-own
17+
transport, retries, failover, cancellation, and accounting that its existing
18+
`buildRequest`/`fetchResponse`/`parseStream` path already provides. The local
19+
terminal therefore belongs immediately after adapter resolution and before
20+
`buildRequest`.
21+
22+
## wp2 — two execution-path blockers, both accepted
23+
24+
1. **The outer drain.** Skipping the inner flush at `src/adapters/kiro.ts:1467`
25+
is not enough: `parseKiroAttempt` independently drains `deferred` at
26+
`src/adapters/kiro.ts:996-999` after the inner generator returns. The final
27+
answer would be emitted first and the commentary after it — the duplicate
28+
survives, reversed. Found independently while reading the same file, so this
29+
is confirmed twice. The deferred collection must be consumed, and the claim
30+
that this stays clear of retention machinery is withdrawn.
31+
2. **`text_fallback` has the same shape through a different collection.** It
32+
retains in `fallbackEvents`, and `src/adapters/kiro.ts:1470-1477` emits all of
33+
them and then the completion answer. The rule must apply independently inside
34+
each inference.
35+
36+
Criterion 3 was also overbroad — "any turn whose completion never arrives enters
37+
the fallback exactly once" is false for real tools, provider/protocol failures,
38+
and explicit stops like `MAX_TOKENS`. Narrowed to a clean required-mode
39+
inference, with added controls for failure, explicit incomplete stop,
40+
text_fallback duplication, and budget return-to-baseline.
41+
42+
The six release paths the reviewer enumerated from source are now a table in
43+
`020_wp2_duplicate_answer.md` and are treated as controls.
44+
45+
## HEAD movement
46+
47+
Verified: `60537f067..761cb4cfe` touches only
48+
`src/adapters/cursor/tool-result-normalize.ts` and
49+
`tests/cursor-exec-empty-result.test.ts`. No Kiro adapter, adapter contract,
50+
Responses core, bridge, or Kiro test file changed. The plan is unaffected.
51+
52+
## Disposition
53+
54+
wp1 proceeds to implementation. wp2's plan page is corrected here and will be
55+
re-audited as part of its own cycle rather than blocking wp1.
Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
# wp2 — one visible answer per turn
2+
3+
Consumes: `000_research.md` mechanism 1.
4+
5+
## Problem
6+
7+
Kiro emits answer-like ordinary text and then calls the private completion tool
8+
in the SAME inference. The adapter releases the prose as `phase: "commentary"`
9+
and the completion `answer` as `phase: "final_answer"`;
10+
`src/bridge.ts` closes the commentary message on the phase change and opens a
11+
new assistant message. The client renders two assistant messages whose text is
12+
nearly identical. This is what the user saw.
13+
14+
The existing suite pins this pair, so the fix necessarily REPLACES an asserted
15+
expectation rather than adding to it:
16+
17+
```
18+
tests/kiro-stream.test.ts
19+
{ type: "text_delta", text: "Done.", phase: "commentary" },
20+
{ type: "text_delta", text: "Done.", phase: "final_answer" },
21+
```
22+
23+
## Constraint that shapes the design
24+
25+
Progress prose is load-bearing UX: a long tool-using turn streams commentary so
26+
the user is not left staring at nothing. Withholding ALL commentary until the
27+
turn resolves would trade a cosmetic duplicate for a silent turn, which the
28+
repository's own comments call out (`#520` gates exist precisely to avoid
29+
re-emitting or losing flushed progress).
30+
31+
So the buffering must be NARROW: hold back only the trailing commentary run that
32+
has not yet been followed by a real tool call, and only while the completion tool
33+
is still capable of arriving. Release it unchanged the moment a real tool starts,
34+
the stream ends without a completion answer, or the bounded fallback engages.
35+
36+
## Options
37+
38+
1. **Consume the same inference's retained text on a valid completion.** (CHOSEN —
39+
narrowed in audit round 1, corrected in round 2.) No new buffer is needed:
40+
required-mode commentary is ALREADY deferred. But skipping the inner flush is
41+
NOT sufficient, and this is the correction that matters:
42+
43+
- The inner flush is at `src/adapters/kiro.ts:1467`.
44+
- `parseKiroAttempt` INDEPENDENTLY drains whatever remains in `deferred` at
45+
`src/adapters/kiro.ts:996-999`, after the inner generator returns.
46+
47+
So merely skipping the inner flush emits the final answer first and then the
48+
commentary from the outer drain — the duplicate survives, in reversed order.
49+
The deferred collection must be CONSUMED on a valid completion: discard and
50+
release the redundant `text_delta` events, preserve and release every
51+
non-text event, and leave nothing for the outer drain. That necessarily
52+
touches retention ownership, so the earlier claim that this change stays
53+
clear of the retention machinery is withdrawn.
54+
55+
`text_fallback` has the SAME shape through a DIFFERENT collection: it retains
56+
in `fallbackEvents`, and `src/adapters/kiro.ts:1470-1477` emits all of them
57+
and then the completion answer. The rule must therefore apply independently
58+
inside EACH inference:
59+
60+
- required inference + valid completion -> suppress its deferred text, keep
61+
non-text events, emit the completion answer;
62+
- text_fallback inference + valid completion -> suppress that inference's
63+
retained text, keep non-text events, emit the completion answer;
64+
- never retain one inference's progress across the next inference.
65+
2. **Retain commentary ACROSS the bounded fallback.** Rejected by the audit: the
66+
second inference can be long, so withholding first-attempt progress across it
67+
would make the turn look dead and contradicts the deliberate "first attempt
68+
already flushed" gate.
69+
3. **Suppress only on redundancy.** Emit commentary live, and skip the completion
70+
answer if it is substantially the same text. Rejected: "substantially the same"
71+
is a similarity heuristic, and a wrong guess either drops the real answer or
72+
keeps the duplicate.
73+
4. **Bridge-side coalesce.** Merge a commentary message and an immediately
74+
following final answer into one assistant message. Rejected as the primary
75+
seam: the phase distinction is deliberate protocol information, and the same
76+
split is correct when the commentary genuinely preceded tool work.
77+
78+
Because the deferral already exists, this is a change to WHEN the deferred run is
79+
released, not a new retention mechanism — which also keeps it clear of the
80+
retention/budget machinery where duplication bugs have previously lived.
81+
82+
## Criteria
83+
84+
1. A single inference emitting answer-like prose plus a completion answer yields
85+
exactly ONE visible answer to the client.
86+
2. Commentary followed by a REAL tool call is still emitted live and in order.
87+
3. A CLEAN required-mode inference — text or reasoning present, no real tool, no
88+
completion answer, no explicit non-completion stop reason — still shows its
89+
progress prose and enters the bounded fallback exactly once. (Narrowed in
90+
audit round 2: the unqualified form was false for real tools, provider and
91+
protocol failures, and explicit stops such as `MAX_TOKENS` or
92+
`CONTENT_FILTERED`.)
93+
4. A Responses-protocol-level assertion, not only adapter events: the user-visible
94+
duplicate must be proven gone through the bridge. Adapter-event coverage cannot
95+
prove this, because the split happens in the bridge on the phase change. The
96+
assertion belongs in `tests/server-kiro-completion-e2e.test.ts`: one upstream
97+
request, exactly one visible assistant answer, one terminal completion, and the
98+
near-duplicate prose absent.
99+
5. The existing `tests/kiro-stream.test.ts` expectation that asserts the
100+
commentary/final pair is UPDATED, not deleted — the replacement states the new
101+
contract for the same scenario.
102+
6. `text_fallback` ordinary text plus a valid completion also yields exactly one
103+
visible answer.
104+
7. The translator budget returns to baseline after suppressed events — suppression
105+
must release retention, not leak it.
106+
107+
## Release paths that must remain intact
108+
109+
Enumerated from source in audit round 2; each is a control the implementation may
110+
not regress:
111+
112+
| trigger | release point |
113+
|---------|---------------|
114+
| a real tool starts (`sawRealTool`) | `src/adapters/kiro.ts:1172-1177`, released with the tool event |
115+
| clean no-completion turn needing fallback | flush before `needsFallback` returns, `src/adapters/kiro.ts:1535-1542` / `1600-1603` |
116+
| stream / protocol / provider failure | outer failure drain, `src/adapters/kiro.ts:996-999` |
117+
| explicit non-completion stop | released before the incomplete/error branches, `src/adapters/kiro.ts:1545-1598` |
118+
| plain-text fallback without completion | retained text promoted to `final_answer`, `src/adapters/kiro.ts:1488-1501` |
119+
| empty / reasoning-only fallback | diagnostics released before the structured incomplete, `src/adapters/kiro.ts:1503-1517` |
120+
121+
## Out of scope
122+
123+
- The terminal-boundary fix (wp1).
124+
- Changing what `phase` means in the Responses protocol.

0 commit comments

Comments
 (0)