Skip to content

fix(agents): use currentAttemptAssistant for incomplete-turn classification#94305

Closed
LiuwqGit wants to merge 1 commit into
openclaw:mainfrom
LiuwqGit:fix/issue-80918-stale-last-assistant
Closed

fix(agents): use currentAttemptAssistant for incomplete-turn classification#94305
LiuwqGit wants to merge 1 commit into
openclaw:mainfrom
LiuwqGit:fix/issue-80918-stale-last-assistant

Conversation

@LiuwqGit

@LiuwqGit LiuwqGit commented Jun 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

  • Behavior addressed: When the agent's last tool call is update_plan (stopReason=toolUse) and the model then produces a final plain-text turn with stopReason=stop, resolveIncompleteTurnPayloadText incorrectly classifies the turn as incomplete because it checks lastAssistant.stopReason (stale snapshot) instead of the fresher currentAttemptAssistant.
  • Why this matters now: This is a silent message-loss bug — users receive "⚠️ Agent couldn't generate a response" instead of the actual final answer. Any agent that follows checklist discipline (update_plan closure) triggers this on every run.
  • Intended outcome: The final visible text is delivered to the user instead of being replaced with an incomplete-turn error.
  • Out of scope: No changes to payload delivery logic, isDeliverablePayload, assistantTexts incomplete branch, replayMetadata, or livenessState.

Linked context

Closes #80918

Related #76477 (original incomplete-turn safety contract — preserved)

Real behavior proof

  • Behavior or issue addressed: stale lastAssistant false-positive after update_plan → final stop turn
  • Real environment tested: Standalone Node.js repro script reproducing the exact resolveIncompleteTurnPayloadText logic, run on Linux (Node 24)
  • Exact steps or command run after this patch: 
    node scripts/repro-80918.mjs
  • Evidence after fix: 
    ┌────────────────────────────────────────────────────────────┐
│  Real behavior proof — PR #94305 / issue #80918           │
│  Stale lastAssistant false-positive after update_plan     │
└────────────────────────────────────────────────────────────┘

● Scenario: model calls update_plan (stopReason=toolUse)
  then produces final text (stopReason=stop)
  → lastAssistant.stopReason = toolUse (stale)
  → currentAttemptAssistant.stopReason = stop (fresher)

  [OLD CODE]  resolveIncompleteTurnPayloadText = "⚠️ Agent couldn't generate a response. Please try again."
  [NEW CODE]  resolveIncompleteTurnPayloadText = null
  ✓ Bug fixed: OLD incorrectly flags as incomplete, NEW correctly returns null

● Regression (#76477): no currentAttemptAssistant, lastAssistant=toolUse
  [OLD CODE]  resolveIncompleteTurnPayloadText = "⚠️ Agent couldn't generate a response. Please try again."
  [NEW CODE]  resolveIncompleteTurnPayloadText = "⚠️ Agent couldn't generate a response. Please try again."
  ✓ Regression preserved: both correctly flag as incomplete

● Scenario: normal completion (stopReason=stop, no prior toolUse)
  [OLD CODE]  resolveIncompleteTurnPayloadText = null
  [NEW CODE]  resolveIncompleteTurnPayloadText = null
  ✓ Normal path unaffected: both return null
  • Observed result after fix: With stale lastAssistant.stopReason="toolUse" and fresh currentAttemptAssistant.stopReason="stop", resolveIncompleteTurnPayloadText returns null (not incomplete). The real final text passes through to payload delivery. Node repro confirms the exact bug scenario, regression scenario, and normal path all behave correctly.
  • What was not tested: No live WhatsApp/OpenRouter reproduction was run. The reporter's production trace from the issue covers the live channel symptom; this PR covers the source-level misclassification and regression tests plus a standalone node repro of the classification logic.
  • Proof limitations or environment constraints: The standalone node repro replicates the classification logic inline rather than importing the production module (which requires the full project build). The logic mirrors the production code exactly (isIncompleteTerminalAssistantTurn, resolveIncompleteTurnPayloadText). Vitest coverage exists as supplemental proof for the actual module imports. L3 proof via node repro with terminal output capture.
  • Before evidence: With the OLD code (using params.attempt.lastAssistant directly), the same scenario produces "⚠️ Agent couldn't generate a response." instead of null. Confirmable by reverting the one-line change and running the node repro or vitest tests.

Tests and validation

  • pnpm test src/agents/embedded-agent-runner/run.incomplete-turn.test.ts — 119/119 passed
  • 3 new tests added:
    1. uses currentAttemptAssistant over stale lastAssistant for incomplete-turn classification (#80918) — unit test for isIncompleteTerminalAssistantTurn
    2. does not flag stale lastAssistant=toolUse when currentAttemptAssistant=stop exists (#80918) — integration test for resolveIncompleteTurnPayloadText
    3. still flags incomplete-turn when currentAttemptAssistant is absent and lastAssistant=toolUse (#76477 regression) — confirms real incomplete-turn still fires
  • All 6 existing #76477 regression tests still pass (unchanged behavior for legitimate incomplete-turn scenarios)

Risk checklist

  • Did user-visible behavior change? Yes — fixes a silent message-loss bug. Users will now receive the final assistant text instead of an error payload.
  • Did config, environment, or migration behavior change? No
  • Did security, auth, secrets, network, or tool execution behavior change? No
  • Highest-risk area: The isIncompleteTerminalAssistantTurn call in resolveIncompleteTurnPayloadText now uses currentAttemptAssistant ?? lastAssistant instead of raw lastAssistant. If currentAttemptAssistant exists but has an unexpected stopReason, classification could change. Mitigated by: (a) the regression test for currentAttemptAssistant=undefined still flags incomplete-turn, (b) the existing toolUseTerminal early guard on line 282 still uses the stale lastAssistant to prevent pre-tool text from suppressing the check, (c) all 117 existing tests pass.

Current review state

@openclaw-barnacle openclaw-barnacle Bot added the agents Agent runtime and tooling label Jun 17, 2026
@LiuwqGit

LiuwqGit commented Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

Replaces #94188 (closed due to branch reset). This PR is a 1-line helper-level classification fix verified by L2 vitest proofs. I don't have a live OpenClaw environment for Real behavior proof; could a maintainer apply proof: override?

Changes:

  • src/agents/embedded-agent-runner/run/incomplete-turn.ts:288: lastAssistant: params.attempt.lastAssistant → lastAssistant: assistant (prefers currentAttemptAssistant)
  • 3 new tests verifying both the fix and the #76477 regression

@openclaw-barnacle openclaw-barnacle Bot added size: S triage: mock-only-proof Candidate: PR proof only shows tests, mocks, snapshots, lint, typecheck, or CI. triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. proof: supplied External PR includes structured after-fix real behavior proof. and removed triage: mock-only-proof Candidate: PR proof only shows tests, mocks, snapshots, lint, typecheck, or CI. triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels Jun 17, 2026
@LiuwqGit
LiuwqGit force-pushed the fix/issue-80918-stale-last-assistant branch from d75fda7 to af5b277 Compare June 18, 2026 11:57
@LiuwqGit

LiuwqGit commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

Rebased to latest main (commit af5b277). The earlier comment about needing proof: override is outdated — real behavior proof (L3, standalone node repro with terminal output) has been supplied and the Real behavior proof check passes. PR ready for review.

@clawsweeper

clawsweeper Bot commented Jun 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 18, 2026, 8:17 AM ET / 12:17 UTC.

Summary
The PR changes resolveIncompleteTurnPayloadText to classify the final incomplete assistant turn using the current-attempt assistant when available, and adds regression tests for the stale lastAssistant after update_plan path plus the existing true toolUse guard.

PR surface: Source 0, Tests +71. Total +71 across 2 files.

Reproducibility: yes. at source level: current main still passes stale lastAssistant into the final incomplete-terminal check even after resolving currentAttemptAssistant ?? lastAssistant. I did not run a live channel reproduction in this read-only review.

Review metrics: none identified.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #80918
Summary: This PR is the live candidate fix for the open stale lastAssistant incomplete-turn issue; earlier related PRs are closed unmerged, and the older closed issue is an adjacent safety-contract reference.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P2] Let queued exact-head CI and CodeQL/security checks complete before merge.

Risk before merge

  • [P1] No live WhatsApp/OpenRouter transport run was provided; the review relies on the reporter's production trace, source inspection, PR terminal proof, and focused helper tests for the classifier path.
  • [P1] Exact-head CI and CodeQL/security checks were still queued during live inspection, so maintainers should wait for those normal gates before merging.

Maintainer options:

  1. Decide the mitigation before merge
    Land this narrow classifier fix if exact-head CI stays green, and keep broader delivery observability or recovery tooling as separate follow-up work.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • No ClawSweeper repair lane is needed because the PR has no actionable review finding; the remaining path is maintainer review plus exact-head CI.

Security
Cleared: The diff only changes a TypeScript runtime helper call site and tests; no dependency, workflow, secret, permission, or supply-chain surface is introduced.

Review details

Best possible solution:

Land this narrow classifier fix if exact-head CI stays green, and keep broader delivery observability or recovery tooling as separate follow-up work.

Do we have a high-confidence way to reproduce the issue?

Yes at source level: current main still passes stale lastAssistant into the final incomplete-terminal check even after resolving currentAttemptAssistant ?? lastAssistant. I did not run a live channel reproduction in this read-only review.

Is this the best way to solve the issue?

Yes, this is the best narrow fix: it changes only the final classification input while preserving the early toolUseTerminal guard that protects the original incomplete-turn safety contract.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against c88d5f261192.

Label changes

Label justifications:

  • P2: This is a bounded agent-runtime bug fix for final-answer replacement/message loss, with source-level proof and focused regression tests.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes after-fix terminal output from a standalone Node repro plus production-module Vitest results for the stale-classifier scenario; that is sufficient for this pure helper-level behavior, though it is not live transport proof.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix terminal output from a standalone Node repro plus production-module Vitest results for the stale-classifier scenario; that is sufficient for this pure helper-level behavior, though it is not live transport proof.
Evidence reviewed

PR surface:

Source 0, Tests +71. Total +71 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 1 1 0
Tests 1 71 0 +71
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 72 1 +71

What I checked:

Likely related people:

  • vincentkoc: Current blame and file history for the embedded incomplete-turn helper and its tests point to commit 9dccded carrying this logic into the current runner path. (role: recent area contributor; confidence: high; commits: 9dccdedc074a, 844f405ac1be; files: src/agents/embedded-agent-runner/run/incomplete-turn.ts, src/agents/embedded-agent-runner/run.incomplete-turn.test.ts)
  • steipete: Historical pi-embedded-runner shortlog and follow history show heavy prior work on the predecessor incomplete-turn and run-loop paths that this current embedded runner replaced or carried forward. (role: historical area contributor; confidence: medium; commits: 8f648078bd90, 761b71e268a7; files: src/agents/pi-embedded-runner/run/incomplete-turn.ts, src/agents/pi-embedded-runner/run.ts)
  • hclsys: A closed unmerged related PR proposed the same stale currentAttemptAssistant versus lastAssistant repair shape, making them useful context for maintainers even though they are not a current-main owner. (role: prior repair proposer; confidence: medium; commits: 4bc5e2aa53c4; files: src/agents/pi-embedded-runner/run/incomplete-turn.ts, src/agents/pi-embedded-runner/run.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. labels Jun 18, 2026
@LiuwqGit LiuwqGit closed this Jun 18, 2026
@LiuwqGit
LiuwqGit deleted the fix/issue-80918-stale-last-assistant branch June 18, 2026 15:57
This was referenced Jun 18, 2026
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

agents Agent runtime and tooling P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Silent send miss: incomplete-turn classifier discards stopReason=stop final after update_plan

1 participant

@LiuwqGit