fix #100368: mark undelivered message_tool_only finals in replay#101618
fix #100368: mark undelivered message_tool_only finals in replay#101618mushuiyu886 wants to merge 5 commits into
Conversation
|
Codex review: needs changes before merge. Reviewed July 11, 2026, 3:19 AM ET / 07:19 UTC. Summary PR surface: Source +262, Tests +338. Total +600 across 11 files. Reproducibility: yes. at source level. The queued runner can observe a successful unrelated source payload and then suppress the marker for a different undelivered final; the supplied live proof covers the ordinary no-send recovery path but not this branch. Review metrics: 1 noteworthy metric.
Stored data model Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Risk before merge
Maintainer options:
Copy recommended automerge instructionNext step before merge
Maintainer decision needed
Security Review findings
Review detailsBest possible solution: Share one normalized exact-final/current-source-route delivery classifier between direct and queued runners, retain the durable marker and replay behavior only after that invariant has focused fresh-session and rotation coverage, and have maintainers explicitly adopt the custom record as the canonical persisted representation. Do we have a high-confidence way to reproduce the issue? Yes at source level. The queued runner can observe a successful unrelated source payload and then suppress the marker for a different undelivered final; the supplied live proof covers the ordinary no-send recovery path but not this branch. Is this the best way to solve the issue? No in its current form. A durable annotation at the runner/session boundary is a plausible root-cause solution, but both runners must enforce the same exact-final delivery invariant before the representation is safe. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 34a847227473. Label changesLabel justifications:
Evidence reviewedPR surface: Source +262, Tests +338. Total +600 across 11 files. View PR surface stats
Acceptance criteria:
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
Review history (8 earlier review cycles)
|
ff72d71 to
c07cebb
Compare
c07cebb to
3acef2b
Compare
Summary
message_tool_onlyfinal entered the transcript but was not delivered to the source channel.messageIdto the repository QA channel's successful send receipt. The real runtime proof exposed that its previous nested-only receipt made a successful recovery send look incomplete.What Problem This Solves
A
message_tool_onlyassistant final can be stored in session history even though the user never received it. Without a durable delivery marker, later model turns treat that private final as user-visible conversation state, desynchronizing agent memory from what the user actually saw.The same failure also behaved differently across runners: direct runs recorded the undelivered state, while queued follow-up runs could enter recovery without doing so. This patch makes that session-state invariant runner-independent.
Root Cause
message_tool_onlystranded final: direct runs appended the undelivered marker, while queued follow-ups could enqueue recovery without recording that the final remained private.persistSessionTranscriptTurnis the canonical durable write, and the existing one-shot stranded-reply follow-up remains the canonical retry. Currentorigin/mainstill needs this patch, and the existing-PR diagnosis found no canonical replacement PR to defer to.Under
messages.visibleReplies: "message_tool", the runtime normalizes source delivery tosourceReplyDeliveryMode: "message_tool_only". An assistant final can therefore be durably present in the session transcript while remaining private because the model did not callmessage(action=send).The existing direct-run recovery path appended a hidden undelivered-final marker, but the queued follow-up recovery path did not. Both paths could enqueue the same one-shot stranded-reply recovery, yet the same user-visible failure produced different durable model context depending on which runner executed it. A restart or later model turn could consequently treat a queued private final as text the user had seen.
The source of truth is the completed run's normalized assistant final plus committed source-route delivery evidence. The marker is the durable projection of the negative delivery fact; it does not infer provider state or replace transport receipts.
The outside-Vitest proof also found an adjacent QA boundary defect.
qa-channelsuccessfully wrote the recovery message to its HTTP bus but returned only{ message: { id } }. The shared delivery classifier intentionally requires a standard committed receipt such as top-levelmessageId, so the successful send was classified as incomplete and an incorrect second failure diagnostic was emitted. The narrow repair is at the QA action boundary: retain the full message and also returnmessageId: message.id.Canonical reachability
The append is guarded by the active session id and lifecycle revision when a store path is available. Marker persistence is awaited before recovery enqueue. A persistence failure remains contained so it cannot suppress the already-established recovery path.
What changed
strandedReplyRetryruns do not append the marker again.messageIdto successfulqa-channelsend results and covered it at the channel action boundary.Real behavior proof
message_tool_onlymust append one hidden marker before recovery so later model context does not treat private text as user-visible history.messages.visibleReplies: "message_tool"→ schema/type/ingestion/normalization → runtime objectsourceReplyDeliveryMode: "message_tool_only"on the embedded/follow-up run → request/effect: guarded transcript marker append, queue-front recovery request, message-tool send, and committed channel receipt.followup-runner.ts, session JSONL, recovery message tool,qa-channelplugin, and loopback HTTP bus.persistMessageToolOnlyUndeliveredFinalNotice, which delegates topersistSessionTranscriptTurn; recovery and source-delivery classifiers remain shared. No provider-owned numeric/range contract is changed, and receipt classification is not widened globally.qa-mock-not-a-real-key.agent:qa:main; inject a normal QA-channel direct message into the same session; wait for queue/recovery/provider completion and a stable bus cursor; inspect gateway logs, session JSONL, provider requests, bus state, and final session state.message(action=send); the stable bus had one outbound; failure diagnostic count was 0; session status wasdonewithabortedLastRun: false.return truepreserves the queue mock's success contract; productionreturn falserejects missing source-route evidence. Broad catches intentionally keep established recovery available if marker persistence fails. Replay preserves marker adjacency across rotation rather than rewriting user content. The QA receipt adds an already-supported standard field at a private test-channel boundary. No casts, skipped tests, dependency changes, lockfile churn, or unrelated formatting are included.Verdict: PASS for the local real OpenClaw CLI/gateway/session/channel-plugin boundary.
The proof launched the actual OpenClaw CLI and gateway with isolated home, state, config, workspace, and session directories. It used the repository's loopback
qa-channelHTTP bus and loopback mock OpenAI server. The model output was deterministic, but the CLI command, gateway socket, session lane, queue,followup-runner.ts, transcript persistence, message tool dispatch, channel plugin, and HTTP bus were real runtime paths.No network channel credentials were configured. The only auth value was the local placeholder
qa-mock-not-a-real-key.Driven scenario
agent:qa:main; the local provider held it for 60 seconds.qa-channelHTTP bus.messages.queue.mode=followup.message(action=send).Same-session queue evidence
The second normal user message waited 58.238 seconds behind the active CLI run, proving it entered the same real session queue before
followup-runner.tshandled it.Durable transcript evidence
The redacted marker was:
{ "role": "custom", "customType": "openclaw.message-tool-only-undelivered-final", "display": false, "details": { "sourceReplyDeliveryMode": "message_tool_only", "delivered": false, "finalTextLength": 240 } }Marker-before-recovery ordering
The durable marker existed 194 ms before recovery enqueue and 976 ms before the recovery tool call.
Recovery receipt and stable channel state
{ "messageId": "5318c425-c8f1-4cd4-872b-3cc396551bf2", "message": { "direction": "outbound", "conversation": { "id": "qa-pr-101618-queued-fixed", "kind": "direct" }, "text": "QA-STRANDED-85714" } }After all provider requests completed and the bus cursor stabilized:
{ "outboundCount": 1, "outboundTexts": ["QA-STRANDED-85714"] }The final session state was:
{ "key": "agent:qa:main", "status": "done", "abortedLastRun": false }The provider observed four gateway requests: the slow CLI run, the queued substantive-final run, the recovery tool-planning run, and the post-tool completion. Only the recovery request planned the
messagetool.Adjacent runtime probe
The first full runtime attempt produced one successful QA bus delivery followed by an incorrect failure diagnostic and left the session
killedwithabortedLastRun: true. Tracing the tool result showed thatqa-channelreturned only nestedmessage.id, so committed source-delivery evidence never became true.A TDD regression first observed:
After returning
messageId: message.id, the focused test passed, all 14qa-channeltests passed, and the repeated real runtime produced exactly one outbound, no failure diagnostic, and a completed session.Proof boundary
This is a redacted artifact from a local real OpenClaw CLI/gateway/session/channel-plugin run outside Vitest. The run intentionally remained on isolated loopback QA infrastructure; no network transport implementation or account was involved.
Evidence
TDD and fresh verification
The queued-run regression was written first and failed because the JSONL captured at enqueue did not contain
openclaw.message-tool-only-undelivered-final. After the shared helper was wired into the follow-up runner, the same assertion passed.The QA receipt regression was also observed RED before the one-line production repair, then GREEN after the top-level receipt was added.
Commands run against the final pending diff:
The final auto-reply and E2E logs contain no module-resolution errors. Each run prints the repository's existing Vitest
envFiledeprecation notice before the passing summary; this patch does not change the test configuration.Regression Test Plan
src/auto-reply/reply/followup-runner.test.ts, the specified five-file auto-reply suite,src/auto-reply/reply/agent-runner.runreplyagent.e2e.test.ts, andextensions/qa-channel/src/channel.test.ts.message_tool_onlysubstantive final with no successful source delivery writes the hidden JSONL marker before enqueueing one protected recovery; retry runs do not duplicate it; a successful QA send exposes top-levelmessageIdand completes without a fallback diagnostic.Review findings addressed
followup-runner.tsnow awaits marker persistence before queue-front recovery enqueue, with both RED-to-GREEN JSONL ordering coverage and real runtime ordering evidence.git diff --checkpassed with no output.Merge risk
Risk labels considered:
compatibility,message-delivery,security-boundary, andsession-state.Risk explanation: The marker is persistent model-consumed state preserved across rotation; the runtime decision controls whether private text is treated as user-visible; and the QA receipt affects committed delivery classification. Incorrect scope, ordering, or ownership could create false conversation history or duplicate recovery.
Why acceptable: The marker is restricted to nonempty, non-silent, allowed
message_tool_onlyfinals without successful source delivery; writes are session/lifecycle guarded and occur before enqueue; retry runs cannot duplicate it; the QA change adds a standard receipt already recognized by the shared classifier; the real runtime ended with exactly one marker, one outbound, no failure diagnostic, and a completed session.What did NOT change: Public config schema/defaults, model service contracts, network channel implementations, message-tool API, retry count/prompt/priority, room-event handling, send-policy denial, intentional silence, dependencies, and lockfiles are unchanged. The related-main scan confirmed this existing PR remains necessary.
Compatibility/session-state: The hidden custom record survives replay/rotation and is consumed by later model context. That is deliberate: removing it would restore the user/model history mismatch. The record is limited to nonempty, non-silent, allowed
message_tool_onlyfinals without successful source delivery.Ordering: Persistence is awaited before recovery enqueue, so an exit between the original run and queue drain cannot lose the negative delivery fact.
Duplication: Recovery retries are marked and do not append another undelivered-final marker; the real transcript contained exactly one marker.
Stale ownership: Store-backed writes use expected session id and lifecycle revision guards.
Delivery: Successful committed source delivery suppresses the marker and recovery. The QA-only receipt change exposes the standard evidence already recognized by the shared classifier rather than weakening receipt classification globally.
Failure containment: A marker append failure is logged but does not block the existing one-shot recovery path.
Out of scope