Skip to content

fix #100368: mark undelivered message_tool_only finals in replay#101618

Closed
mushuiyu886 wants to merge 5 commits into
openclaw:mainfrom
mushuiyu886:feat/issue-100368
Closed

fix #100368: mark undelivered message_tool_only finals in replay#101618
mushuiyu886 wants to merge 5 commits into
openclaw:mainfrom
mushuiyu886:feat/issue-100368

Conversation

@mushuiyu886

@mushuiyu886 mushuiyu886 commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Fixes message_tool_only: withheld finals stay in the agent's context as normal turns, desynchronizing agent memory from what the user saw #100368 by recording when a substantive message_tool_only final entered the transcript but was not delivered to the source channel.
  • Gives direct runs and queued follow-up runs the same durable behavior: append one hidden undelivered-final marker before enqueueing the existing one-shot recovery retry.
  • Keeps retry runs from appending a duplicate marker and preserves the marker through transcript replay/rotation so later model context matches what the user actually saw.
  • Adds a standard top-level messageId to the repository QA channel's successful send receipt. The real runtime proof exposed that its previous nested-only receipt made a successful recovery send look incomplete.

What Problem This Solves

A message_tool_only assistant final can be stored in session history even though the user never received it. Without a durable delivery marker, later model turns treat that private final as user-visible conversation state, desynchronizing agent memory from what the user actually saw.

The same failure also behaved differently across runners: direct runs recorded the undelivered state, while queued follow-up runs could enter recovery without doing so. This patch makes that session-state invariant runner-independent.

Root Cause

  • Root cause: Direct and queued follow-up runners applied different durable transcript behavior after the same message_tool_only stranded final: direct runs appended the undelivered marker, while queued follow-ups could enqueue recovery without recording that the final remained private.
  • Why this is root-cause fix: The patch restores the invariant at the shared runner/session boundary where normalized final text and committed source-delivery evidence become durable session state, before either runner starts recovery. It does not mask the later transcript symptom or infer transport success downstream.
  • Architecture / source-of-truth check: The source of truth is the completed run's normalized assistant final plus committed granular source-route delivery evidence. persistSessionTranscriptTurn is the canonical durable write, and the existing one-shot stranded-reply follow-up remains the canonical retry. Current origin/main still needs this patch, and the existing-PR diagnosis found no canonical replacement PR to defer to.

Under messages.visibleReplies: "message_tool", the runtime normalizes source delivery to sourceReplyDeliveryMode: "message_tool_only". An assistant final can therefore be durably present in the session transcript while remaining private because the model did not call message(action=send).

The existing direct-run recovery path appended a hidden undelivered-final marker, but the queued follow-up recovery path did not. Both paths could enqueue the same one-shot stranded-reply recovery, yet the same user-visible failure produced different durable model context depending on which runner executed it. A restart or later model turn could consequently treat a queued private final as text the user had seen.

The source of truth is the completed run's normalized assistant final plus committed source-route delivery evidence. The marker is the durable projection of the negative delivery fact; it does not infer provider state or replace transport receipts.

The outside-Vitest proof also found an adjacent QA boundary defect. qa-channel successfully wrote the recovery message to its HTTP bus but returned only { message: { id } }. The shared delivery classifier intentionally requires a standard committed receipt such as top-level messageId, so the successful send was classified as incomplete and an incorrect second failure diagnostic was emitted. The narrow repair is at the QA action boundary: retain the full message and also return messageId: message.id.

Canonical reachability

messages.visibleReplies: "message_tool"
  -> config schema/type/ingestion/normalization
  -> sourceReplyDeliveryMode: "message_tool_only"
  -> real gateway session lane and follow-up queue
  -> runEmbeddedAgent() final + committed delivery evidence
  -> direct runner or followup-runner.ts delivery decision
  -> guarded persistSessionTranscriptTurn(..., updateMode: "file-only")
  -> hidden openclaw.message-tool-only-undelivered-final record
  -> existing queue-front stranded-reply recovery
  -> message(action=send)
  -> channel action receipt
  -> SessionManager/transcript replay consumes the durable marker

The append is guarded by the active session id and lifecycle revision when a store path is available. Marker persistence is awaited before recovery enqueue. A persistence failure remains contained so it cannot suppress the already-established recovery path.

What changed

  • Extracted the direct runner's guarded marker append into a shared helper.
  • Called the same helper from the queued follow-up runner before evaluating/enqueueing stranded-final recovery.
  • Reused the run's successful source-delivery result for both marker suppression and stranded-final detection.
  • Preserved the existing retry guard so strandedReplyRetry runs do not append the marker again.
  • Added a regression assertion that reads the real temporary session JSONL from the enqueue callback and proves the marker already exists.
  • Added top-level messageId to successful qa-channel send results and covered it at the channel action boundary.

Real behavior proof

  • Behavior or issue addressed: A queued substantive final that was saved but not delivered under message_tool_only must append one hidden marker before recovery so later model context does not treat private text as user-visible history.
  • Canonical reachability path: User/config/input messages.visibleReplies: "message_tool" → schema/type/ingestion/normalization → runtime object sourceReplyDeliveryMode: "message_tool_only" on the embedded/follow-up run → request/effect: guarded transcript marker append, queue-front recovery request, message-tool send, and committed channel receipt.
  • Boundary crossed: Actual OpenClaw CLI command, gateway socket, same-session lane, follow-up queue, followup-runner.ts, session JSONL, recovery message tool, qa-channel plugin, and loopback HTTP bus.
  • Shared helper / provider constraint check: Both runners reuse persistMessageToolOnlyUndeliveredFinalNotice, which delegates to persistSessionTranscriptTurn; recovery and source-delivery classifiers remain shared. No provider-owned numeric/range contract is changed, and receipt classification is not widened globally.
  • Real environment tested: Linux OpenClaw worktree with isolated home/state/config/workspace/session directories, real CLI/gateway runtime, repository loopback QA channel, and repository loopback mock provider. Authentication used only the local placeholder qa-mock-not-a-real-key.
  • Exact steps or command run after this patch: Start private-QA mock provider, QA HTTP bus, and gateway; start a held CLI run on agent:qa:main; inject a normal QA-channel direct message into the same session; wait for queue/recovery/provider completion and a stable bus cursor; inspect gateway logs, session JSONL, provider requests, bus state, and final session state.
  • Evidence after fix: The queue waited 58.238 seconds behind the active run; marker count was 1; the marker preceded recovery enqueue by 194 ms; the recovery called message(action=send); the stable bus had one outbound; failure diagnostic count was 0; session status was done with abortedLastRun: false.
  • Observed result after fix: The queued private final was followed by the hidden durable marker before recovery, the recovery send produced a standard committed receipt, no duplicate marker or fallback diagnostic was emitted, and later transcript replay retained the delivery fact.
  • What was not tested: The proof did not leave the isolated loopback QA environment. That is the intended boundary because this PR changes session-state handling and the repository QA action receipt; network transport implementations and accounts are not modified.
  • Why it matters / User impact: Without the marker, later agent turns can reason from a reply the user never saw. Restoring runner parity keeps agent memory synchronized with the user's actual conversation and survives process exits between the original run and recovery drain.
  • Fix classification: Root-cause fix at the runner/session-state source-of-truth boundary, plus a narrow repository-QA receipt contract repair discovered by the runtime proof.
  • Maintainer-ready confidence: High for the local OpenClaw runtime/session/channel-plugin boundary: code order, TDD regressions, replay coverage, fresh suites, and outside-Vitest runtime evidence agree. Maintainer acceptance remains explicit for the persistent model-consumed marker representation.
  • Patch quality notes: The 11-file/734-line cumulative PR includes the original marker, replay, direct-run coverage, and this repair; the new repair commit itself is limited to six files. Retry-related scanner hits are assertions for the existing one-shot recovery, not retry-policy changes. Test return true preserves the queue mock's success contract; production return false rejects missing source-route evidence. Broad catches intentionally keep established recovery available if marker persistence fails. Replay preserves marker adjacency across rotation rather than rewriting user content. The QA receipt adds an already-supported standard field at a private test-channel boundary. No casts, skipped tests, dependency changes, lockfile churn, or unrelated formatting are included.

Verdict: PASS for the local real OpenClaw CLI/gateway/session/channel-plugin boundary.

The proof launched the actual OpenClaw CLI and gateway with isolated home, state, config, workspace, and session directories. It used the repository's loopback qa-channel HTTP bus and loopback mock OpenAI server. The model output was deterministic, but the CLI command, gateway socket, session lane, queue, followup-runner.ts, transcript persistence, message tool dispatch, channel plugin, and HTTP bus were real runtime paths.

OpenClaw 2026.6.11 (version string HEAD 974adc9, with the pending PR diff built)
gateway:       http://127.0.0.1:18829
qa-channel:    http://127.0.0.1:43163
mock provider: http://127.0.0.1:44083
session key:   agent:qa:main
queue mode:    followup
visibleReplies: message_tool

No network channel credentials were configured. The only auth value was the local placeholder qa-mock-not-a-real-key.

Driven scenario

  1. A real CLI agent run started on agent:qa:main; the local provider held it for 60 seconds.
  2. While that run was active, a normal direct inbound message was posted to the real qa-channel HTTP bus.
  3. The gateway routed the second message to the same session lane under messages.queue.mode=followup.
  4. The queued run produced a substantive final without calling the message tool.
  5. The follow-up runner persisted the hidden marker, enqueued recovery, and the recovery run called message(action=send).
  6. Evidence was frozen only after provider work completed and the QA bus cursor remained stable.

Same-session queue evidence

11:25:52.419 lane enqueue: lane=session:agent:qa:main queueSize=2
11:26:50.656 lane wait exceeded: waitedMs=58238 activeAhead=1
11:26:50.659 lane dequeue: waitMs=58238 queueSize=0

The second normal user message waited 58.238 seconds behind the active CLI run, proving it entered the same real session queue before followup-runner.ts handled it.

Durable transcript evidence

line 8   user       queued qa-channel inbound
line 9   assistant  substantive private final, length 240
line 11  custom     openclaw.message-tool-only-undelivered-final
line 12  assistant  message(action=send, message=QA-STRANDED-85714)
line 13  assistant  delivery mirror
line 15  toolResult successful qa-channel outbound receipt
line 16  assistant  empty post-tool completion

The redacted marker was:

{
  "role": "custom",
  "customType": "openclaw.message-tool-only-undelivered-final",
  "display": false,
  "details": {
    "sourceReplyDeliveryMode": "message_tool_only",
    "delivered": false,
    "finalTextLength": 240
  }
}
marker count:             1
failure diagnostic count: 0

Marker-before-recovery ordering

11:26:51.844 transcript marker appended
11:26:52.038 recovery lane enqueue
11:26:52.820 recovery message tool call appended

The durable marker existed 194 ms before recovery enqueue and 976 ms before the recovery tool call.

Recovery receipt and stable channel state

{
  "messageId": "5318c425-c8f1-4cd4-872b-3cc396551bf2",
  "message": {
    "direction": "outbound",
    "conversation": {
      "id": "qa-pr-101618-queued-fixed",
      "kind": "direct"
    },
    "text": "QA-STRANDED-85714"
  }
}

After all provider requests completed and the bus cursor stabilized:

{
  "outboundCount": 1,
  "outboundTexts": ["QA-STRANDED-85714"]
}

The final session state was:

{
  "key": "agent:qa:main",
  "status": "done",
  "abortedLastRun": false
}

The provider observed four gateway requests: the slow CLI run, the queued substantive-final run, the recovery tool-planning run, and the post-tool completion. Only the recovery request planned the message tool.

Adjacent runtime probe

The first full runtime attempt produced one successful QA bus delivery followed by an incorrect failure diagnostic and left the session killed with abortedLastRun: true. Tracing the tool result showed that qa-channel returned only nested message.id, so committed source-delivery evidence never became true.

A TDD regression first observed:

expected undefined to be '<qa message id>'

After returning messageId: message.id, the focused test passed, all 14 qa-channel tests passed, and the repeated real runtime produced exactly one outbound, no failure diagnostic, and a completed session.

Proof boundary

This is a redacted artifact from a local real OpenClaw CLI/gateway/session/channel-plugin run outside Vitest. The run intentionally remained on isolated loopback QA infrastructure; no network transport implementation or account was involved.

Evidence

TDD and fresh verification

The queued-run regression was written first and failed because the JSONL captured at enqueue did not contain openclaw.message-tool-only-undelivered-final. After the shared helper was wired into the follow-up runner, the same assertion passed.

The QA receipt regression was also observed RED before the one-line production repair, then GREEN after the top-level receipt was added.

Commands run against the final pending diff:

node scripts/run-vitest.mjs \
  --config test/vitest/vitest.auto-reply-reply.config.ts \
  reply/agent-runner.misc.runreplyagent.test.ts \
  reply/followup-runner.test.ts \
  reply/session-transcript-replay.test.ts \
  reply/stranded-reply-recovery.test.ts \
  reply/agent-runner-payloads.test.ts

Test Files  5 passed (5)
Tests       293 passed (293)
node scripts/run-vitest.mjs \
  --config test/vitest/vitest.e2e.config.ts \
  src/auto-reply/reply/agent-runner.runreplyagent.e2e.test.ts

Test Files  1 passed (1)
Tests       92 passed (92)
qa-channel focused receipt regression: 1 passed
qa-channel complete test file:          14 passed
git diff --check:                       passed with no output

The final auto-reply and E2E logs contain no module-resolution errors. Each run prints the repository's existing Vitest envFile deprecation notice before the passing summary; this patch does not change the test configuration.

Regression Test Plan

  • Target test file: src/auto-reply/reply/followup-runner.test.ts, the specified five-file auto-reply suite, src/auto-reply/reply/agent-runner.runreplyagent.e2e.test.ts, and extensions/qa-channel/src/channel.test.ts.
  • Scenario locked in: A queued message_tool_only substantive final with no successful source delivery writes the hidden JSONL marker before enqueueing one protected recovery; retry runs do not duplicate it; a successful QA send exposes top-level messageId and completes without a fallback diagnostic.
  • Why this is the smallest reliable guardrail: The focused follow-up test observes the real temporary JSONL at the enqueue callback, the existing E2E/replay suites protect durable ordering and model consumption, and the QA action test checks the receipt at the exact plugin boundary. The outside-Vitest run separately proves those seams operate together in the actual CLI/gateway/session flow.

Review findings addressed

  • RF-001: Added the redacted outside-Vitest OpenClaw runtime/session artifact above.
  • RF-002: Drove the actual CLI, gateway socket, same-session lane, follow-up queue, transcript JSONL, message tool, repository channel plugin, and loopback HTTP bus. This matches the changed runtime/session boundary.
  • RF-003: Direct and queued follow-up runners now call the same guarded persistence helper and produce the same durable model context for equivalent undelivered finals.
  • RF-004: The marker is intentionally persistent, model-consumed session state rather than an internal diagnostic. Replay/rotation preserves it because later runs must know the preceding assistant final was private. This compatibility/session-state representation remains an explicit maintainer acceptance decision.
  • RF-005: The narrow shared-runner repair is complete. Contributor-supplied real-runtime proof is supplied separately above; durable-marker product acceptance remains a separate maintainer judgment rather than being hidden inside the code fix.
  • RF-006: followup-runner.ts now awaits marker persistence before queue-front recovery enqueue, with both RED-to-GREEN JSONL ordering coverage and real runtime ordering evidence.
  • RF-007: Fresh specified auto-reply suite passed: 5 files, 293 tests.
  • RF-008: Fresh specified E2E file passed: 1 file, 92 tests.
  • RF-009: git diff --check passed with no output.

Merge risk

  • Risk labels considered: compatibility, message-delivery, security-boundary, and session-state.

  • Risk explanation: The marker is persistent model-consumed state preserved across rotation; the runtime decision controls whether private text is treated as user-visible; and the QA receipt affects committed delivery classification. Incorrect scope, ordering, or ownership could create false conversation history or duplicate recovery.

  • Why acceptable: The marker is restricted to nonempty, non-silent, allowed message_tool_only finals without successful source delivery; writes are session/lifecycle guarded and occur before enqueue; retry runs cannot duplicate it; the QA change adds a standard receipt already recognized by the shared classifier; the real runtime ended with exactly one marker, one outbound, no failure diagnostic, and a completed session.

  • What did NOT change: Public config schema/defaults, model service contracts, network channel implementations, message-tool API, retry count/prompt/priority, room-event handling, send-policy denial, intentional silence, dependencies, and lockfiles are unchanged. The related-main scan confirmed this existing PR remains necessary.

  • Compatibility/session-state: The hidden custom record survives replay/rotation and is consumed by later model context. That is deliberate: removing it would restore the user/model history mismatch. The record is limited to nonempty, non-silent, allowed message_tool_only finals without successful source delivery.

  • Ordering: Persistence is awaited before recovery enqueue, so an exit between the original run and queue drain cannot lose the negative delivery fact.

  • Duplication: Recovery retries are marked and do not append another undelivered-final marker; the real transcript contained exactly one marker.

  • Stale ownership: Store-backed writes use expected session id and lifecycle revision guards.

  • Delivery: Successful committed source delivery suppresses the marker and recovery. The QA-only receipt change exposes the standard evidence already recognized by the shared classifier rather than weakening receipt classification globally.

  • Failure containment: A marker append failure is logged but does not block the existing one-shot recovery path.

Out of scope

  • No public config schema/default, model service contract, network channel implementation, retry count, retry priority, recovery prompt, room-event behavior, send-policy denial, or intentional-silence behavior changes.
  • No dependency or lockfile changes.
  • Runtime evidence is scoped to the isolated loopback QA boundary exercised above.

@clawsweeper

clawsweeper Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs changes before merge. Reviewed July 11, 2026, 3:19 AM ET / 07:19 UTC.

Summary
The PR adds durable undelivered-final transcript markers for direct and queued message_tool_only runs, preserves them during transcript replay, and standardizes successful QA-channel send receipts.

PR surface: Source +262, Tests +338. Total +600 across 11 files.

Reproducibility: yes. at source level. The queued runner can observe a successful unrelated source payload and then suppress the marker for a different undelivered final; the supplied live proof covers the ordinary no-send recovery path but not this branch.

Review metrics: 1 noteworthy metric.

  • Persistent session records: 1 added. The PR adds a hidden custom transcript record that later model runs consume and session rotation preserves.

Stored data model
Persistent data-model change detected: serialized state: src/auto-reply/reply/agent-runner.runreplyagent.e2e.test.ts, serialized state: src/auto-reply/reply/session-transcript-replay.test.ts, serialized state: src/config/sessions/transcript-replay.ts, serialized state: src/config/sessions/undelivered-final-notice.ts, unknown-data-model-change: src/auto-reply/reply/session-transcript-replay.test.ts, unknown-data-model-change: src/config/sessions/transcript-replay.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #100368
Summary: This PR is the candidate fix for the durable withheld-final context issue; merged recovery work and open pre-tool replay work cover adjacent portions of delivery-context consistency.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🦞 diamond lobster
Patch quality: 🦐 gold shrimp
Result: needs maintainer review before merge.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P2] Make queued suppression require committed delivery of the normalized final text itself and add queued unrelated-send and fragment regressions.

Risk before merge

  • [P1] A queued run can send unrelated text or a fragment successfully and then return a different undelivered final; the current classifier suppresses the marker and leaves later model context inconsistent with what the user saw.
  • [P1] The new hidden custom record becomes durable, model-consumed session state and survives transcript rotation, so its long-term compatibility and representation semantics require explicit maintainer acceptance.

Maintainer options:

  1. Fix queued text matching (recommended)
    Reuse the direct runner's normalized exact-final/current-route classifier in the follow-up runner and add queued unrelated-send and fragment regressions before merge.
  2. Accept coarse queued evidence
    Merge while intentionally treating any successful source-route send as delivery of the queued final, accepting possible transcript and user-state drift.
  3. Pause persistent markers
    Defer the PR if maintainers do not want a new durable model-consumed transcript record or its replay semantics.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
In the queued follow-up path, suppress the undelivered-final marker only when the normalized final text itself has committed delivery evidence for the current source route; reuse a shared classifier with the direct runner and add regression coverage for an unrelated successful send and a fragment send followed by a different substantive final.

Next step before merge

  • [P2] A narrow mechanical repair can align the queued runner with the direct runner's exact-final-text delivery test; maintainer acceptance of the persistent marker remains a separate decision afterward.

Maintainer decision needed

  • Question: After the queued exact-text defect is fixed, should OpenClaw adopt this hidden custom transcript record as the canonical durable representation of an undelivered message_tool_only final?
  • Rationale: The PR changes persisted, model-consumed session semantics and carries the record across rotation; code review can verify the mechanics but cannot choose the permanent transcript contract.
  • Likely owner: steipete — Current-main history shows the strongest available routing signal across the affected auto-reply and session-runner paths.
  • Options:
    • Adopt the durable marker (recommended): Accept the hidden custom record and replay semantics after both runners share exact-final delivery classification and compatibility coverage is complete.
    • Use a replay-only projection: Leave stored transcripts unchanged and derive an undelivered annotation only while constructing model context, requiring a different ownership and lifecycle design.
    • Defer the representation: Pause this PR until maintainers choose a broader delivery-state model covering withheld finals, proactive sends, and pre-tool text.

Security
Cleared: The diff adds no dependency, workflow, permission, secret, download, publishing, or third-party execution changes; the QA plugin receipt addition remains inside its existing action boundary.

Review findings

  • [P1] Match the final text before suppressing the queued marker — src/auto-reply/reply/followup-runner.ts:1691-1694
Review details

Best possible solution:

Share one normalized exact-final/current-source-route delivery classifier between direct and queued runners, retain the durable marker and replay behavior only after that invariant has focused fresh-session and rotation coverage, and have maintainers explicitly adopt the custom record as the canonical persisted representation.

Do we have a high-confidence way to reproduce the issue?

Yes at source level. The queued runner can observe a successful unrelated source payload and then suppress the marker for a different undelivered final; the supplied live proof covers the ordinary no-send recovery path but not this branch.

Is this the best way to solve the issue?

No in its current form. A durable annotation at the runner/session boundary is a plausible root-cause solution, but both runners must enforce the same exact-final delivery invariant before the representation is safe.

Full review comments:

  • [P1] Match the final text before suppressing the queued marker — src/auto-reply/reply/followup-runner.ts:1691-1694
    hasSuccessfulFollowupSourceReplyDelivery becomes true for any visible source payload, so a queued run that sends a status update or fragment and then returns a different substantive final skips the durable undelivered marker. Reuse the direct runner's normalized exact-final/current-route check here and cover unrelated and fragment sends in followup-runner.test.ts.
    Confidence: 0.99

Overall correctness: patch is incorrect
Overall confidence: 0.98

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 34a847227473.

Label changes

Label justifications:

  • P1: The PR targets an active message-loss and session-state mismatch, but its queued path can still misclassify an undelivered final after an unrelated successful send.
  • merge-risk: 🚨 compatibility: The patch introduces a new durable transcript record and changes replay output for existing session-rotation flows.
  • merge-risk: 🚨 session-state: Incorrect suppression or replay of the marker can leave model context inconsistent with what the user actually received.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦞 diamond lobster and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (logs): The PR body provides detailed after-fix CLI, gateway, session-transcript, provider-request, and QA-channel logs showing marker ordering, one recovery send, committed receipt classification, and successful session completion.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body provides detailed after-fix CLI, gateway, session-transcript, provider-request, and QA-channel logs showing marker ordering, one recovery send, committed receipt classification, and successful session completion.
Evidence reviewed

PR surface:

Source +262, Tests +338. Total +600 across 11 files.

View PR surface stats
Area Files Added Removed Net
Source 6 290 28 +262
Tests 5 377 39 +338
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 11 667 67 +600

Acceptance criteria:

  • [P1] node scripts/run-vitest.mjs src/auto-reply/reply/agent-runner.misc.runreplyagent.test.ts src/auto-reply/reply/followup-runner.test.ts.
  • [P1] node scripts/run-vitest.mjs src/auto-reply/reply/agent-runner.runreplyagent.e2e.test.ts src/auto-reply/reply/session-transcript-replay.test.ts.
  • [P1] node scripts/run-vitest.mjs extensions/qa-channel/src/channel.test.ts.
  • [P1] pnpm check:changed -- src/auto-reply/reply/agent-runner.ts src/auto-reply/reply/followup-runner.ts src/auto-reply/reply/undelivered-final-notice.ts src/config/sessions/transcript-replay.ts extensions/qa-channel/src/channel-actions.ts.
  • [P1] git diff --check.

What I checked:

  • Queued classifier remains too coarse: The follow-up runner computes successfulSourceReplyDelivery from any visible source payload and passes that boolean as finalTextDeliveredToCurrentSourceRoute, without checking whether the delivered text equals the normalized final. (src/auto-reply/reply/followup-runner.ts:1691, fd1ff593c1f1)
  • Direct runner uses the stronger invariant: The direct path compares normalized sent text with the normalized assistant final and constrains delivery evidence to the current source route, demonstrating the intended classifier already exists in this PR. (src/auto-reply/reply/agent-runner.ts:229, fd1ff593c1f1)
  • Prior blocker is unchanged: The merge-from-main update did not modify the follow-up runner or its tests relative to the previously reviewed head, so the prior exact-text finding remains unresolved rather than being a late discovery. (src/auto-reply/reply/followup-runner.ts:1691, 3acef2bb4ff8)
  • Persistent session representation added: The helper appends a hidden custom transcript message using the canonical session writer, and replay explicitly carries that record alongside its assistant turn across session rotation. (src/auto-reply/reply/undelivered-final-notice.ts:44, fd1ff593c1f1)
  • Existing recovery provenance: Merged pull request Recover stranded message-tool finals by default #99536 introduced default stranded-final recovery on current main; this PR extends that behavior with durable delivery-state context rather than duplicating the recovery implementation. (src/auto-reply/reply/followup-runner.ts:1637, ae63a48e943f)
  • After-fix runtime proof supplied: The PR body documents a real CLI/gateway/same-session follow-up run where one marker preceded recovery, the recovery produced one QA-channel outbound with a committed receipt, no duplicate diagnostic appeared, and the session finished successfully. (fd1ff593c1f1)

Likely related people:

  • 100yenadmin: Authored the merged default stranded-final recovery work that this PR extends with durable transcript state. (role: introduced adjacent behavior; confidence: high; commits: ae63a48e943f; files: src/auto-reply/reply/agent-runner.ts, src/auto-reply/reply/followup-runner.ts)
  • steipete: Recent current-main history and blame show continued work across the central auto-reply and follow-up runner surfaces. (role: recent area contributor; confidence: medium; commits: f94a7dc183c6, e0862e609acb; files: src/auto-reply/reply/agent-runner.ts, src/auto-reply/reply/followup-runner.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (8 earlier review cycles)
  • reviewed 2026-07-07T12:12:47.845Z sha 304428d :: needs changes before merge. :: [P2] Do not skip the notice after unrelated tool sends
  • reviewed 2026-07-07T13:33:00.126Z sha ef5c13c :: needs maintainer review before merge. :: none
  • reviewed 2026-07-07T14:03:41.113Z sha ec54007 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-07T17:02:56.003Z sha ff72d71 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-10T21:14:49.688Z sha c07cebb :: needs changes before merge. :: [P1] Annotate stranded finals from the follow-up runner too
  • reviewed 2026-07-10T21:22:54.319Z sha c07cebb :: needs real behavior proof before merge. :: [P1] Annotate stranded finals from the follow-up runner too
  • reviewed 2026-07-10T21:32:17.920Z sha c07cebb :: needs real behavior proof before merge. :: [P1] Annotate stranded finals from the follow-up runner too
  • reviewed 2026-07-11T06:37:07.826Z sha 3acef2b :: needs changes before merge. :: [P1] Match the final text before suppressing the queued marker

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels Jul 7, 2026
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jul 7, 2026
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jul 10, 2026
@openclaw-barnacle openclaw-barnacle Bot added the channel: qa-channel Channel integration: qa-channel label Jul 11, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: qa-channel Channel integration: qa-channel merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P1 High-priority user-facing bug, regression, or broken workflow. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. size: L status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

message_tool_only: withheld finals stay in the agent's context as normal turns, desynchronizing agent memory from what the user saw

1 participant