Skip to content

Fix heartbeat reasoning payload selection#92263

Closed
harjothkhara wants to merge 2 commits into
openclaw:mainfrom
harjothkhara:codex/92260-heartbeat-reasoning
Closed

Fix heartbeat reasoning payload selection#92263
harjothkhara wants to merge 2 commits into
openclaw:mainfrom
harjothkhara:codex/92260-heartbeat-reasoning

Conversation

@harjothkhara

@harjothkhara harjothkhara commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #92260.

This fixes heartbeat main-reply selection so reasoning payloads are not delivered as the visible heartbeat reply when includeReasoning is unset. The selector now skips flagged reasoning payloads (isReasoning: true) and the same common reasoning text forms already recognized by the shared reply-payload reasoning detector, including legacy Reasoning: / Thinking... forms and blockquoted reasoning.

The runner still preserves the explicit opt-in path: when agents.defaults.heartbeat.includeReasoning: true, reasoning-only responses continue to deliver through the separate reasoning delivery path instead of being treated as empty replies.

Patch-level comparison

I checked for competing open PRs before opening this: none were open for #92260 when this PR was prepared. The closest competing patch was closed PR #92262.

Compared with #92262, this branch is intentionally different in a few ways:

  • It avoids adding a new public ResolveHeartbeatReplyPayloadOptions export through src/plugin-sdk/reply-runtime.ts.
  • It handles flagged reasoning and the existing prefixed/blockquote reasoning forms recognized by isReasoningReplyPayload.
  • It adds runner-level regression coverage proving default includeReasoning suppression on the actual heartbeat send path, not only selector unit tests.
  • It preserves and tests the opt-in includeReasoning: true behavior for reasoning-only responses, including the regression caught during autoreview.

Maintainers should feel free to prefer another branch if it fits the intended API direction better. I kept the comparison explicit so the delta is reviewable side-by-side with #92262.

Real behavior proof (required for external PRs)

  • Behavior or issue addressed: Heartbeat replies no longer select reasoning payloads as the main visible reply when reasoning is disabled by default.
  • Real environment tested: Local OpenClaw checkout at 1d36bdf2ce876dc18c7644da8e308f049fa23fc8, Node 24.16.0, production runHeartbeatOnce, real temporary OpenClawConfig, real session-store JSON file, real plugin registry registration, isolated OPENCLAW_HOME / OPENCLAW_STATE_DIR under /tmp, and a local Telegram-shaped outbound adapter that records delivery attempts without contacting Telegram.
  • Exact steps or command run after this patch:
OPENCLAW_PROOF_HEAD=1d36bdf2ce876dc18c7644da8e308f049fa23fc8 \
OPENCLAW_HOME=/tmp/openclaw-hb-proof-home \
OPENCLAW_STATE_DIR=/tmp/openclaw-hb-proof-home/state \
node --import tsx /tmp/openclaw-heartbeat-proof.ts
  • Evidence after fix (terminal capture / copied console output):
OpenClaw heartbeat behavior proof
head=1d36bdf2ce876dc18c7644da8e308f049fa23fc8
[
  {
    "case": "default suppresses late blockquoted reasoning after HEARTBEAT_OK",
    "includeReasoning": false,
    "status": "ran",
    "deliveryCount": 0,
    "deliveredTexts": [],
    "disallowedReasoningDelivered": false,
    "sessionStore": "sessions.json"
  },
  {
    "case": "default sends visible final and ignores later reasoning",
    "includeReasoning": false,
    "status": "ran",
    "deliveryCount": 1,
    "deliveredTexts": [
      "Visible heartbeat update"
    ],
    "disallowedReasoningDelivered": false,
    "sessionStore": "sessions.json"
  },
  {
    "case": "includeReasoning true still delivers reasoning-only output",
    "includeReasoning": true,
    "status": "ran",
    "deliveryCount": 1,
    "deliveredTexts": [
      "Thinking\n\n_private chain of thought_"
    ],
    "disallowedReasoningDelivered": false,
    "sessionStore": "sessions.json"
  }
]
  • Observed result after fix: With includeReasoning unset, a late blockquoted reasoning payload after HEARTBEAT_OK produced zero outbound sends, and a visible final heartbeat update followed by reasoning delivered only the visible final text. With includeReasoning: true, a reasoning-only response still delivered through the explicit reasoning path.
  • What was not tested: No live xAI/OpenAI Responses model and no real Telegram network delivery were used. The model reply ordering and outbound network call were locally controlled to isolate the heartbeat payload-selection behavior.
  • Proof limitations or environment constraints: Live Telegram/provider proof would require credentials and a model response ordering that reproduces the reporter's payload order; this local proof targets the heartbeat runner layer identified as the root cause in [Bug]: Heartbeat delivers reasoning payload as main reply when includeReasoning is false (resolveHeartbeatReplyPayload ignores isReasoning) #92260.

Tests and validation

Focused commands run:

node scripts/run-vitest.mjs src/auto-reply/heartbeat-reply-payload.test.ts src/infra/heartbeat-runner.returns-default-unset.test.ts src/plugin-sdk/reply-payload.test.ts
git diff --check
/Users/harjothk/.cache/codex-runtimes/codex-primary-runtime/dependencies/python/bin/python3 .agents/skills/autoreview/scripts/autoreview --mode local

Results:

  • node scripts/run-vitest.mjs ... passed 3 Vitest shards: 4 selector tests, 44 infra heartbeat runner tests, and 53 plugin-sdk reply-payload tests.
  • git diff --check passed.
  • Autoreview initially found two regressions; both were fixed. Final autoreview result: autoreview clean: no accepted/actionable findings reported.

Regression coverage added:

  • Selector tests cover late flagged reasoning, legacy-prefixed reasoning, blockquoted reasoning, reasoning-only inputs, and scan-through behavior.
  • Heartbeat runner tests cover default-hidden late reasoning after HEARTBEAT_OK, legacy-prefixed and blockquoted late reasoning, and includeReasoning: true reasoning-only delivery.

@openclaw-barnacle openclaw-barnacle Bot added size: S triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels Jun 11, 2026
@clawsweeper

clawsweeper Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Thanks for the context here. I swept through the related work, and this is now duplicate or superseded.

Close because the same heartbeat reasoning leak is now covered by the open, mergeable, proof-sufficient sibling PR at #92356; this branch is useful and plausible, but it no longer has a unique implementation or test gap that should keep a second landing candidate open.

Canonical path: Land one canonical heartbeat reasoning selector fix through #92356, then close the linked bug and overlapping branch.

So I’m closing this here and keeping the remaining discussion on #92356.

Review details

Best possible solution:

Land one canonical heartbeat reasoning selector fix through #92356, then close the linked bug and overlapping branch.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection of current main shows resolveHeartbeatReplyPayload can select a trailing reasoning payload, and the linked report plus PR proof give the concrete heartbeat payload order that leaks reasoning as the visible reply.

Is this the best way to solve the issue?

No for keeping this branch open as the landing path. The patch is plausible, but the cleaner current solution is the sibling PR that directly reuses the SDK classifier in the selector and runner and already has a completed proof-sufficient review.

Security review:

Security review cleared: The diff changes heartbeat payload-selection source and tests only; no dependency, workflow, secret, package, install, or code-execution surface change was found.

AGENTS.md: found and applied where relevant.

What I checked:

  • Repository policy read: Root AGENTS.md was read fully from the checkout and its PR review, proof, message-delivery, SDK-boundary, and duplicate cleanup rules shaped this review. (AGENTS.md:1, 865e4db1cd0a)
  • SDK boundary policy read: The plugin SDK scoped guide was read because the PR imports the shared reply-payload classifier; it favors one shared helper when core runtime and plugin-facing paths must stay aligned. (src/plugin-sdk/AGENTS.md:1, 865e4db1cd0a)
  • Current main root cause: Current main still returns scalar heartbeat payloads unchanged and selects the last outbound array payload without checking whether it is reasoning, so the linked bug remains unfixed on main. (src/auto-reply/heartbeat-reply-payload.ts:12, 865e4db1cd0a)
  • Existing expected behavior: Current docs say heartbeat delivery defaults to the final payload only, and separate Thinking delivery requires includeReasoning to be enabled. Public docs: docs/tools/thinking.md. (docs/tools/thinking.md:122, 865e4db1cd0a)
  • Shared classifier contract: The existing SDK helper already recognizes explicit isReasoning payloads, lowercase reasoning/thinking prefixes, and Markdown blockquoted forms, making it the right shared classifier for this fix. (src/plugin-sdk/reply-payload.ts:89, 865e4db1cd0a)
  • This PR implementation: At this PR head, the selector imports the SDK classifier through local heartbeat wrappers and skips reasoning payloads in scalar and array forms before selecting the visible reply. (src/auto-reply/heartbeat-reply-payload.ts:25, 1d36bdf2ce87)

Likely related people:

  • vincentkoc: Current blame for the selector, heartbeat runner, and SDK reply-payload helper points to the release build support rewrite, and earlier history shows the plugin SDK reply-payload subpath routing went through Vincent's commit. (role: recent area contributor and SDK subpath contributor; confidence: high; commits: 94833b2c9072, ef1346e50339; files: src/auto-reply/heartbeat-reply-payload.ts, src/infra/heartbeat-runner.ts, src/plugin-sdk/reply-payload.ts)
  • steipete: Targeted history shows Peter authored optional heartbeat reasoning delivery, a follow-up send-heartbeat-reasoning fix, and the reply-payload helper deduplication that this fix now relies on. (role: introduced heartbeat reasoning delivery and reply-payload helper history; confidence: high; commits: 3166cc911b4a, 1f9b4e3af64e, 62edfdffbdd0; files: src/infra/heartbeat-runner.ts, docs/gateway/heartbeat.md, src/plugin-sdk/reply-payload.ts)
  • antons: The history pass found the commit that delivered reasoning alongside HEARTBEAT_OK directly adjacent to the includeReasoning behavior preserved by these PRs. (role: adjacent heartbeat reasoning contributor; confidence: medium; commits: c7caa9a87d42; files: src/infra/heartbeat-runner.ts)

Codex review notes: model internal, reasoning high; reviewed against 865e4db1cd0a.

@clawsweeper clawsweeper Bot added the rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. label Jun 11, 2026
@openclaw-barnacle openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels Jun 11, 2026
@harjothkhara
harjothkhara marked this pull request as ready for review June 11, 2026 21:37
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. and removed rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. labels Jun 15, 2026
@clawsweeper

clawsweeper Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper applied the proposed close for this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Heartbeat delivers reasoning payload as main reply when includeReasoning is false (resolveHeartbeatReplyPayload ignores isReasoning)

1 participant