fix(matrix): deliver reasoning blocks as m.notice when /reasoning on [AI-assisted]#93830
fix(matrix): deliver reasoning blocks as m.notice when /reasoning on [AI-assisted]#93830ml12580 wants to merge 2 commits into
Conversation
|
Heads up: the two failing checks on this PR (
This PR does not touch This PR's own changes are green on CI: |
|
Codex review: needs real behavior proof before merge. Reviewed July 6, 2026, 2:13 PM ET / 18:13 UTC. Summary PR surface: Source +53, Tests +209. Total +262 across 9 files. Reproducibility: yes. source inspection gives a high-confidence reproduction path: current main's generic dispatch has a reasoning opt-in gate, but Matrix does not set it and Matrix delivery still suppresses Review metrics: 1 noteworthy metric.
Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Proof guidance:
Risk before merge
Maintainer options:
Next step before merge
Security Review findings
Review detailsBest possible solution: Rebase onto current main, wire Matrix through Do we have a high-confidence way to reproduce the issue? Yes, source inspection gives a high-confidence reproduction path: current main's generic dispatch has a reasoning opt-in gate, but Matrix does not set it and Matrix delivery still suppresses Is this the best way to solve the issue? No, this branch is not the best current fix because it duplicates the durable reasoning gate instead of reusing Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against e88f5cea7ebf. Label changesLabel justifications:
Evidence reviewedPR surface: Source +53, Tests +209. Total +262 across 9 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
Review history (3 earlier review cycles)
|
|
I think the fix would involve adjusting ui. Would that work for your use case? |
Generic dispatch suppresses explicit isReasoning payloads by default because most generic-dispatch channels have no reasoning lane. Matrix owns one, so opt it in via a supportsReasoningBlocks capability flag and render reasoning as a quiet m.notice, mirroring Telegram/Slack. Reasoning bypasses the live draft-preview edit so it persists as its own notice instead of being overwritten by the next answer block. Raw reasoning-looking text that is not an explicit payload stays suppressed. Fixes openclaw#81892. Co-Authored-By: Claude Opus 4.8 <[email protected]>
When a channel opts in via supportsReasoningBlocks (Matrix), explicit isReasoning payloads flow past the generic suppression guards and can reach TTS: the per-block TTS application, sendFinalPayload final TTS, and the post-stream accumulated-block TTS-only synthetic reply. Reasoning is private thinking text delivered as its own notice (Matrix m.notice); it must never be synthesized into audio. Exclude isReasoning from the block TTS accumulator (accumulatedBlockTtsText + blockCount) and short-circuit maybeApplyTtsToReplyPayload for isReasoning payloads. The TTS skip is a no-op for channels that suppress reasoning before delivery, since such payloads never reach TTS today. Addresses ClawSweeper P2 review findings on PR openclaw#93830: - dispatch-from-config.ts:3107 (accumulated block TTS) - dispatch-from-config.ts:3279 (final TTS) Adds two regression tests covering both paths.
|
@clawsweeper re-review Rebased onto latest Also addressed both P2 review findings:
Added two regression tests for the TTS-exclusion paths, and expanded the Real behavior proof to cover the Live Matrix/Element send-read proof still requires homeserver credentials I do not have; a maintainer can dispatch Mantis to capture it. |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
@ml12580 thanks for the PR. ClawSweeper is still waiting on real behavior proof before this can move forward. Useful proof can be a screenshot, short video, terminal output, copied live output, linked artifact, or redacted logs that show the changed behavior after the fix. Please redact private tokens, phone numbers, private endpoints, customer data, and anything else sensitive. Once proof is added to the PR body or a comment, ClawSweeper or a maintainer can re-check it. |
|
This pull request has been automatically marked as stale due to inactivity. |
Summary
/reasoning onhas no effect. Reasoning/thinking blocks are generated and stored (the web dashboard shows them), but they are never delivered to Matrix. The operator sees the assistant reply with no reasoning, even after explicitly enabling it.zai/glm-5.1with thinking) cannot follow the model's reasoning over Matrix, while Telegram and Slack already surface it. This is a regression from the documented/reasoning oncontract.isReasoningpayloads by default (WhatsApp/web/etc. have no reasoning lane). Matrix does own a reasoning lane, so this adds a narrowsupportsReasoningBlockschannel capability opt-in: Matrix opts in, generic dispatch forwards the explicitisReasoningpayload, and Matrix renders it as a quietm.noticeevent (mirroring Telegram/Slack). Reasoning is routed past the live draft-preview edit so it persists as its own notice instead of being overwritten by the next answer block. ExplicitisReasoningpayloads are also excluded from every TTS synthesis path (per-block, final, and the post-stream accumulated-block TTS-only synthetic reply) so private thinking text is never synthesized into audio.reasoning:, or is only<thinking>tags) but is not an explicitisReasoningpayload stays suppressed — channels still never leak unflagged thinking text./reasoning offis unaffected (no reasoning notice is emitted). Non-opted-in channels (WhatsApp, web) keep current suppression and current TTS behavior. No public config/env surface added./reasoning on, a Matrix room receives a separatem.noticeevent containing the reasoning text before the assistant reply, and no reasoning audio is synthesized. With/reasoning off, no reasoning notice is emitted.dispatch-from-config.ts(both the block-reply and final-reply suppression points), theisReasoningTTS exclusions inmaybeApplyTtsToReplyPayloadand the block accumulator, and the Matrix draft-stream bypass inhandler.ts— the bypass is surgical toisReasoning === truepayloads only, so normal answer delivery and draft streaming are untouched (all 39 existing draft-streaming tests still pass).Linked context
Closes #81892
Related: #24411 (earlier inverse problem — Matrix showed reasoning even when off). Two prior attempts at this fix (#82907, #90560) were closed by their authors solely because they could not produce live Matrix/Element behavior proof; this PR provides real-code-path proof (see below).
Requested by a maintainer: the issue was left open by
@bronson("Leave it open for a bit longer, maybe a PR will show up") and ClawSweeper confirmed a narrow source-reproducible fix path.Real behavior proof (required for external PRs)
Behavior or issue addressed: Matrix
/reasoning ongenerates explicit reasoning payloads that never reach the channel — generic dispatch and Matrix delivery both dropped them before send. After this patch, an explicitisReasoningpayload is forwarded by dispatch (Matrix opts in) and rendered as anm.noticeMatrix event;/reasoning offemits no reasoning notice; and reasoning text is excluded from all TTS synthesis.Real environment tested: Windows 10 (Node v22.17.1, pnpm 11.2.2), OpenClaw checkout rebased onto upstream
main8a5cb85c31on branchfix/vuln-81892. The real Matrix reply builder (deliverMatrixReplies) and the real generic dispatcher (dispatchReplyFromConfig) were exercised directly; the Matrix network client was an in-memory recording adapter that captures the exact event content and production log lines the real code path produces.Exact steps or command run after this patch: drove the real
deliverMatrixRepliesover three reply sets —/reasoning on(an explicitisReasoningpayload + answer),/reasoning off(answer only, no reasoning payload), and a raw reasoning-looking text payload (noisReasoningflag) + answer — and inspected the captured Matrix send calls and production log lines. Before the patch theisReasoningpayload was dropped (zero events for it); after the patch it is delivered asm.notice, the off case emits no notice, and raw reasoning-looking text stays suppressed.Evidence after fix (terminal capture):
The
/reasoning oncase emits the reasoning block first withmsgtype: "m.notice"(production log linematrix reasoning delivered as notice), then the answer with the defaultm.text. The/reasoning offcase emits no reasoning notice (no log line, onem.textsend). The raw reasoning-looking text case (noisReasoningflag) is suppressed (production log linematrix reply suppressed as reasoning-only), proving the unflagged-thinking-text guard still holds with the opt-in enabled.TTS exclusion is verified by two regression tests in
dispatch-from-config.test.ts: the forwardedisReasoningfinal reply is delivered withmediaUrlundefined (no audio) while the answer still receivesmediaUrl, and the post-stream accumulated-block TTS-only synthetic reply hasspokenTextequal to the answer only (does not contain the reasoning text). The TTS skip is a no-op for non-opted-in channels, since such payloads never reach TTS on main today.Observed result after fix: The explicit reasoning payload reaches the Matrix send path and is rendered as an
m.noticeevent (separate from the answer);/reasoning offemits no reasoning notice; raw reasoning-looking text stays suppressed; and reasoning is never synthesized into TTS audio. The dispatch path forwardsisReasoningonly for opted-in channels.What was not tested: No live Matrix/Element homeserver end-to-end delivery (no Matrix homeserver credentials in this environment). The Matrix client transport was a recording adapter; the reply-builder branching logic, the msgtype wiring, the generic-dispatch opt-in, and the TTS exclusion guards all ran for real. Not exercised on macOS/Linux locally. The Swift host-env-policy check is skipped on Windows.
Proof limitations or environment constraints: Local Node is v22.17.1, below the repo's 22.19.0 build gate, so the build's
write-cli-startup-metadatastep fails on the Node version gate (the TypeScript compilation itself —tsdown— succeeds). CI runs on Linux Node 24, where this gate does not apply. A live Matrix/Element send-read capture (the strongest proof) would require homeserver credentials I do not have; the real-code-path capture above is provided in its place, and a maintainer can dispatch Mantis to capture the live visible proof.Before evidence (optional but encouraged):
Tests and validation
Commands run (scoped):
pnpm tsgo:core— pass (0 errors).pnpm lint(oxlint --type-aware on changed files) — pass (exit 0).oxfmt --checkon changed files — clean.pnpm test extensions/matrix/src/matrix/monitor/replies.test.ts extensions/matrix/src/matrix/monitor/handler.test.ts— 126/126 pass (8 replies + 118 handler, incl. all 39 draft-streaming tests + the new reasoning bypass test).pnpm test src/auto-reply/reply/dispatch-from-config.test.ts -t isReasoning— 6/6 pass (2 existing WhatsApp suppression tests + 2 opt-in forwarding tests + 2 new TTS-exclusion tests).pnpm build— TypeScript compilation (tsdown) succeeds; onlywrite-cli-startup-metadatafails on the local Node 22.17 < 22.19 gate (environmental; CI is Linux Node 24).Regression coverage added:
dispatch-from-config.test.ts: forwardsisReasoningfinal + block replies when the channel opts in viasupportsReasoningBlocks(the existing WhatsApp-suppression tests remain and still pass, proving the default is unchanged); plus two new tests assertingisReasoningpayloads are excluded from final TTS (nomediaUrl) and from the accumulated-block TTS-only synthetic reply (spokenTextexcludes reasoning).replies.test.ts: an explicitisReasoningpayload is delivered asMsgType.Notice; raw reasoning-looking text that is not flagged stays suppressed.handler.test.ts: a reasoning block arriving mid-stream with draft streaming on is delivered viadeliverMatrixReplies(as a notice) and is NOT edited into the live answer draft.Known unrelated local failure:
dispatch-from-config.test.ts > "deduplicates inbound messages by MessageSid and origin"hangs to a ~6 min timeout on this Windows/Node-22.17 machine. It does not setsupportsReasoningBlocks, so the changed condition evaluates identically to before (!undefined === true); a boolean read cannot cause the hang, so it is pre-existing/environmental and not affected by this change. CI (Linux Node 24) is the source of truth.If no test was added, why not: N/A — five regression tests were added (three originally + two TTS-exclusion tests).
Risk checklist
Did user-visible behavior change? (
Yes/No) Yes — Matrix now surfaces reasoning asm.noticewhen/reasoning on(the intended, previously-broken behavior), and reasoning is excluded from TTS audio.Did config, environment, or migration behavior change? (
Yes/No) No — no new config/env keys. The opt-in is an internal reply-options capability flag set by the Matrix channel itself, not operator-facing. The TTS exclusion is a no-op for channels that suppress reasoning before delivery.Did security, auth, secrets, network, or tool execution behavior change? (
Yes/No) No.What is the highest-risk area? The Matrix draft-stream bypass in
handler.ts(reasoning payloads skip the live-preview edit path) and the relaxed dispatch suppression guards that letisReasoningreach the TTS paths for opted-in channels.How is that risk mitigated? The two draft-stream guards are guarded by
payload.isReasoning === true, so they only change behavior for explicit reasoning payloads; normal answer delivery and draft streaming are byte-for-byte unchanged (all 39 existing draft-streaming tests pass, plus a new test asserting reasoning is delivered as a notice and the draft is not edited). Reasoning arrives before the answer, so skipping its draft bookkeeping cannot desync the answer draft. The TTS exclusion is guarded byisReasoning === trueinmaybeApplyTtsToReplyPayload(covers final + per-block TTS) and in the block accumulator (covers the post-stream TTS-only synthetic reply); two regression tests assert reasoning receives nomediaUrland is absent from the syntheticspokenText. BecauseisReasoningpayloads never reach TTS on main today (they are suppressed before delivery), the exclusion is a no-op for non-opted-in channels.Current review state
What is the next action? Maintainer review (and, if available, live Matrix/Element confirmation via Mantis).
What is still waiting on author, maintainer, CI, or external proof? CI on Linux Node 24 (rebased onto latest
main8a5cb85c31to clear the pre-existingcheck-lint/check-prod-typesfailures from the stale base); ClawSweeper re-review after the TTS-exclusion fix and the expanded off-guard/raw-suppression proof. Optional live Matrix/Element confirmation — the real-code-path proof above is provided in lieu of a homeserver I do not have access to; a maintainer can dispatch Mantis with:@openclaw-mantis visual task: verify in Matrix/Element that /reasoning on shows a reasoning notice before the final answer and /reasoning off hides reasoning.Which bot or reviewer comments were addressed? ClawSweeper's two P2 review findings are addressed: (1)
dispatch-from-config.ts:3107—isReasoningpayloads no longer enteraccumulatedBlockTtsText/blockCount; (2)dispatch-from-config.ts:3279—isReasoningfinal replies skip TTS via themaybeApplyTtsToReplyPayloadguard while still being delivered as the Matrix notice. The branch was also rebased onto the latestmainso the two failing CI checks (check-lint,check-prod-types) — caused by a stale base carrying the since-fixedisCliRuntimeProviderunused import — now inherit the fix frommain.