fix(agents): preserve structured tool result visible text by snowzlmbot · Pull Request #97268 · openclaw/openclaw

snowzlmbot · 2026-06-27T15:57:35Z

Summary

Preserve structured non-image tool-result blocks as visible text in the subscription/display path.
Keep explicit text blocks unchanged and continue suppressing image-only blocks from text output.
Add guardrails for structured fallback serialization: circular protection, inline data-URI value redaction, and output length capping.

What Problem This Solves

The visible tool-result extraction path could drop structured non-image content because it only collected explicit text blocks. When a tool returned JSON/resource blocks without a text block, the visible transcript could fall back to an attachment-style placeholder instead of useful text.

Why This Change Was Made

Structured non-image tool results can carry the only human-readable result payload. The fallback is intentionally narrow: it serializes non-text/non-image content blocks only after normal text extraction, so existing text-first behavior remains stable and image payloads do not become visible text.

Review Follow-Up

Addressed the ClawSweeper review blockers on current head f907f28856e9e745eab43da2e0b9c1990baa59dc:

[P2] Text-first fallback: extractToolResultText now returns existing text blocks before structured serialization.
[P2] Regression coverage: added mixed text + structured JSON coverage proving only the existing text output is emitted.
[P1] Runtime proof: captured installed OpenClaw visible-output path proof on test server using the global OpenClaw runtime under /usr/lib/node_modules/openclaw/dist.
[P1] Sibling coordination: fix(agents): serialize structured non-image tool-result blocks in extractToolResultText (fixes #97267) #97276 targets the same helper/root cause; this PR is the text-first fallback variant and maintainers should land only one of the two approaches.

Implementation

Add stringifyStructuredToolResultContent in src/agents/embedded-agent-subscribe.tools.ts.
Skip text and image blocks in the structured fallback.
Return existing text output before attempting structured fallback serialization.
Serialize structured non-image blocks with circular guard and inline data-URI value redaction.
Cap combined fallback output with the existing TOOL_RESULT_MAX_CHARS behavior.

Evidence

Validated on Linux at PR head f907f28856e9e745eab43da2e0b9c1990baa59dc.

Focused test command on test server:

pnpm test src/agents/embedded-agent-subscribe.tools.test.ts

Focused test result:

Test Files  1 passed (1)
Tests       28 passed (28)
[test] passed 1 Vitest shard in 15.42s

Installed runtime helper proof on test server after hotfixing and restarting OpenClaw:

structured_fallback=ok
text_first_mixed=ok
image_only_suppressed=ok
data_uri_redacted=ok

Installed CLI visible-output path proof from /usr/lib/node_modules/openclaw/dist/reply-usage-state-_oArqMDo.js:

--- delivered[0] ---
🧩 Structured Tool
```txt
{"type":"json","data":{"status":"ok","value":42}}
```
--- delivered[1] ---
🧩 Mixed Tool
```txt
hello
```
visible_structured_only=ok
visible_mixed_text_first=ok

Installed session proof also verified a real embedded OpenClaw agent run after restart:

toolSummary.calls=1
toolSummary.tools=["session_status"]
finalAssistantVisibleText=PR97268_PROOF_DONE
runner=embedded

Additional current-install verification (Linux local OpenClaw install, 2026-06-28):

structured={"type":"json","data":{"status":"ok","value":42}}
mixed=hello
image=undefined
dataUri={"type":"json","uri":"[inline data URI: 59 chars]"}
installed_probe=fixed

Scope And Limitations

This PR only changes visible tool-result text extraction and related helper tests.
It does not change messaging delivery, media extraction, tool execution, approval policy, or stored runtime data.
No storage/schema migration is needed; the changed files are source/test code, not persisted data model definitions.
The broader investigation remains tracked in bug(agents): visible tool-result text can collapse to attachment placeholder #97267, so this PR intentionally uses Fixes #97267 rather than closing it.

Fixes #97267
Supersedes #97183

snowzlmbot · 2026-06-27T15:57:36Z

@clawsweeper re-review

clawsweeper · 2026-06-27T15:57:39Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

clawsweeper · 2026-06-27T15:58:49Z

Codex review: needs maintainer review before merge. Reviewed June 28, 2026, 5:32 AM ET / 09:32 UTC.

Summary
The branch extends extractToolResultText to keep text-first output, serialize structured non-image results as capped/redacted visible text, and adds helper plus CLI verbose-output coverage.

PR surface: Source +130, Tests +173. Total +303 across 3 files.

Reproducibility: yes. at source level: current main returns undefined for structured-only non-image blocks because it only collects explicit text blocks, and the visible-output handler only emits output when this helper returns text. I did not run tests or live repro commands because this review is read-only.

Review metrics: 1 noteworthy metric.

Open Helper Fix Candidates: 2 open PRs. This PR and fix(agents): serialize structured non-image tool-result blocks in extractToolResultText (fixes #97267) #97276 change the same helper for the same canonical issue, so branch selection matters before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #97267
Summary: This PR is a candidate fix for the canonical visible structured tool-result text issue; a same-root helper PR is also open, while broader provider-context placeholder work remains adjacent.

Members:

canonical: bug(agents): visible tool-result text can collapse to attachment placeholder #97267 - The issue describes the same extractToolResultText structured-only visible-output failure and remains open with this PR as a closing candidate.
fixed_by_candidate: fix(agents): serialize structured non-image tool-result blocks in extractToolResultText (fixes #97267) #97276 - The sibling PR also changes extractToolResultText for the same canonical issue, but it is a separate open implementation path.
superseded: fix(llm): preserve structured tool results as text across providers #97183 - This PR body says it supersedes the earlier broader unmerged branch, which is closed.
partial_overlap: Normal tool text outputs can degrade to “(see attached image)” placeholders in agent context #96857 - That issue reports broader agent-context placeholder behavior across ordinary tool outputs, while this PR targets the visible subscription helper path.
partial_overlap: fix(llm): preserve structured tool results as text across providers #96895 - That open PR targets provider conversion paths for broader structured tool-result replay rather than this visible subscription helper alone.
1 more in the report.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

[P1] Two open PRs target the same helper/root cause; merging both could create duplicate formatting logic or drift in visible tool-result delivery.
[P1] This PR intentionally changes previously missing visible output into compact JSON for structured non-text blocks, so maintainers should accept that presentation before landing.

Maintainer options:

Use This Hardened Helper Branch (recommended)
Treat this PR as the canonical helper fix, accept compact JSON as the visible fallback format, and close or supersede fix(agents): serialize structured non-image tool-result blocks in extractToolResultText (fixes #97267) #97276 rather than merging both.
Choose The Sibling Instead
If maintainers prefer the smaller sibling shape, transfer any wanted hardening or proof from this PR and close this branch after that decision.
Wait For Provider-Wide Work
If maintainers want the broader provider-context placeholder fix to lead, pause this PR until fix(llm): preserve structured tool result text across providers #97450 is reviewed and its boundary is settled.

Next step before merge

[P2] Maintainers should choose the canonical helper branch and decide whether to accept the compact JSON fallback; there is no narrow automated repair needed on this head.

Security
Cleared: No concrete security or supply-chain concern was found; the diff changes agent helper/test code and adds redaction/capping without touching dependencies, workflows, permissions, lockfiles, install scripts, or credential storage.

Review details

Best possible solution:

Land exactly one helper-boundary fallback for the canonical visible-output bug, preserve text-first/image-skip/redaction/capping behavior, and close or supersede the same-root helper sibling.

Do we have a high-confidence way to reproduce the issue?

Yes at source level: current main returns undefined for structured-only non-image blocks because it only collects explicit text blocks, and the visible-output handler only emits output when this helper returns text. I did not run tests or live repro commands because this review is read-only.

Is this the best way to solve the issue?

Yes for the scoped visible-output bug: the helper boundary already owns display text extraction, and the PR preserves text-first and image-skip behavior while adding redaction and caps. The broader provider-context problem remains adjacent in #97450.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against d17b970bb50d.

Label changes

Label changes:

add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
remove rating: 🐚 platinum hermit: Current PR rating is rating: 🦞 diamond lobster, so this older rating label is no longer current.

Label justifications:

P2: This is a normal-priority agent visible-output bug fix with a narrow helper blast radius and no evidence of a current core outage.
merge-risk: 🚨 message-delivery: The PR changes delivered visible tool-result text, and duplicate same-root helper PRs could alter or conflict on message/output presentation if both land.
rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes after-fix terminal proof from an installed OpenClaw runtime showing structured-only visible text, mixed text-first output, image suppression, data-URI redaction, and a real embedded session run.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix terminal proof from an installed OpenClaw runtime showing structured-only visible text, mixed text-first output, image suppression, data-URI redaction, and a real embedded session run.

Evidence reviewed

PR surface:

Source +130, Tests +173. Total +303 across 3 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	136	6	+130
Tests	2	173	0	+173
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	3	309	6	+303

What I checked:

Repository policy read: Root AGENTS.md was read fully and its deep-review, sibling-surface, and merge-risk guidance applies to this agents-path PR. (AGENTS.md:1, d17b970bb50d)
Scoped agents policy read: The scoped agents guide was read fully; it supports focused agent tests that preserve production behavior without broad runtime imports. (src/agents/AGENTS.md:1, d17b970bb50d)
Current main drops structured-only visible text: Current main extractToolResultText only collects explicit text content blocks and returns undefined when none exist, which matches the canonical visible-output failure mode. (src/agents/embedded-agent-subscribe.tools.ts:270, d17b970bb50d)
Visible output caller depends on this helper: The tool-result handler emits visible output only when extractToolResultText(sanitizedResult) returns text, so this helper is the relevant display boundary. (src/agents/embedded-agent-subscribe.handlers.tools.ts:739, d17b970bb50d)
PR implementation at current head: At PR head, the helper resolves top-level arrays/typed blocks, returns existing text before fallback, skips image/audio text extraction, serializes structured fallback output, redacts inline data URIs/opaque fields/headers, and caps output. (src/agents/embedded-agent-subscribe.tools.ts:283, db76698b0d48)
PR regression coverage: The added tests cover structured fallback, top-level CLI result arrays/objects, text-first behavior, truncation, data-URI redaction, binary suppression, opaque fields, and structured header redaction. (src/agents/embedded-agent-subscribe.tools.test.ts:399, db76698b0d48)

Likely related people:

vincentkoc: GitHub path history shows repeated recent changes in the embedded-agent subscribe helper and handler surfaces around visible delivery and tool-result handling. (role: recent area contributor; confidence: high; commits: 0771ac856379, d75613e794da, fec6a0407f44; files: src/agents/embedded-agent-subscribe.tools.ts, src/agents/embedded-agent-subscribe.handlers.tools.ts)
steipete: History shows the shared collectTextContentBlocks helper was introduced by steipete, and the current PR head includes several hardening commits authored by Peter Steinberger. (role: shared helper introducer and current branch hardening contributor; confidence: high; commits: 85ebdf88b0a9, 0314819f918a, db76698b0d48; files: src/agents/content-blocks.ts, src/agents/embedded-agent-subscribe.tools.ts, src/agents/embedded-agent-subscribe.handlers.tools.ts)
yetval: Recent path history shows a focused bug fix in the same helper file around preserving pending thread evidence during partial send result reconciliation. (role: recent adjacent bugfix contributor; confidence: medium; commits: 7fc124dcf1dc; files: src/agents/embedded-agent-subscribe.tools.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

snowzlmbot · 2026-06-27T16:31:19Z

Follow-up evidence / scope clarification:

Local source branch verification passed for the focused patch and test.
Runtime hot-patch attempts against the already-running local install were rolled back after direct probe validation still showed placeholder behavior; this PR therefore intentionally stays source-level and narrow.
The remaining runtime verification work should happen in a clean rebuilt/reinstalled local environment after review, not by further editing installed dist chunks.

snowzlmbot · 2026-06-27T16:48:59Z

@clawsweeper re-review

snowzlmbot · 2026-06-28T01:11:08Z

@clawsweeper re-review

clawsweeper · 2026-06-28T01:11:11Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

snowzlmbot · 2026-06-28T02:16:36Z

@clawsweeper re-review

clawsweeper · 2026-06-28T02:16:39Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

steipete · 2026-06-28T09:38:28Z

Land-ready verification for prepared head db76698b0d4865bc98cb0b7062c63af4a50304c9:

Improved the contributor implementation at the shared visible-output boundary: actual top-level Claude/Gemini CLI arrays and typed Anthropic result objects now render, canonical tool-error statuses remain correct, and both text and JSON paths are capped at 8,000 characters.
Hardened structured output before serialization: scalar/array-valued secrets and auth headers are redacted; image, audio, document/base64, MCP blob, data-URI, encrypted_content, and encrypted_stdout payloads are skipped or replaced while useful metadata remains visible.
Focused proof: node scripts/run-vitest.mjs src/agents/embedded-agent-subscribe.tools.test.ts src/auto-reply/reply/agent-runner-cli-dispatch.test.ts src/agents/cli-output.test.ts src/agents/embedded-agent-subscribe.handlers.tools.test.ts — 159 tests passed across three Vitest shards.
Direct production-helper probe: typed code-execution stdout remained visible; opaque search ciphertext was replaced before the cap; array-valued set-cookie was redacted; useful search fields remained visible.
Formatting: oxfmt --check on all touched files and git diff --check passed.
Upstream contract: inspected installed @anthropic-ai/[email protected] source/types for code-execution, encrypted code-execution, and web-search result shapes.
Fresh autoreview: clean after accepting and resolving nine findings; final confidence 0.91.
Exact-head hosted CI: run 28317871184 passed, including all Node shards, lint, production/test types, guards, QA smoke, security-fast, CodeQL, OpenGrep, and dependency checks.
Existing installed-runtime proof covers structured visible output, text-first mixed output, image/data-URI suppression, and a real embedded-agent turn.

Proof gap: no additional Crabbox/Testbox scenario could run because this machine has no configured provider/coordinator credentials. Exact-head hosted CI and the existing installed live proof cover the land gate.

steipete · 2026-06-28T09:38:48Z

Merged via squash.

Prepared head SHA: db76698b0d4865bc98cb0b7062c63af4a50304c9
Landed commit: 8d168c836af38332a75529ca8c1af2cf28494427

…7268) * fix(agents): preserve structured tool result visible text * fix(agents): address structured tool result review blockers * fix(agents): cover real structured tool result shapes * fix(agents): preserve canonical tool error statuses * fix(agents): harden structured result rendering * fix(agents): preserve typed structured results * fix(agents): bound provider result media --------- Co-authored-by: snowzlmbot <[email protected]> Co-authored-by: Peter Steinberger <[email protected]>

snowzlmbot mentioned this pull request Jun 27, 2026

fix(llm): preserve structured tool results as text across providers #97183

Closed

openclaw-barnacle Bot added agents Agent runtime and tooling size: M labels Jun 27, 2026

clawsweeper Bot mentioned this pull request Jun 27, 2026

bug(agents): visible tool-result text can collapse to attachment placeholder #97267

Closed

snowzlmbot marked this pull request as ready for review June 27, 2026 16:20

clawsweeper Bot mentioned this pull request Jun 27, 2026

fix(agents): serialize structured non-image tool-result blocks in extractToolResultText (fixes #97267) #97276

Closed

openclaw-barnacle Bot added size: S and removed size: M labels Jun 28, 2026

clawsweeper Bot added P2 Normal backlog priority with limited blast radius. and removed P1 High-priority user-facing bug, regression, or broken workflow. labels Jun 28, 2026

snowzlmbot force-pushed the fix/local-tool-result-display-placeholder branch 2 times, most recently from 9b28834 to f907f28 Compare June 28, 2026 02:09

clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. labels Jun 28, 2026

clawsweeper Bot added status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 28, 2026

github-actions Bot mentioned this pull request Jun 28, 2026

🦞 OpenClaw 生态日报 2026-06-28 96loveslife/big_model_radar#69

Open

steipete self-assigned this Jun 28, 2026

clawsweeper Bot mentioned this pull request Jun 28, 2026

fix(llm): preserve structured tool result text across providers #97450

Closed

snowzlmbot and others added 7 commits June 28, 2026 02:27

fix(agents): preserve structured tool result visible text

e7f1df8

fix(agents): address structured tool result review blockers

fa62444

fix(agents): cover real structured tool result shapes

1694f07

fix(agents): preserve canonical tool error statuses

6fe8b9f

fix(agents): harden structured result rendering

b0ce8e2

fix(agents): preserve typed structured results

923b953

fix(agents): bound provider result media

db76698

steipete force-pushed the fix/local-tool-result-display-placeholder branch from f907f28 to db76698 Compare June 28, 2026 09:27

openclaw-barnacle Bot added size: M and removed size: S labels Jun 28, 2026

clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jun 28, 2026

steipete merged commit 8d168c8 into openclaw:main Jun 28, 2026
112 of 116 checks passed

github-actions Bot mentioned this pull request Jun 28, 2026

📡 Upstream Digest — 2026-06-28 10:21 UTC curtismercier/openclaw-mods#1165

Open

clawsweeper Bot mentioned this pull request Jun 29, 2026

fix(llm): preserve structured tool result text across providers #97742

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(agents): preserve structured tool result visible text#97268

fix(agents): preserve structured tool result visible text#97268
steipete merged 7 commits into
openclaw:mainfrom
snowzlmbot:fix/local-tool-result-display-placeholder

snowzlmbot commented Jun 27, 2026 •

edited

Loading

Uh oh!

snowzlmbot commented Jun 27, 2026

Uh oh!

clawsweeper Bot commented Jun 27, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 27, 2026 •

edited

Loading

Uh oh!

snowzlmbot commented Jun 27, 2026

Uh oh!

snowzlmbot commented Jun 27, 2026

Uh oh!

snowzlmbot commented Jun 28, 2026

Uh oh!

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading

Uh oh!

snowzlmbot commented Jun 28, 2026

Uh oh!

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading

Uh oh!

steipete commented Jun 28, 2026

Uh oh!

Uh oh!

steipete commented Jun 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

snowzlmbot commented Jun 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What Problem This Solves

Why This Change Was Made

Review Follow-Up

Implementation

Evidence

Scope And Limitations

Uh oh!

snowzlmbot commented Jun 27, 2026

Uh oh!

clawsweeper Bot commented Jun 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clawsweeper Bot commented Jun 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

snowzlmbot commented Jun 27, 2026

Uh oh!

snowzlmbot commented Jun 27, 2026

Uh oh!

snowzlmbot commented Jun 28, 2026

Uh oh!

clawsweeper Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

snowzlmbot commented Jun 28, 2026

Uh oh!

clawsweeper Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steipete commented Jun 28, 2026

Uh oh!

Uh oh!

steipete commented Jun 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

snowzlmbot commented Jun 27, 2026 •

edited

Loading

clawsweeper Bot commented Jun 27, 2026 •

edited

Loading

clawsweeper Bot commented Jun 27, 2026 •

edited

Loading

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading