fix: avoid image placeholder for empty text tool results by scribe-dandelion-cult · Pull Request #97423 · openclaw/openclaw

scribe-dandelion-cult · 2026-06-28T06:05:11Z

What Problem This Solves

Fixes a replay/projection bug where an empty non-image tool result can be replayed to OpenAI-compatible providers as the literal image placeholder:

(see attached image)

That placeholder should only represent actual image-bearing tool output. Empty non-image tool output should not make the model believe it received an image.

This matches symptoms reported in #96857, where normal text-producing tool results can degrade into (see attached image) placeholders in agent context. Related/duplicate history includes #97321, #96861, and #75674.

A downstream live reproduction using github-copilot/gpt-5.5 found a strong breadcrumb around update_plan: it returns content: [] with plan data in details, and it is replay-safe. Once that empty non-image result enters replay history, later tool output can appear poisoned in the session. This PR fixes the deterministic bad replay branch; it does not claim update_plan is the only trigger for every session-local poisoning case.

Evidence

The bug is a deterministic conversion branch:

empty non-image tool result before: replayed as (see attached image);
empty non-image tool result after: replays as (no output);
image-bearing tool result still preserves image placeholder behavior when a text fallback is needed.

The regression tests model update_plan-style empty non-image output and assert the corrected replay directly across OpenAI-compatible paths:

src/llm/providers/openai-responses-shared.test.ts
src/agents/openai-transport-stream.test.ts
src/llm/providers/openai-completions.test.ts

Validation command run on the fork branch:

node scripts/run-vitest.mjs \
  src/llm/providers/openai-responses-shared.test.ts \
  src/agents/openai-transport-stream.test.ts \
  src/llm/providers/openai-completions.test.ts

Result: 3 files / 340 tests passed.

Behavioral proof: N/A — this is a deterministic replay/projection logic error captured by unit tests. No live model call is required to prove the branch condition.

Summary

Replay empty non-image OpenAI-compatible tool results as (no output).
Keep (see attached image) reserved for actual image-bearing tool results.
Cover OpenAI Responses shared, OpenAI transport replay, and OpenAI completions paths.

Non-goals

Does not change update_plan semantics or require it to emit user-visible text.
Does not remove image support or explicit image-tool behavior.

Live operational proof (separate from continuation)

This proof is not part of the continuation PR and does not change the 2723dbee continuation proof basis. It is scoped only to this placeholder/replay fix.

Live docs proof corpus:

karmaterminal/karmaterminal-openclaw-docs@852173d710cee49c16cc52479d9e0b16cdba077b
PR-1120/PROOFS/empty-tool-placeholder-live/INDEX.json
PR-1120/PROOFS/empty-tool-placeholder-live/manifest.json
PR-1120/PROOFS/empty-tool-placeholder-live/cael-live-mitigation.md

Operational sequence:

Cael reported the live poison shape before the fix: ordinary text tools (echo, read, write, gh api, web_fetch) rendered only as (see attached image), so he refused blind mutation.
We deployed frond-build/20260628/2723dbee-empty-tool-placeholder at 64324505fcb8be367abe91234bbb811b005466b4, whose parent is exact runtime 2723dbee783c113cae70e4fb63a4cff9f55402e3 plus only this placeholder fix.
After deploy, Cael confirmed a plain echo rendered as text: figs love you and this is shit and youre so brave.
Cael then completed a direct docs proof commit requiring byte-readable tool output: R-CW-DELEGATE-CHILD-LIVE -> commit d2b0158471c85dddf0eeb1be302e008e85017fbc.

Interpretation: this is live mitigation evidence for the same placeholder poisoning class. The deterministic proof remains the unit coverage in this PR; the live proof shows the frond-build containing this fix restored Cael’s ability to read text tool output and complete a docs commit.

Co-authored-by: Copilot <[email protected]>

clawsweeper · 2026-06-28T06:07:30Z

Codex review: needs maintainer review before merge. Reviewed June 28, 2026, 4:28 AM ET / 08:28 UTC.

Summary
The branch changes OpenAI Responses, OpenAI-compatible chat completions, and managed OpenAI transport replay so empty non-image tool results emit (no output) while image-bearing results keep image placeholder behavior.

PR surface: Source +27, Tests +325. Total +352 across 6 files.

Reproducibility: yes. Current main source-reproduces the scoped bug: empty non-image tool results are converted to (see attached image) in the affected OpenAI-compatible conversion paths.

Review metrics: none identified.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #96857
Summary: This PR is a focused candidate fix for one deterministic branch of the canonical open tool-result placeholder issue; broader session-state and failover symptoms remain tracked by the canonical issue.

Members:

canonical: Normal tool text outputs can degrade to “(see attached image)” placeholders in agent context #96857 - Open issue with the same ordinary text-output placeholder symptom and broader Gateway truncation/provider-failover evidence.
duplicate: Tool results can degrade into literal '(see attached image)' placeholder for text output #97321 - Closed as a duplicate of the canonical placeholder-degradation report while preserving session-context details.
duplicate: Normal tool text outputs can degrade to “(see attached image)” placeholders in agent context #96861 - Earlier same-symptom report was closed by its reporter as a duplicate of the canonical issue.
adjacent_distinct: MCP tool result with type: "resource" is converted to "(see attached image)" placeholder #75674 - Closed MCP EmbeddedResource-specific placeholder bug; useful context, but already fixed separately and narrower than the current ordinary text-output symptom.
partial_overlap: fix(llm): preserve structured tool results as text across providers #96895 - Open overlapping PR targets structured tool-result placeholder behavior but is broader and review history marks it unready, so it does not supersede this PR.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster ✨ media proof bonus
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

[P2] The open canonical report also describes prompt-history truncation, provider failover, and persistent session-state poisoning symptoms that this focused empty-result fallback change does not prove resolved.
[P1] An older overlapping open PR, fix(llm): preserve structured tool results as text across providers #96895, still exists but is broader and not a safe superseder because its own review history marks it unready.

Maintainer options:

Decide the mitigation before merge
Merge this scoped empty non-image replay fix after maintainer review, and keep or narrow the canonical issue for broader session-poisoning symptoms that remain.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

No automated repair is needed; the remaining action is maintainer merge and scope judgment against the broader canonical issue.

Security
Cleared: The diff changes provider replay logic and tests only; it does not touch dependencies, lockfiles, workflows, permissions, install scripts, secrets, or other supply-chain surfaces.

Review details

Best possible solution:

Merge this scoped empty non-image replay fix after maintainer review, and keep or narrow the canonical issue for broader session-poisoning symptoms that remain.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main source-reproduces the scoped bug: empty non-image tool results are converted to (see attached image) in the affected OpenAI-compatible conversion paths.

Is this the best way to solve the issue?

Yes for the scoped bug. Distinguishing empty non-image output from image-bearing output at each OpenAI-compatible replay boundary is the narrow maintainable fix, but it is not complete closure proof for #96857.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 9c95abd49d45.

Label changes

Label justifications:

P2: The PR fixes a normal-priority agent/provider replay correctness bug with limited code blast radius and no security, crash-loop, config, or migration impact.
rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (linked_artifact): Linked artifact proof is sufficient: the PR body points to inspected GitHub proof files showing a deployed build with this fix, readable text output after the fix, and a proof commit completed from that lane.
proof: sufficient: Contributor real behavior proof is sufficient. Linked artifact proof is sufficient: the PR body points to inspected GitHub proof files showing a deployed build with this fix, readable text output after the fix, and a proof commit completed from that lane.

Evidence reviewed

PR surface:

Source +27, Tests +325. Total +352 across 6 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	3	36	9	+27
Tests	3	325	0	+325
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	6	361	9	+352

What I checked:

Repository policy read: Full root AGENTS.md and scoped src/agents/AGENTS.md were read; their provider/runtime and sibling-surface review guidance applies because the PR touches OpenAI provider and agent transport replay. (AGENTS.md:1, 9c95abd49d45)
Current main Responses fallback: Current main converts tool results with no text into function_call_output output (see attached image) even when the result contains no image blocks. (src/llm/providers/openai-responses-shared.ts:406, 9c95abd49d45)
Current main completions fallback: Current main sends (see attached image) for chat-completions tool messages whenever extracted text is empty, independent of whether the tool result has images. (src/llm/providers/openai-completions.ts:1142, 9c95abd49d45)
Current main managed transport fallback: The active managed OpenAI Responses transport has the same empty-result fallback to (see attached image) for non-image tool output. (src/agents/openai-transport-stream.ts:1299, 9c95abd49d45)
OpenClaw update_plan source: OpenClaw's built-in update_plan tool returns content: [] with structured details, which gives a concrete empty non-image tool-result shape for the PR's regression tests. (src/agents/tools/update-plan-tool.ts:94, 9c95abd49d45)
PR Responses fix: At PR head, Responses conversion now distinguishes image-bearing results from empty non-image results and falls back to (no output) for the latter. (src/llm/providers/openai-responses-shared.ts:415, c556ded7269a)

Likely related people:

vincentkoc: Commit 4798e12 added src/agents/openai-transport-stream.ts, and recent path history shows OpenAI transport/provider conversion work on the same replay boundary. (role: introduced transport seam and recent provider contributor; confidence: high; commits: 4798e125f40c, aa69b12d0086; files: src/agents/openai-transport-stream.ts, src/llm/providers/openai-responses-shared.ts, src/llm/providers/openai-completions.ts)
steipete: Recent commits touched OpenAI transport, OpenAI tool-schema, and adjacent truncated tool-call handling that overlap this PR's provider payload and agent runtime surface. (role: recent adjacent contributor; confidence: medium; commits: 56259606d15f, 439a9e97fd61; files: src/agents/openai-transport-stream.ts, src/llm/providers/openai-responses-shared.ts, src/llm/providers/openai-completions.ts)
amersheeny: Commit 9fbc8a7 changed both shared Responses conversion and managed OpenAI transport replay behavior, which share the invariant fixed here. (role: recent OpenAI Responses contributor; confidence: medium; commits: 9fbc8a74efa7; files: src/agents/openai-transport-stream.ts, src/llm/providers/openai-responses-shared.ts)
849261680: Commit b1e4b6b fixed a prior MCP resource-to-placeholder variant by normalizing non-text/image tool-result blocks before provider conversion. (role: adjacent prior placeholder-fix contributor; confidence: medium; commits: b1e4b6b65e2c; files: src/agents/agent-bundle-mcp-materialize.ts, src/agents/agent-bundle-mcp-tools.materialize.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

…w#97423) Co-authored-by: scribe-dandelion-cult <[email protected]> Co-authored-by: Copilot <[email protected]>

fix: use no-output placeholder for empty OpenAI tool results

c556ded

Co-authored-by: Copilot <[email protected]>

scribe-dandelion-cult mentioned this pull request Jun 28, 2026

Tool results can degrade into literal '(see attached image)' placeholder for text output karmaterminal/openclaw#1107

Closed

openclaw-barnacle Bot added agents Agent runtime and tooling size: M labels Jun 28, 2026

clawsweeper Bot mentioned this pull request Jun 28, 2026

fix(llm): preserve structured tool result text across providers #97450

Closed

vincentkoc merged commit dd4e2ab into openclaw:main Jun 28, 2026
163 of 176 checks passed

github-actions Bot mentioned this pull request Jun 28, 2026

📡 Upstream Digest — 2026-06-28 19:05 UTC curtismercier/openclaw-mods#1168

Open

This was referenced Jun 28, 2026

fix: avoid image placeholder for empty text tool results karmaterminal/openclaw#1120

Closed

fix: avoid image placeholder for empty text tool results karmaterminal/openclaw#1119

Closed

clawsweeper Bot mentioned this pull request Jun 29, 2026

fix(llm): preserve structured tool result text across providers #97742

Merged

QiuYuang pushed a commit to QiuYuang/openclaw that referenced this pull request Jul 1, 2026

fix: use no-output placeholder for empty OpenAI tool results (opencla…

580ec1e

…w#97423) Co-authored-by: scribe-dandelion-cult <[email protected]> Co-authored-by: Copilot <[email protected]>

aspalagin mentioned this pull request Jul 1, 2026

Normal tool text outputs can degrade to “(see attached image)” placeholders in agent context #96857

Open

This was referenced Jul 3, 2026

fix(llm): tool results vanish for OpenAI-compatible providers (DeepSeek) #99338

Closed

fix(tool-result): preserve truncation notice instead of empty string #99354

Closed

This was referenced Jul 6, 2026

fix: replace image blocks comprehensively in non-vision model downgrade #100281

Closed

[Bug]: All tool outputs display "(see attached image)" after uploading image to a non-multimodal model #99881

Closed

clawsweeper Bot mentioned this pull request Jul 11, 2026

[Bug]: Gateway 群级渲染状态污染 —— 群899721780所有工具输出强制转图片 #104497

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix: avoid image placeholder for empty text tool results#97423

fix: avoid image placeholder for empty text tool results#97423
vincentkoc merged 1 commit into
openclaw:mainfrom
karmaterminal:frond-scribe/20260627/fix-1107-empty-tool-result-placeholder

scribe-dandelion-cult commented Jun 28, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

scribe-dandelion-cult commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Evidence

Summary

Non-goals

Live operational proof (separate from continuation)

Uh oh!

clawsweeper Bot commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

scribe-dandelion-cult commented Jun 28, 2026 •

edited

Loading

clawsweeper Bot commented Jun 28, 2026 •

edited

Loading