Skip to content

fix(llm): preserve structured tool results as text across providers#96895

Closed
arkyu2077 wants to merge 1 commit into
openclaw:mainfrom
arkyu2077:prf-96857-tool-result-text
Closed

fix(llm): preserve structured tool results as text across providers#96895
arkyu2077 wants to merge 1 commit into
openclaw:mainfrom
arkyu2077:prf-96857-tool-result-text

Conversation

@arkyu2077

Copy link
Copy Markdown
Contributor

Summary

  • preserve structured non-image tool result blocks as text instead of collapsing them to (see attached image)
  • apply the fallback across OpenAI Responses, OpenAI-compatible chat completions, and Google/Gemini message conversion
  • add regression coverage for structured session_status-style tool results in all three provider paths

Testing

  • ./node_modules/.bin/vitest run src/llm/providers/openai-responses-shared.test.ts src/llm/providers/google-shared.convert.test.ts src/llm/providers/openai-completions.test.ts

Closes #96857

@openclaw-barnacle openclaw-barnacle Bot added size: S triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. labels Jun 26, 2026
@clawsweeper

clawsweeper Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed June 25, 2026, 10:19 PM ET / 02:19 UTC.

Summary
The PR adds a shared structured tool-result text extractor and uses it in OpenAI Responses, OpenAI-compatible completions, and Google/Gemini provider converters with regression tests.

PR surface: Source +53, Tests +91. Total +144 across 7 files.

Reproducibility: yes. from source: current provider and transport converters filter only text blocks and fall back to (see attached image) for structured non-text, non-image tool-result blocks. I did not run a live provider turn in this read-only review.

Review metrics: 2 noteworthy metrics.

  • Typecheck blockers: 3 changed source call sites and 2 changed test files affected. The PR cannot land while changed code fails the repository's TypeScript lanes.
  • Responses replay coverage: 1 provider converter changed, 1 active transport converter unchanged. The unchanged active transport path shares the reported placeholder invariant and can keep the user-visible bug alive.

Stored data model
Persistent data-model change detected: serialized state: src/llm/providers/tool-result-text.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #96857
Summary: This PR is a candidate fix for the canonical structured/text tool-result placeholder issue, but it is incomplete for an active sibling path and not yet merge-ready.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🧂 unranked krab
Result: blocked until real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Make the helper signature, changed call sites, and new tests pass TypeScript checks.
  • [P1] Apply the structured tool-result extraction to src/agents/openai-transport-stream.ts or prove that path cannot receive these blocks.
  • [P1] Add redacted live output, terminal output, logs, or a recording showing a structured tool result reaching the relevant provider path as text.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body lists test commands but no redacted terminal output, logs, live output, recording, or linked artifact showing the fixed behavior in a real setup. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] Merging as-is can close Normal tool text outputs can degrade to “(see attached image)” placeholders in agent context #96857 while the active OpenAI Responses transport path can still deliver structured tool output as (see attached image).
  • [P1] The changed source and tests currently fail TypeScript checks, so the branch is not build-ready even before runtime proof.
  • [P1] The contributor has not provided after-fix real behavior proof, so unit tests alone do not prove the live provider/session path is fixed.

Maintainer options:

  1. Fix transport coverage and type errors before merge (recommended)
    Update the shared extractor and all active Responses replay paths, repair the test fixtures/API calls, and keep the linked issue open until proof shows structured tool results arrive as text.
  2. Land provider-only coverage intentionally
    Maintainers could accept the provider-only improvement, but the linked issue should stay open or be narrowed because the active transport gap would remain.
  3. Pause until runtime proof exists
    If the live failure path is still uncertain, wait for redacted terminal or live-session proof before deciding whether this branch is the right fix.

Next step before merge

  • [P1] Needs contributor or maintainer follow-up to fix the TypeScript blockers, cover the active transport path, and supply real behavior proof before merge review can clear.

Security
Cleared: The diff changes provider message conversion and tests only; it does not add dependencies, workflows, lockfile changes, secret handling, or new code execution paths.

Review findings

  • [P1] Make the extractor type-compatible — src/llm/providers/tool-result-text.ts:33
  • [P1] Use the real completions compat fixture — src/llm/providers/openai-completions.test.ts:1083
  • [P1] Cover the active Responses transport path too — src/llm/providers/openai-responses-shared.ts:373
Review details

Best possible solution:

Make one type-safe extractor cover every active tool-result replay path, including src/agents/openai-transport-stream.ts, then add redacted real behavior proof before merge.

Do we have a high-confidence way to reproduce the issue?

Yes from source: current provider and transport converters filter only text blocks and fall back to (see attached image) for structured non-text, non-image tool-result blocks. I did not run a live provider turn in this read-only review.

Is this the best way to solve the issue?

No. The helper direction is plausible, but this PR is not the best complete fix until it is type-safe, covers the active Responses transport path, and includes real behavior proof.

Full review comments:

  • [P1] Make the extractor type-compatible — src/llm/providers/tool-result-text.ts:33
    extractToolResultText takes Record<string, unknown>[], but the changed converters pass ToolResultMessage.content, which is typed as (TextContent | ImageContent)[]; CI reports TS2352 on all three changed source call sites. Make the helper accept readonly unknown[] and narrow internally, or use a compatible content-block type so the production code typechecks.
    Confidence: 0.95
  • [P1] Use the real completions compat fixture — src/llm/providers/openai-completions.test.ts:1083
    The new test calls convertMessages(..., false), but the current signature expects a ResolvedOpenAICompletionsCompat object, so check-test-types fails with TS2345. Use the existing compat fixture/helper rather than a boolean placeholder.
    Confidence: 0.96
  • [P1] Cover the active Responses transport path too — src/llm/providers/openai-responses-shared.ts:373
    This updates the provider-side Responses converter, but OpenAI Responses can also run through src/agents/openai-transport-stream.ts; that converter still filters only type: "text" blocks and falls back to (see attached image). The linked agent-context failure can therefore still happen on the native/managed transport path unless the helper is shared there or that path is proven unreachable for these blocks.
    Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.93

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 6830aa39eaa1.

Label changes

Label justifications:

  • P1: The linked bug can make a live agent unable to inspect ordinary tool output, breaking an active agent workflow.
  • merge-risk: 🚨 message-delivery: The diff fixes some provider replay paths while an active OpenAI Responses transport path can still deliver structured tool results as (see attached image).
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body lists test commands but no redacted terminal output, logs, live output, recording, or linked artifact showing the fixed behavior in a real setup. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +53, Tests +91. Total +144 across 7 files.

View PR surface stats
Area Files Added Removed Net
Source 4 63 10 +53
Tests 3 92 1 +91
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 7 155 11 +144

Acceptance criteria:

  • [P1] node scripts/run-vitest.mjs src/llm/providers/openai-responses-shared.test.ts src/llm/providers/google-shared.convert.test.ts src/llm/providers/openai-completions.test.ts.
  • [P1] node scripts/run-vitest.mjs <focused src/agents/openai-transport-stream test file>.
  • [P1] Relevant TypeScript/check lane that covers the changed provider and test files.

What I checked:

  • Live PR context: The PR closes Normal tool text outputs can degrade to “(see attached image)” placeholders in agent context #96857 and its body lists only a Vitest command, with no live output, terminal log, recording, or other after-fix real behavior proof. (63ecf452addc)
  • Current provider failure mode: Current main builds OpenAI Responses tool output text by filtering only type: "text" blocks and falls back to (see attached image) when that text is empty, so a structured non-image block reproduces the reported failure by source inspection. (src/llm/providers/openai-responses-shared.ts:374, 6830aa39eaa1)
  • Active sibling transport: OpenAI Responses models can route through createOpenAIResponsesTransportStreamFn, so the agent transport converter is an active sibling implementation for the same replay invariant. (src/agents/provider-transport-stream.ts:85, 6830aa39eaa1)
  • Missing sibling fix: The active OpenAI Responses transport still filters only text blocks and falls back to (see attached image), while this PR only changes the provider-side converter. (src/agents/openai-transport-stream.ts:1273, 6830aa39eaa1)
  • Typecheck failure evidence: CI logs show diff-caused TypeScript failures: the new helper signature rejects the typed ToolResultMessage.content call sites, the new completions test passes false where ResolvedOpenAICompletionsCompat is required, and the structured test fixture is not assignable to current Context. (src/llm/providers/tool-result-text.ts:33, 63ecf452addc)
  • OpenAI dependency contract: OpenAI Node v6.39.1 allows Responses function-call output to be a string or a list of output content, so the intended text-output shape is compatible with the upstream SDK contract.

Likely related people:

  • openclaw-clownfish[bot]: git blame attributes the current text-only tool-result conversions in both the provider and active transport paths to the same grafted commit. (role: introduced current implementation; confidence: medium; commits: ace22feb3fe2; files: src/llm/providers/openai-responses-shared.ts, src/agents/openai-transport-stream.ts)
  • Ayaan Zaidi: git show records Ayaan Zaidi as the committer for the grafted commit that introduced the current provider and transport implementations in this checkout. (role: committer; confidence: medium; commits: ace22feb3fe2; files: src/llm/providers/openai-responses-shared.ts, src/agents/openai-transport-stream.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@harjothkhara

Copy link
Copy Markdown
Contributor

I think this PR is now superseded by #97742 / current main rather than something maintainers need to repair here.

I rechecked the current main code against this PR's old diff. #96895 only updates the provider-side converters in src/llm/providers/openai-responses-shared.ts, src/llm/providers/openai-completions.ts, and src/llm/providers/google-shared.ts, plus a simpler tool-result-text helper. Current main already has the landed #97742 shape: src/llm/providers/tool-result-text.ts serializes structured non-media blocks with redaction/binary omission and media-aware placeholders, and the Responses, Chat Completions, Google, and previously-missing active sibling src/agents/openai-transport-stream.ts paths all call that shared extractor.

That means the best fix path appears to be the already-merged #97742, not reviving #96895. If maintainers want to keep #96857 open for the remaining "text-only after long session/truncation" reports, that should be tracked separately from this structured-provider converter PR.

@openclaw-barnacle

Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Jul 16, 2026
@openclaw-barnacle

Copy link
Copy Markdown

Closing due to inactivity.
If you believe this PR should be revived, post in #clawtributors on Discord to talk to a maintainer.
That channel is the escape hatch for high-quality PRs that get auto-closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. P1 High-priority user-facing bug, regression, or broken workflow. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: S stale Marked as stale due to inactivity status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Normal tool text outputs can degrade to “(see attached image)” placeholders in agent context

2 participants