fix(llm): preserve structured tool result text across providers#97742
Conversation
|
Codex review: needs maintainer review before merge. Reviewed July 1, 2026, 2:21 AM ET / 06:21 UTC. Summary PR surface: Source +320, Tests +1027, Docs 0, Other 0. Total +1347 across 29 files. Reproducibility: yes. Current main filters several tool-result replay paths to explicit text blocks before falling back to image/no-output placeholders, and the linked issue includes live Discord/Telegram evidence of text outputs degrading to placeholders. Review metrics: 1 noteworthy metric.
Stored data model Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Risk before merge
Maintainer options:
Next step before merge
Security Review detailsBest possible solution: Keep this PR open as the active fix candidate, resolve the base conflicts, refresh the stale PR body head/evidence, and rerun exact-head provider, SDK surface, redaction, and package-boundary checks before maintainer merge review. Do we have a high-confidence way to reproduce the issue? Yes. Current main filters several tool-result replay paths to explicit text blocks before falling back to image/no-output placeholders, and the linked issue includes live Discord/Telegram evidence of text outputs degrading to placeholders. Is this the best way to solve the issue? Yes, with upgrade-risk handling. A shared extractor used across provider and transport sibling paths is the right layer, but the live branch needs conflict resolution and maintainer acceptance of the public SDK export before merge. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 0c7bac34ae68. Label changesLabel changes:
Label justifications:
Evidence reviewedPR surface: Source +320, Tests +1027, Docs 0, Other 0. Total +1347 across 29 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
|
🦞🧹 I asked ClawSweeper to review this item again. |
600fb30 to
04b0448
Compare
This comment was marked as outdated.
This comment was marked as outdated.
|
🦞🧹 I asked ClawSweeper to review this item again. |
This comment was marked as outdated.
This comment was marked as outdated.
|
🦞🧹 I asked ClawSweeper to review this item again. |
This comment was marked as outdated.
This comment was marked as outdated.
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
Landed via squash onto main.
Thanks @snowzlmbot! |
What Problem This Solves
Tool results with structured non-text content were being collapsed into generic placeholders during provider replay and transport conversion. In practice, structured non-media blocks could become
(see attached image),(see attached media), or empty/no-output text, which loses useful tool output and can mislead the model.This PR preserves explicit text first, serializes structured non-media blocks as redacted JSON text, and keeps media-only fallbacks type-aware.
Current Head
721b4898965b2b01504271face7d52d2d4af277dWhy This Change Was Made
The canonical user-facing problem is tracked in #96857. The branch already covered core provider replay paths, but the latest ClawSweeper review identified one remaining touched sibling path:
extensions/google/transport-stream.ts: the bundled Google transport still builtfunctionResponse.response.outputfrom explicittextblocks or a media placeholder only, so structured-only non-media tool results could still become empty output.This head closes that Google transport gap without importing core
src/**helpers into plugin production code. The extension now has boundary-safe structured extraction for Google tool results before media/no-output fallback.User Impact
texttool-result blocks still win and are not replaced by serialized structured fallback.(see attached image)(see attached audio)(see attached media)datafields are redacted or omitted before provider-bound replay text is generated.Implementation
functionResponse.response.outputRegression Coverage
Added or updated focused coverage for:
src/llm/providers/tool-result-text.test.tssrc/llm/providers/openai-responses-shared.test.tssrc/agents/openai-transport-stream.test.tssrc/agents/anthropic-transport-stream.test.tsextensions/google/transport-stream.test.tsfunctionResponse.response.outputtextextensions/xai/stream.test.tsEvidence
Review result addressed
Latest ClawSweeper review on
1ee3b0a60fcdbe4686cc4860bc986e742fb9c483reported:[P2] Serialize structured Google tool results before fallbackextensions/google/transport-stream.ts:705-707functionResponse.response.output.This head addresses that finding with extension-boundary-safe structured extraction and focused Google transport regression coverage.
Current branch author-side validation
node scripts/run-vitest.mjs extensions/google/transport-stream.test.ts --maxWorkers=1 --reporter=dotpnpm exec oxfmt extensions/google/transport-stream.ts extensions/google/transport-stream.test.tsnode --check extensions/google/transport-stream.tsnode --check extensions/google/transport-stream.test.tsgit diff --checkLatest patch-quality update
Current head
721b4898965b2b01504271face7d52d2d4af277dkeeps replay helpers out of the public Plugin SDK while preserving the bundled Google transport repair:src/plugin-sdk/provider-transport-runtime.tsdoes not exportextractToolResultTextordescribeToolResultMediaPlaceholder; those helpers remain outside the public provider transport runtime surface.src/plugin-sdk/provider-transport-replay-runtime.tsremains a private/local-only replay helper subpath and is excluded from packaged public SDK files.src/plugins/sdk-alias.tsnow registersprovider-transport-replay-runtimeas a private Plugin SDK subpath owned by the bundled Google provider plugin /@openclaw/google-plugin, so Google can resolve it withoutOPENCLAW_ENABLE_PRIVATE_QA_CLIwhile unrelated bundled or installed plugins cannot.src/plugins/sdk-alias.test.tscovers both the new Google private-owner alias path and normal public QA alias resolution without private QA mode.extensions/tsconfig.package-boundary.paths.jsonkeeps the private replay runtime available to package-boundary compilation without making it a public package export.Author-side validation for this head:
node scripts/run-vitest.mjs src/plugins/sdk-alias.test.ts --maxWorkers=1 --reporter=dotpassed, covering 1 file / 77 tests.pnpm plugin-sdk:surface:checkpassed; public SDK surface remains at 322 public entrypoints / 10401 public exports / 5220 public function exports, withprovider-transport-replay-runtimecounted only in local-only SDK entrypoints.pnpm lint:plugins:plugin-sdk-subpaths-exportedpassed.OPENCLAW_EXTENSION_BOUNDARY_CONCURRENCY=4 pnpm test:extensions:package-boundary:compilepassed for 116 bundled extension packages, includinggoogleandxai.node scripts/run-vitest.mjs src/plugins/contracts/plugin-sdk-subpaths.test.ts test/scripts/plugin-sdk-surface-report.test.ts src/llm/providers/tool-result-text.test.ts src/logging/redact.test.ts extensions/google/transport-stream.test.ts --maxWorkers=1 --reporter=dotpassed across 5 focused shards.pnpm exec oxlint src/plugins/sdk-alias.ts src/plugins/sdk-alias.test.ts src/plugin-sdk/provider-transport-runtime.ts src/plugin-sdk/provider-transport-replay-runtime.ts extensions/google/transport-stream.ts extensions/google/transport-stream.test.ts src/llm/providers/tool-result-text.ts src/llm/providers/tool-result-text.test.ts src/logging/redact.ts src/logging/redact.test.tspassed with 0 warnings and 0 errors.git diff --checkpassed.Current-head GitHub Actions evidence:
Real behavior proofpassed for721b4898965b2b01504271face7d52d2d4af277d: https://github.com/openclaw/openclaw/actions/runs/28491898588/job/84450193365Real behavior proof: https://github.com/openclaw/openclaw/actions/runs/28491993083/job/84450473922check-lintpassed andcheck-additional-extension-package-boundarypassed in https://github.com/openclaw/openclaw/actions/runs/28491899284. Remaining checks are still in progress.Existing provider replay proof retained from the previous detailed PR body
The previous current-head evidence remains relevant to the core provider replay slice and is repeated here so the PR body stays globally consistent instead of losing proof while the branch advances:
function_call_output.outputas text instead of(see attached image).1ee3b0a60fcdbe4686cc4860bc986e742fb9c483passed before the body-only Evidence-section gate failed:CI failures observed before this update
Real behavior prooffailed because the PR body no longer had an authoredEvidencesection:QA Smoke CIfailed inRuntime inventory drift checkwith a gateway timeout afterconfig.patchand hot reload, not in the provider/media placeholder tests:runtime-inventory-drift-check.This body update restores authored Evidence, and the code commit retriggers CI for the Google transport repair.
Latest CI follow-up repair
Current head
721b4898965b2b01504271face7d52d2d4af277drefreshes the latest patch-quality repair after the public SDK surface review:plugin-sdk/provider-transport-runtime.provider-transport-replay-runtimeis private/local-only and owned by Google provider runtime loading rather than exported as public Plugin SDK API.Risk
Low to moderate:
Rollback
Revert this PR to restore the previous provider/transport placeholder behavior.