fix(google-media): bound JSON response reads by mushuiyu886 · Pull Request #96920 · openclaw/openclaw

mushuiyu886 · 2026-06-26T03:22:37Z

Summary

Bound Google media understanding video/audio JSON response parsing through readProviderJsonResponse instead of direct res.json().
Added a Google media video regression test using a local loopback HTTP server that streams an oversized JSON response and proves the connection closes before the full response is sent.
Kept request construction, Google endpoint validation, auth headers, model normalization, and text extraction semantics unchanged.

What Problem This Solves

Google media understanding shared video/audio response parsing used a local helper that called res.json() directly after the provider HTTP status check. A successful Google Generative AI response with an oversized body could be materialized through the native JSON reader before parsing, unlike provider paths already using the shared bounded JSON reader.

User Impact

Why it matters / User impact: Users configuring Google media understanding get the same 16 MiB JSON response cap used by other provider response readers. Oversized video/audio success responses now fail early with a provider-owned error instead of being buffered through the native JSON reader.
What did NOT change: This patch only changes the Google media understanding success-response JSON reader and its video regression test. It does not change request payload construction, Google base URL validation, API key/header handling, model normalization, missing-text behavior, image description behavior, or generated text extraction.
Architecture / source-of-truth check: Google media understanding still owns Google-specific request/response shape handling in generateGeminiInlineDataText, while the canonical readProviderJsonResponse contract remains the source-of-truth for provider JSON byte limits and malformed JSON wrapping.

Origin / follow-up

Follows #96605.

Follows / completes: fix(google): bound Veo video operation response reads #96605 established the same bounded provider response-body pattern for Google video operation reads.
Gap this fills: extensions/google/media-understanding-provider.ts still parsed Google media understanding video/audio success responses through direct res.json().
Consistency: This patch reuses the existing SDK facade helper instead of reimplementing byte-limit logic in the Google extension.

Competition / linked PR analysis

Before submission I rechecked open PR overlap for Google/Gemini/media-understanding/bounded-reader work, including high-frequency bounded-reader submissions.

fix(googlechat): replace unbounded response.json() with readProviderJsonResponse #96772 touches extensions/googlechat/src/api.ts and extensions/googlechat/src/auth.ts, not Google media understanding.
fix(feishu,browser,msteams,azure-speech,bedrock-mantle,googlechat,huggingface,perplexity): bound JSON response reads #96620 touches GoogleChat plus other channel/provider files, not extensions/google/media-understanding-provider.ts.
fix(discord,telegram,github-copilot,google-meet): bound JSON response reads to prevent OOM #96618 touches Google Meet plus other integrations, not extensions/google/media-understanding-provider.ts.
fix(runway): bound video create/poll response reads #96907 is the canonical Runway create/poll bounded-reader PR, not Google media understanding.

No open PR found in this pass covers extensions/google/media-understanding-provider.ts or the Google media understanding video/audio success JSON reader.

Real behavior proof

Behavior or issue addressed: Google media understanding video/audio success responses are read through a bounded provider JSON reader, and oversized streamed provider JSON responses fail at the 16 MiB cap while closing the stream early.
Real environment tested: Local OpenClaw provider runtime path in /media/vdb/code/ai/aispace/openclaw-worktrees/followup-96605, using describeGeminiVideo() with a local loopback HTTP server returning a real streamed Response body.
Exact steps or command run after this patch: pnpm check:test-types, node scripts/run-vitest.mjs extensions/google/media-understanding-provider.video.test.ts, and a standalone loopback proof invoking describeGeminiVideo() against an oversized local HTTP response.
Evidence after fix: /media/vdb/code/ai/aispace/openclaw-followup-96605-evidence/google-media-test-types.txt, /media/vdb/code/ai/aispace/openclaw-followup-96605-evidence/google-media-vitest.txt, and /media/vdb/code/ai/aispace/openclaw-followup-96605-evidence/google-media-loopback-proof.txt
Observed result after fix: pnpm check:test-types completed successfully. The Google media video Vitest shard reported Test Files 1 passed (1) and Tests 6 passed (6). The loopback proof observed Video description failed: JSON response exceeds 16777216 bytes, closed_bytes=16908288, exceeded_cap=true, and closed_before_full_response=true.
What was not tested: Credential-backed Google Generative AI network call: out of scope for this follow-up because the changed contract is the response-body reader boundary, which is exercised through the provider function with a real loopback HTTP stream.
Fix classification: Root cause fix

Review findings addressed

N/A — new follow-up PR.

Regression Test Plan

Target test file: extensions/google/media-understanding-provider.video.test.ts
Scenario locked in: The regression test routes describeGeminiVideo() through a local loopback HTTP server that streams an 18 MiB JSON response and asserts the provider reports Video description failed: JSON response exceeds 16777216 bytes.
Why this is the smallest reliable guardrail: The test uses the public describeGeminiVideo() provider entry point and a real HTTP streamed Response, so it fails if Google media understanding regresses to direct res.json() or stops closing oversized response streams early.
pnpm check:test-types
node scripts/run-vitest.mjs extensions/google/media-understanding-provider.video.test.ts

Risk / Compatibility

Low. Successful Google media understanding JSON responses still parse into the same candidate text shape and still flow through the same missing-text validation. Responses larger than 16 MiB now fail with a bounded-reader error, matching existing provider JSON response limits.

Merge risk

Risk labels considered: merge-risk: compatibility, provider/auth surface, media-understanding provider surface.
Risk explanation: This touches a provider implementation, but production behavior is limited to replacing direct JSON materialization with the existing provider JSON reader. The compatibility change is limited to oversized JSON success bodies that now fail closed at the shared 16 MiB provider cap.
Why acceptable: Normal JSON success payloads continue through the same Google candidate text extraction. HTTP error formatting still uses the existing provider error path, and malformed JSON now gets the shared provider-owned malformed JSON wrapper.
Maintainer-ready confidence: High: the diff is small, the source-of-truth helper is reused rather than reimplemented, the test exercises the public provider entry point, and the loopback proof observes early stream closure at the response-body boundary.
Patch quality notes: No production broad catch, fallback, feature flag, or custom byte-limit implementation was added. The added test server is confined to the Google media provider test file and exists only to exercise a real streamed response boundary.

What was not changed

Did not change Google media request construction, API key/header behavior, base URL validation, model normalization, or text extraction.
Did not change Google image description providers or unrelated Google extension surfaces.

Root Cause

Root cause: The source of the unbounded read was the Google media understanding shared response helper: it used direct res.json() for HTTP success responses instead of the shared bounded provider JSON reader contract.
Why this is root-cause fix: The helper now reads through the shared provider JSON reader contract, so every Google media understanding video/audio JSON success response consumed through that helper is capped before JSON parsing while preserving the existing Google response text extraction semantics.

clawsweeper · 2026-06-26T03:23:39Z

Codex review: needs maintainer review before merge. Reviewed June 27, 2026, 9:45 AM ET / 13:45 UTC.

Summary
The PR replaces Google media understanding success-response res.json() parsing with readProviderJsonResponse and adds an oversized loopback response regression test.

PR surface: Source +1, Tests +67. Total +68 across 2 files.

Reproducibility: yes. Current main has a direct res.json() success read in the shared Google media understanding helper, and the PR's loopback streamed-response proof gives a concrete reproduction path; I did not execute it in this read-only review.

Review metrics: 1 noteworthy metric.

Provider Success JSON Cap: 1 reader changed to shared 16 MiB cap. This is the compatibility-relevant behavior change maintainers should explicitly accept before merge.

Root-cause cluster
Relationship: canonical
Canonical: #96920
Summary: This PR is the focused candidate for the Google media understanding success JSON reader; related bounded-reader PRs cover adjacent provider or channel surfaces rather than this helper.

Members:

adjacent_distinct: fix(google): bound Veo video operation response reads #96605 - Merged Google Veo video-generation response bounding touched extensions/google/video-generation-provider.ts, not the media-understanding helper.
adjacent_distinct: fix(googlechat): replace unbounded response.json() with readProviderJsonResponse #96772 - Merged GoogleChat bounded-reader work covers extensions/googlechat, not extensions/google/media-understanding-provider.ts.
adjacent_distinct: fix(runway): bound video create/poll response reads #96907 - Open Runway bounded-reader work follows the same response-limit campaign but covers a different provider.
adjacent_distinct: fix(discord,telegram,github-copilot,google-meet): bound JSON response reads to prevent OOM #96618 - The batch bounded-reader PR covers Discord, Telegram, GitHub Copilot, and Google Meet surfaces, not this Google media helper.
adjacent_distinct: fix(feishu,browser,msteams,azure-speech,bedrock-mantle,googlechat,huggingface,perplexity): bound JSON response reads #96620 - The batch bounded-reader PR covers other channel/provider modules including GoogleChat, but not Google media understanding.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

[P1] Google media understanding success responses that are otherwise valid but larger than 16 MiB will now fail at the shared provider cap instead of being parsed by the native JSON reader.

Maintainer options:

Accept The Shared Provider Cap (recommended)
Merge after maintainers agree Google media understanding success JSON should fail at the same 16 MiB cap used by other provider JSON readers.
Specialize The Google Media Cap
If this provider has a documented need for larger success JSON, choose and test an explicit Google media cap before merge.

Next step before merge

No automated repair is indicated; maintainers need to accept the compatibility cap and let normal merge checks finish.

Security
Cleared: The diff reduces unbounded provider-body memory exposure and adds no dependency, workflow, credential, package-resolution, or secret-handling surface.

Review details

Best possible solution:

Use the shared 16 MiB provider JSON cap for this helper unless maintainers explicitly document a Google media-specific larger cap before merge.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main has a direct res.json() success read in the shared Google media understanding helper, and the PR's loopback streamed-response proof gives a concrete reproduction path; I did not execute it in this read-only review.

Is this the best way to solve the issue?

Yes. Replacing the single shared audio/video success reader with the existing SDK bounded JSON helper is the narrowest maintainable fix, and adjacent Google/GoogleChat bounded-reader work does not cover this helper.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against c5d34c8376f8.

Label changes

Label justifications:

P2: This is a normal-priority provider hardening fix with limited blast radius in the Google media understanding path.
merge-risk: 🚨 compatibility: Merging changes oversized Google media understanding success responses from unbounded native parsing to fail closed at the shared 16 MiB provider JSON cap.
rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes after-fix loopback HTTP proof and copied output showing the provider path fails at the 16 MiB cap and closes before the full oversized response is sent.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix loopback HTTP proof and copied output showing the provider path fails at the 16 MiB cap and closes before the full oversized response is sent.

Evidence reviewed

PR surface:

Source +1, Tests +67. Total +68 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	3	2	+1
Tests	1	67	0	+67
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	70	2	+68

What I checked:

Current-main gap: Current main still parses Google media understanding success JSON through native res.json() after the provider HTTP status check. (extensions/google/media-understanding-provider.ts:100, c5d34c8376f8)
Latest release gap: The latest release tag still has the same unbounded success-body res.json() call, so the PR is not obsolete by shipped behavior. (extensions/google/media-understanding-provider.ts:100, aa69b12d0086)
PR fix shape: The PR imports readProviderJsonResponse from the plugin SDK provider HTTP facade and uses it in the shared helper that feeds both audio and video media understanding. (extensions/google/media-understanding-provider.ts:101, d7d942ca918b)
Bounded-reader contract: readProviderJsonResponse reads through the shared 16 MiB default provider JSON cap and wraps overflow as ${label}: JSON response exceeds ... bytes. (src/agents/provider-http-errors.ts:312, c5d34c8376f8)
Stream cancellation contract: The underlying readResponseWithLimit cancels the body reader when the next chunk would exceed the cap, which is the load-bearing behavior the PR test asserts. (packages/media-core/src/read-response-with-limit.ts:96, c5d34c8376f8)
Regression coverage: The PR adds a loopback HTTP server test that drives describeGeminiVideo, expects the 16 MiB JSON overflow, and checks the server closes before the planned 18 MiB response is fully sent. (extensions/google/media-understanding-provider.video.test.ts:161, d7d942ca918b)

Likely related people:

vincentkoc: Vincent Koc carried recent Google media provider request-shaping refactors and also authored the PR's follow-up test assertion relaxation commit, making him the strongest routing candidate for this provider path. (role: recent area contributor; confidence: high; commits: f28f0f29ba90, 65842aabada0, d7d942ca918b; files: extensions/google/media-understanding-provider.ts, src/media-understanding/shared.ts, extensions/google/media-understanding-provider.video.test.ts)
steipete: Peter Steinberger introduced the Google media understanding provider during the move of media understanding into vendor plugins, which is the central feature history for this file. (role: introduced feature area; confidence: medium; commits: c081dc52b7c6; files: extensions/google/media-understanding-provider.ts, extensions/google/index.ts)
Alix-007: Alix-007 authored the merged adjacent Google Veo bounded response fix that this PR explicitly follows, though that work covered video generation rather than media understanding. (role: recent adjacent contributor; confidence: medium; commits: bfc33ac11405; files: extensions/google/video-generation-provider.ts, extensions/google/video-generation-provider.test.ts)
wangmiao0668000666: wangmiao0668000666 recently landed a sibling GoogleChat change replacing unbounded response.json() with readProviderJsonResponse, useful context for the response-limit pattern. (role: recent sibling bounded-reader contributor; confidence: low; commits: 4f3d81b91855; files: extensions/googlechat/src/api.ts, extensions/googlechat/src/auth.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

vincentkoc · 2026-06-27T13:46:44Z

Maintainer review and merge prep done.

What changed in prep:

Kept the production fix: Google media understanding now reads successful audio/video JSON through the shared bounded provider JSON reader instead of direct res.json().
Added one maintainer test fix: relaxed the server-observed byte lower-bound assertion so the oversized-response test proves stable behavior without timing-sensitive byte accounting.

Evidence:

node scripts/run-vitest.mjs extensions/google/media-understanding-provider.video.test.ts passed on the PR head before the maintainer fix.
node scripts/run-vitest.mjs extensions/google/media-understanding-provider.video.test.ts passed again after the maintainer fix.
Autoreview with Codex gpt-5 found the test-flake issue; I fixed it. The remaining explicit-maxBytes suggestion was rejected because readProviderJsonResponse is the shared provider JSON cap owner and this call site should not duplicate that constant.
scripts/pr review-validate-artifacts 96920 passed.
OPENCLAW_TESTBOX=1 scripts/pr prepare-run 96920 passed.
Exact-head hosted CI is green for d7d942ca918b5b28218c57aa7c1ab225a8d37ef1 with no attention checks.

Known proof gap: no credential-backed live Google Generative AI call was run; the changed contract is the response-body reader boundary and is exercised through the provider entry point with a loopback streamed Response.

vincentkoc · 2026-06-27T13:47:03Z

Merged via squash.

Prepared head SHA: d7d942ca918b5b28218c57aa7c1ab225a8d37ef1
Landed commit: a7bfc06f4597ee946cbc066cb41750a7e8f59c52

* fix(google-media): bound JSON response reads * test(google): relax media response cap assertion --------- Co-authored-by: Vincent Koc <[email protected]>

openclaw-barnacle Bot added extensions: google size: S labels Jun 26, 2026

This was referenced Jun 26, 2026

fix(runway): bound video create/poll response reads #96907

Merged

fix(googlechat): replace unbounded response.json() with readProviderJsonResponse #96772

Merged

vincentkoc self-assigned this Jun 27, 2026

mushuiyu886 and others added 2 commits June 27, 2026 06:37

fix(google-media): bound JSON response reads

1aea6ec

test(google): relax media response cap assertion

d7d942c

vincentkoc force-pushed the fix/followup-96605 branch from 58f2c78 to d7d942c Compare June 27, 2026 13:37

vincentkoc merged commit a7bfc06 into openclaw:main Jun 27, 2026
88 checks passed

github-actions Bot mentioned this pull request Jun 27, 2026

📡 Upstream Digest — 2026-06-27 15:09 UTC curtismercier/openclaw-mods#1159

Open

clawsweeper Bot mentioned this pull request Jun 28, 2026

test: stabilize startup session migration flake #97370

Merged

wangmiao0668000666 mentioned this pull request Jul 3, 2026

fix(ollama): bound native streaming 200 success-body NDJSON reads at 16 MiB #99585

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(google-media): bound JSON response reads#96920

fix(google-media): bound JSON response reads#96920
vincentkoc merged 2 commits into
openclaw:mainfrom
mushuiyu886:fix/followup-96605

mushuiyu886 commented Jun 26, 2026

Uh oh!

clawsweeper Bot commented Jun 26, 2026 •

edited

Loading

Uh oh!

vincentkoc commented Jun 27, 2026

Uh oh!

Uh oh!

vincentkoc commented Jun 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

mushuiyu886 commented Jun 26, 2026

Summary

What Problem This Solves

User Impact

Origin / follow-up

Competition / linked PR analysis

Real behavior proof

Review findings addressed

Regression Test Plan

Risk / Compatibility

Merge risk

What was not changed

Root Cause

Uh oh!

clawsweeper Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vincentkoc commented Jun 27, 2026

Uh oh!

Uh oh!

vincentkoc commented Jun 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

clawsweeper Bot commented Jun 26, 2026 •

edited

Loading