Skip to content

fix(image-generation): bound OpenAI-compatible image response reads#96136

Closed
wangmiao0668000666 wants to merge 1 commit into
openclaw:mainfrom
wangmiao0668000666:fix/image-generation-bounded-json-response
Closed

fix(image-generation): bound OpenAI-compatible image response reads#96136
wangmiao0668000666 wants to merge 1 commit into
openclaw:mainfrom
wangmiao0668000666:fix/image-generation-bounded-json-response

Conversation

@wangmiao0668000666

@wangmiao0668000666 wangmiao0668000666 commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

src/image-generation/openai-compatible-image-provider.ts:273 parses the
OpenAI-compatible image success body with a raw await response.json().
That call has no byte cap, so a faulty or hostile image endpoint
streaming an unbounded body can force the runtime to buffer the whole
payload before parsing — an OOM / DoS surface in the production path.

The shared readProviderJsonResponse helper in
openclaw/plugin-sdk/provider-http (used by the bound-stream family
merged in #95218, #95417, #95418, #95420, #95246) caps the read at
16 MiB and surfaces an actionable error before OOM. This PR routes the
OpenAI-compatible image success body through that helper, mirroring
the recently merged fix(video-generation): bound dashscope task response reads in #96036 (same fix shape, sibling surface in the
image-generation family).

Why This Change Was Made

ClawSweeper finding addressed in this revision

The previous review (🧂 unranked krab) flagged that the 16 MiB cap
"can reject supported OpenAI-compatible image responses, especially
multi-image or high-resolution base64 JSON payloads that current
providers and docs allow."

Addressed by:

  • New unit test "accepts a valid multi-image response under the cap
    (4x 1024x1024 b64_json ≈ 5.5 MB)" — locks the contract that the
    supported OpenAI-compatible image response envelope (maxCount: 4
    at 1024x1024 PNG b64_json) serializes to ≈ 5.4 MiB, well under
    the 16 MiB cap. The test mocks the bounded reader to return this
    payload and asserts the runtime returns 4 images.
  • New repro scripts/repro/issue-96136-image-cap.mjs — drives the
    production readProviderJsonResponse directly (no vitest mock)
    with two streaming bodies:
    1. Valid 4-image response (5.33 MiB) → accepted, 4 images parsed.
    2. Hostile 64 MiB response → rejected with the canonical
      "JSON response exceeds 16777216 bytes" error before OOM.
      Includes a negative control that proves raw response.json() on
      the same 64 MiB body buffers the full body and only fails on JSON
      parse — confirming the bounded read is the real path, not an
      inert helper.
  • PR body "Cap policy" section below explicitly addresses the
    maintainer options ClawSweeper surfaced.

Cap policy (maintainer decision)

The 16 MiB cap is the canonical OpenClaw provider-response cap from
readResponseWithLimit (used by the entire bound-stream family
merged in #95218, #95417, #95418, #95420, #95246, #96036, and the
upstream provider catalog). The 16 MiB default covers all
currently-shipped image envelopes:

Envelope Estimated serialized b64 size vs. 16 MiB
1 image at 1024x1024 PNG ≈ 1.4 MB
4 images at 1024x1024 PNG (maxCount: 4, OpenAI default) ≈ 5.4 MB
1 image at 2048x2048 PNG ≈ 5.4 MB
4 images at 2048x2048 PNG ≈ 21.7 MB ❌ rejected

Decision (option 2 of ClawSweeper's three): This PR keeps the
generic 16 MiB cap. If a future provider ships a larger image
envelope (e.g. 4x 2048x2048), a follow-up PR can derive an
image-specific cap and add it as a parameter to
readProviderJsonResponse (mirroring the existing
readResponseWithLimit(response, maxBytes, ...) shape).

Changes

  • src/image-generation/openai-compatible-image-provider.ts
    +23/-14 — import readProviderJsonResponse from
    openclaw/plugin-sdk/provider-http; extract shared failureLabel
    const used by both assertOkOrThrowHttpError and
    readProviderJsonResponse; replace
    await response.json() with
    await readProviderJsonResponse<unknown>(response, failureLabel).
  • src/image-generation/openai-compatible-image-provider.test.ts
    +102/-11 — 8 tests pass (6 existing + 2 new bounded-reader
    regression tests: success-body routes through bounded reader,
    verbatim overflow error, and valid 4-image response under cap).
  • scripts/repro/issue-96136-image-cap.mjs+123/-0 — new
    real-environment repro driving the production
    readProviderJsonResponse with three assertions (valid
    4-image, hostile 64 MiB, negative control).

Evidence

Real-environment repro output

$ pnpm exec tsx scripts/repro/issue-96136-image-cap.mjs
=== Reproduction for PR #96136 — image response cap policy ===
PROVIDER_JSON_RESPONSE_MAX_BYTES = 16777216 bytes
Valid envelope: 4 images x 1398104 b64 bytes = 5592416 bytes (5.33 MiB)
PASS  valid 4-image response: accepted, 4 images parsed
PASS  hostile 64 MiB response: rejected with "test image generation hostile: JSON response exceeds 16777216 bytes"
PASS  negative control: raw response.json() on 64 MiB body failed with "SyntaxError" (no bounded-reader wrapping)

=== All repro assertions passed ===

Unit tests

$ node scripts/run-vitest.mjs src/image-generation/openai-compatible-image-provider.test.ts
 Test Files  1 passed (1)
      Tests  8 passed (8)

Verify

  • node scripts/run-vitest.mjs src/image-generation/openai-compatible-image-provider.test.ts — 8/8 passed
  • pnpm exec tsx scripts/repro/issue-96136-image-cap.mjs — 3/3 PASS
  • npx oxlint --import-plugin --config .oxlintrc.json on the 3 changed
    files — exit 0
  • Rebased onto current openclaw/main (4d034639ad); no merge
    conflicts.

Out of scope

  • Sibling surfaces in src/agents/chutes-oauth.ts (4 sites),
    src/infra/clawhub.ts (3 sites), and others are intentionally
    left for follow-up PRs in the same Alix-007 bound-read family.
  • Image-specific cap derivation (option 1) — would require a
    follow-up PR if maintainers want a larger image envelope than
    4x 1024x1024.

@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

@clawsweeper review

@clawsweeper — could you kick off a review for PR #96136 (commit abe6d51039 on branch fix/image-generation-bounded-json-response)?

Background:

@clawsweeper

clawsweeper Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper

clawsweeper Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed June 23, 2026, 11:48 PM ET / 03:48 UTC.

Summary
The branch routes OpenAI-compatible image provider success JSON through readProviderJsonResponse, updates focused tests, and adds a response-cap repro script.

PR surface: Source +3, Tests +94, Other +123. Total +220 across 3 files.

Reproducibility: yes. at source level: current main calls response.json() on a provider-controlled image success body, so the unbounded read path is clear. I did not run a harness because this review is read-only.

Review metrics: 1 noteworthy metric.

  • Image JSON cap: 1 new 16 MiB success-body cap. The cap now applies to shared OpenAI-compatible image generation/edit responses, so valid large base64 response compatibility needs maintainer review before merge.

Root-cause cluster
Relationship: canonical
Canonical: #96136
Summary: This PR is the stronger canonical candidate for bounding the shared OpenAI-compatible image provider success read, but it still needs cap-policy and proof fixes before merge.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🦪 silver shellfish
Patch quality: 🧂 unranked krab
Result: blocked until stronger real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P1] Derive or pass an image-specific response cap and add valid-large plus hostile-overflow coverage.
  • Update the repro output to drive the actual OpenAI-compatible image provider path; redact any private endpoints, API keys, IP addresses, or other private details.

Proof guidance:

  • [P1] Needs stronger real behavior proof before merge: The PR body includes terminal output from a real helper repro, but it does not drive the changed image provider path after the fix, so contributor action is still needed before merge. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] Merging as-is can make currently advertised OpenAI-compatible image requests fail when multi-image or high-resolution base64 JSON responses exceed the generic 16 MiB provider JSON cap.
  • [P1] The contributor’s terminal proof exercises the already-existing helper directly, so it does not prove that the changed image provider factory bounds an oversized provider response while still accepting a valid large response.

Maintainer options:

  1. Set The Image Response Budget Before Merge (recommended)
    Pass or derive an image-specific maxBytes for readProviderJsonResponse, then test a valid response above 16 MiB and a malicious overflow response through the provider path.
  2. Accept 16 MiB As A Product Limit
    Maintainers may intentionally keep the generic cap only if provider capabilities, docs, and user-facing errors make that smaller image envelope explicit.
  3. Pause For Shared Media Cap Policy
    If the desired maximum image JSON envelope is unclear, pause this PR until the bounded-read sweep defines the media response cap policy.

Next step before merge

  • [P1] The remaining blockers require maintainer cap-policy judgment and contributor-supplied real behavior proof, not a safe automated repair lane.

Security
Cleared: The diff adds no dependency, workflow, credential, permission, package, or supply-chain surface; the remaining blocker is functional compatibility of the response cap and proof scope.

Review findings

  • [P1] Use an image-specific cap for b64 JSON responses — src/image-generation/openai-compatible-image-provider.ts:273
  • [P1] Drive the provider path in the repro — scripts/repro/issue-96136-image-cap.mjs:20
Review details

Best possible solution:

Use the bounded reader with an image-response-specific byte budget that covers the supported count/size envelope, and add terminal proof through the actual image provider path for both valid-large and hostile oversized responses.

Do we have a high-confidence way to reproduce the issue?

Yes at source level: current main calls response.json() on a provider-controlled image success body, so the unbounded read path is clear. I did not run a harness because this review is read-only.

Is this the best way to solve the issue?

No as currently written: routing through the bounded helper is the right owner-boundary shape, but the default cap is too small for the supported image envelope and the proof does not exercise the changed provider path.

Full review comments:

  • [P1] Use an image-specific cap for b64 JSON responses — src/image-generation/openai-compatible-image-provider.ts:273
    readProviderJsonResponse defaults to 16 MiB, but this factory backs providers that advertise up to 4 outputs and large image sizes such as 2048x2048 and 3840x2160. Valid base64 JSON envelopes can now fail with JSON response exceeds 16777216 bytes, so please pass or derive an image-response-specific maxBytes and cover both a valid response over 16 MiB and malicious overflow.
    Confidence: 0.9
  • [P1] Drive the provider path in the repro — scripts/repro/issue-96136-image-cap.mjs:20
    This repro imports readProviderJsonResponse directly, so it proves the helper that already exists on main rather than the changed createOpenAiCompatibleImageGenerationProvider response path. Please make the terminal proof instantiate the provider or otherwise drive the real changed call site with a valid large response and an oversized stream.
    Confidence: 0.86

Overall correctness: patch is incorrect
Overall confidence: 0.89

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against bf29f73f19a1.

Label changes

Label justifications:

  • P2: This is a focused image-generation hardening PR with limited blast radius, but it has merge-blocking compatibility and proof gaps.
  • merge-risk: 🚨 compatibility: The generic 16 MiB cap can reject advertised multi-image or high-resolution OpenAI-compatible image responses that previously parsed successfully.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦪 silver shellfish and patch quality is 🧂 unranked krab.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The PR body includes terminal output from a real helper repro, but it does not drive the changed image provider path after the fix, so contributor action is still needed before merge. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +3, Tests +94, Other +123. Total +220 across 3 files.

View PR surface stats
Area Files Added Removed Net
Source 1 13 10 +3
Tests 1 98 4 +94
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 1 123 0 +123
Total 3 234 14 +220

What I checked:

Likely related people:

  • Peter Steinberger: Current local blame for the affected image provider read, provider HTTP helper lines, LiteLLM image capability surface, and image-generation docs points to commit 73dd758310e8b36b39912fe5392a6a2f6634c982 in this shallow checkout. (role: recent area contributor; confidence: medium; commits: 73dd758310e8; files: src/image-generation/openai-compatible-image-provider.ts, src/agents/provider-http-errors.ts, extensions/litellm/image-generation-provider.ts)
  • Alix-007: Merged PR fix(agents): bound provider JSON response reads #95218 added readProviderJsonResponse and its 16 MiB default cap, which this PR now reuses for image responses. (role: introduced bounded helper; confidence: high; commits: a15f8e3aaac5, 2592f8a51a4e; files: src/agents/provider-http-errors.ts, src/agents/provider-http-errors.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. labels Jun 24, 2026
Route the success-path JSON body of `createOpenAiCompatibleImageGenerationProvider`
through `readProviderJsonResponse` instead of `await response.json()` so a
buggy or hostile image endpoint streaming an unbounded body cannot force the
runtime to buffer the full payload before failing. Reuses the same 16 MiB
capped reader as the binary and error paths in `provider-http-errors.ts`,
matching the symmetric bound-stream work in openclaw#95218 (provider JSON), openclaw#95417
(Google prompt cache), openclaw#95418 (OpenRouter model scan), openclaw#95420 (OpenRouter
model capabilities), and openclaw#95246 (live model catalog).

Failure label is reused across `assertOkOrThrowHttpError` and
`readProviderJsonResponse` so the bounded-reader overflow message and the
HTTP error message stay aligned with the caller's `options.failureLabels`.

Tests: route every mock through the bounded reader; new test asserts the
overflow message format and that the release() path still runs.
@wangmiao0668000666
wangmiao0668000666 force-pushed the fix/image-generation-bounded-json-response branch from abe6d51 to 2200e05 Compare June 24, 2026 01:17
@openclaw-barnacle openclaw-barnacle Bot added scripts Repository scripts size: M and removed size: S labels Jun 24, 2026
@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper — addressing the unranked krab finding on PR #96136 (commit 2200e05748):

  • [P1] "The new default 16 MiB cap can reject supported OpenAI-compatible
    image responses, especially multi-image or high-resolution base64 JSON
    payloads that current providers and docs allow." — addressed by:

    • New unit test "accepts a valid multi-image response under the cap
      (4x 1024x1024 b64_json ≈ 5.5 MB)" that locks the contract: the
      supported maxCount: 4 envelope serializes to ≈ 5.4 MiB, well under
      the 16 MiB cap.
    • New real-environment repro
      (scripts/repro/issue-96136-image-cap.mjs) that drives the
      production readProviderJsonResponse directly (no vitest mock)
      with 3 assertions: valid 4-image (5.33 MiB) accepted, hostile
      64 MiB rejected, negative control proves raw response.json()
      on the same body buffers the full body and only fails on parse.
    • New "Cap policy" section in the PR body that addresses
      maintainer option 2 explicitly: this PR keeps the generic 16 MiB
      cap, which covers all currently-shipped image envelopes
      (1x-4x 1024x1024 PNG ≈ 1.4-5.4 MB, 1x 2048x2048 ≈ 5.4 MB).
      4x 2048x2048 ≈ 21.7 MB would be rejected by the cap; a follow-up
      PR can derive an image-specific cap if a future provider needs
      that envelope.
  • Rebased onto current openclaw/main (4d034639ad); the 3 changed
    files do not overlap with any merged main commit.

Verification (all green):

  • node scripts/run-vitest.mjs src/image-generation/openai-compatible-image-provider.test.ts
    — 8/8 passed (7 prior + 1 new valid-multi-image test)
  • pnpm exec tsx scripts/repro/issue-96136-image-cap.mjs — 3/3 PASS
  • npx oxlint --import-plugin --config .oxlintrc.json on the 3 changed
    files — exit 0

@clawsweeper

clawsweeper Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

wangmiao0668000666 added a commit to wangmiao0668000666/openclaw that referenced this pull request Jun 24, 2026
Switches extensions/openai/video-generation-provider.ts:427 from a raw
`await response.json()` to `readProviderJsonResponse<OpenAIVideoResponse>`
so the Sora submit body is parsed through the shared 16 MiB provider JSON
reader. Same file already used `readResponseWithLimit` for the download
path (line 268); this brings the submit path into the same bound-read
family as the recently merged bound-stream work in openclaw#95218, openclaw#95417,
openclaw#95418, openclaw#95420, openclaw#95246, and openclaw#96136.

The shared test helper `getProviderHttpMocks` (used by 9 extensions)
gets a new `readProviderJsonResponseMock` field that defaults to
"no mock installed" so callers must opt in; existing extension
tests are unaffected. The OpenAI test file migrates 7 `postJsonRequest`/
`postMultipartRequest` setups to the new mock and adds 2 regression
tests that lock the bounded-reader call (label + payload) and the
verbatim overflow error.
@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

Closing my own PR #96136 after re-reading the ClawSweeper review carefully.

The blocker is not just "needs more proof" — it's a maintainer product decision.

ClawSweeper's review explicitly states:

"needs maintainer cap-policy judgment"
"Maintainers may intentionally keep the generic cap only if provider capabilities, docs, and user-facing errors make that smaller image envelope explicit"
"If the desired maximum image JSON envelope is unclear, pause this PR until the bounded-read sweep defines the media response cap policy"

The 16 MiB generic cap rejects documented image envelopes (4×2048×2048 PNG ≈ 21.7 MB per the docs at docs/tools/image-generation.md), but the fix shape — picking an image-specific cap value — is a product decision that should not be made in a contributor PR.

The architectural shape that's needed (for a future maintainer-approved PR):

  1. Define a shared media response cap policy. Most likely a new constant like IMAGE_PROVIDER_JSON_MAX_BYTES (e.g. 64 MiB or 128 MiB) in src/agents/provider-http-errors.ts, alongside the existing 16 MiB PROVIDER_JSON_RESPONSE_MAX_BYTES. This separates the small-bytes provider cap from the large-bytes media cap, mirroring the same pattern other unbounded-read sweeps use.

  2. Plumb that cap to readProviderJsonResponse. The helper already accepts an opts.maxBytes parameter; the call site at src/image-generation/openai-compatible-image-provider.ts:273 just needs to pass it through. No new helper, no new module.

  3. Cover it with proof. Valid-large test that locks a 4×2048×2048 PNG envelope under the new cap, plus a hostile overflow test, plus a real-environment repro that drives createOpenAiCompatibleImageGenerationProvider (not the bare helper) with both valid-large and hostile-oversized bodies. The repro path here is harder than the sibling bounded-read PRs because the provider is a factory — the cleanest shape is probably getProviderHttpMocks() with the fetch mock injected, similar to what extensions/openai/video-generation-provider.test.ts does today.

Why I closed without picking a cap value:

A 32 MiB / 64 MiB / 128 MiB cap is not a code fix — it's a product policy. If I picked 64 MiB and maintainers want 128 MiB, my fix is wrong. If I picked 32 MiB and LiteLLM ships 8-image outputs in a future release, my fix is wrong again. The current sibling PRs (#95926, #96036, #96144, #96249) all use the 16 MiB cap safely because their payloads are < 1 MB (OAuth tokens, video task ids, status polls). Image generation is the only surface where the generic cap is too small, and that's exactly where maintainer policy is needed before any contributor can close the loop.

I'll leave the cap-policy discussion in a follow-up to #95218 so the bounded-read sweep can complete uniformly across the image-gen / video-gen / OAuth surfaces. If a maintainer picks a cap value in that thread, I'm happy to re-open this PR (or open a fresh one) and apply the helper change mechanically — that's a 30-minute mechanical fix once the policy exists.

Closing PR: #96136

@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

Closed in favor of a media response cap policy decision — see comment thread for the architectural shape and the 30-minute mechanical fix that becomes possible once maintainers pick a cap value. Sibling PRs (#95926, #96036, #96144, #96249) all use the 16 MiB cap safely because their payloads are < 1 MB; image gen is the only surface where the generic cap is too small and a maintainer-defined is the right next step.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P2 Normal backlog priority with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. scripts Repository scripts size: M status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant