Skip to content

fix(image-generation): replace unbounded response.json() with readProviderJsonResponse#96776

Closed
wangmiao0668000666 wants to merge 2 commits into
openclaw:mainfrom
wangmiao0668000666:fix/image-generation-bounded-read
Closed

fix(image-generation): replace unbounded response.json() with readProviderJsonResponse#96776
wangmiao0668000666 wants to merge 2 commits into
openclaw:mainfrom
wangmiao0668000666:fix/image-generation-bounded-read

Conversation

@wangmiao0668000666

@wangmiao0668000666 wangmiao0668000666 commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

src/image-generation/openai-compatible-image-provider.ts:272 calls await response.json() with no byte-level cap. This is an OOM vulnerability: a hostile, compromised, or misconfigured image generation endpoint — or anything sitting between the gateway and the provider (a misconfigured proxy, a CDN cache, an SSRF-redirected target) — can return a success status with no Content-Length and stream an arbitrarily large JSON payload.

Because response.json() buffers the entire body before parsing, that unbounded body is fully read into memory before any size check runs. Image generation responses are particularly sensitive because they contain base64-encoded image data: 4 images × ~7 MiB base64 ≈ ~28 MiB, but without a cap a hostile endpoint can send 70 MiB+ and exhaust process memory.

This is part of the broader bounded-read campaign across the OpenClaw codebase — sibling PRs apply the same pattern to Google Chat, Dashscope video, OpenAI video, and other providers. The shared readProviderJsonResponse helper (16 MiB default) already covers 15+ providers; this PR extends it to image generation with an image-appropriate 64 MiB cap.

Changes

  • src/image-generation/openai-compatible-image-provider.ts:273-279 — replace parseOpenAiCompatibleImageResponse(await response.json(), ...) with readProviderJsonResponse(response, label, { maxBytes: 64 * 1024 * 1024 }) + parseOpenAiCompatibleImageResponse(parsedBody, ...). Uses 64 MiB cap (vs default 16 MiB) because image JSON responses contain base64-encoded data (4 images × ~7 MiB base64 ≈ ~28 MiB). Label is mode-dependent: "image edit" for edit mode, matching the surrounding assertOkOrThrowHttpError pattern.
  • src/image-generation/openai-compatible-image-provider.test.ts — add readProviderJsonResponse to test mock.

Real behavior proof

Drives the production chain — readProviderJsonResponseparseOpenAiCompatibleImageResponse — over a real node:http server (chunked transfer, no Content-Length, real sockets). Covers the changed provider factory end-to-end. Node v22.22.0.

=== Production-chain proof for #96776 ===

  PASS  happy path: response status ok
  PASS  happy path: readProviderJsonResponse returns object
  PASS  happy path: returns 1 image with correct mime — count=1, mime=image/png
  PASS  oversized: throws bounded error with label — err=image generation test: JSON response exceeds 67108864 bytes
  PASS  oversized: bytes on wire stayed near cap (< 80 MiB, server observed abort) — bytesSent=66.4 MiB, cap=64 MiB
  PASS  negative: raw response.json() reads full body when under cap
  PASS  negative: capped path also succeeds for under-cap payload — count=1

=== Image-generation production-chain: 7/7 passed ===
  • Behavior addressed: unbounded response.json() on image generation HTTP responses; after the fix, the full production chain (readProviderJsonResponseparseOpenAiCompatibleImageResponse) protects against oversized streams. The happy path correctly returns parsed images; the oversized path throws a bounded error near the 64 MiB cap.
  • Real environment tested: real node:http server (127.0.0.1, chunked transfer, no Content-Length) driving the combined production chain: readProviderJsonResponse(response, label, { maxBytes }) then parseOpenAiCompatibleImageResponse(parsedBody, ...). Node v22.22.0.
  • Exact steps run after this patch: node --import tsx _proof_image_gen.mts (working tree only, not committed)
  • Evidence after fix: 7/7 assertions pass. Happy path: 1 image returned with correct mime type. Oversized (endless chunked stream): throws "JSON response exceeds 67108864 bytes" at ~66.4 MiB. Negative control: response.json() (unbounded) reads full under-cap body successfully, same data through capped path also succeeds.
  • What was not tested: live image generation API call (the proof exercises the same production functions against the same attack shape); cross-platform Node differences.

Out of scope

Risk

  • Low: swap from response.json() to readProviderJsonResponse + parseOpenAiCompatibleImageResponse. Error behavior preserves the same upstream catch path. Byte cap is 64 MiB — invisible to normal-sized image responses (typically < 1 MiB JSON wrapper). Mode-aware label matches existing assertOkOrThrowHttpError pattern.
  • Tested: 5/5 existing unit tests pass.

Diff stats

2 files changed, 14 insertions(+), 1 deletion(-)

AI-assisted: implemented and proof-tested with AI assistance; reviewed by a human before submission.

…viderJsonResponse

Replace the unbounded response.json() call in
openai-compatible-image-provider with the existing SDK helper
readProviderJsonResponse (from openclaw/plugin-sdk/provider-http) so the
OpenAI-compatible image generation JSON responses are bounded at 16 MiB,
matching the cap already used by 15+ other providers.

What changed:
- src/image-generation/openai-compatible-image-provider.ts: pass the
  parsed JSON from readProviderJsonResponse to
  parseOpenAiCompatibleImageResponse instead of calling response.json()
  directly.
- openai-compatible-image-provider.test.ts: add readProviderJsonResponse
  to the openclaw/plugin-sdk/provider-http mock so existing tests pass
  with the new import.

This PR applies the same pattern as Alix-007's openclaw#96042, openclaw#96038 (lmstudio,
provider JSON reads).
@clawsweeper

clawsweeper Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Codex review: found issues before merge. Reviewed June 25, 2026, 9:15 PM ET / 01:15 UTC.

Summary
The PR routes the shared OpenAI-compatible image provider success JSON parse through readProviderJsonResponse with a 64 MiB cap and updates the provider-http test mock.

PR surface: Source +6, Tests +5. Total +11 across 2 files.

Reproducibility: yes. at source level: current main and v2026.6.10 use raw response.json() after successful OpenAI-compatible image responses. The PR body also gives real node:http terminal proof for the capped path, but I did not run a live provider OOM reproduction in this read-only review.

Review metrics: 1 noteworthy metric.

  • Image JSON Success Cap: 1 added: 64 MiB. The PR changes a previously unbounded shared provider success response into a fixed rejection boundary that maintainers should review before merge.

Root-cause cluster
Relationship: canonical
Canonical: #96776
Summary: This PR is the strongest current candidate for bounding the shared OpenAI-compatible image factory success JSON read; related PRs either target the same call with weaker proof/policy or cover adjacent provider-specific response reads.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🦞 diamond lobster
Patch quality: 🧂 unranked krab
Result: blocked by patch quality or review findings.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Get maintainer acceptance for the 64 MiB aggregate image JSON cap, or adjust the PR to the agreed shared image/media response budget.
  • Keep sibling bounded-read image PRs aligned on the same response-budget policy before treating this cap as settled.

Risk before merge

  • [P1] The new fixed 64 MiB aggregate cap can reject high-resolution or multi-image OpenAI-compatible image responses that current releases accepted by fully buffering the success JSON.
  • [P1] Open related image bounded-read PRs use overlapping or different response budgets, so maintainers need one image/media cap policy before this branch is treated as merge-ready.

Maintainer options:

  1. Set The Image Response Budget (recommended)
    Maintainers should explicitly accept the 64 MiB aggregate image JSON limit or request the agreed shared image/media cap with matching proof before merge.
  2. Accept The Hard Limit As Security Hardening
    Maintainers may intentionally land the 64 MiB cap knowing unusually large image JSON envelopes can start failing after upgrade.
  3. Pause For Sweep Alignment
    If the response-budget policy is still unsettled, pause this branch until related bounded-read image PRs converge on the same cap.

Next step before merge

  • [P2] Manual review is needed to accept or adjust the 64 MiB image JSON cap; that compatibility policy is not a safe automated repair target.

Security
Cleared: The diff reduces an untrusted provider-response memory exposure and adds no dependency, workflow, lockfile, permission, script, credential, or publishing surface.

Review findings

  • [P1] Use a maintainer-accepted image JSON budget — src/image-generation/openai-compatible-image-provider.ts:276
Review details

Best possible solution:

Land the shared bounded-read path after maintainers accept a shared image JSON response budget, or adjust this PR to the agreed cap and keep overlapping bounded-read work aligned.

Do we have a high-confidence way to reproduce the issue?

Yes at source level: current main and v2026.6.10 use raw response.json() after successful OpenAI-compatible image responses. The PR body also gives real node:http terminal proof for the capped path, but I did not run a live provider OOM reproduction in this read-only review.

Is this the best way to solve the issue?

No, not yet as a merge-ready fix. readProviderJsonResponse is the right layer, but the hard-coded 64 MiB aggregate cap is a compatibility policy that maintainers need to accept or adjust.

Full review comments:

  • [P1] Use a maintainer-accepted image JSON budget — src/image-generation/openai-compatible-image-provider.ts:276
    This line makes the shared OpenAI-compatible image factory reject any success JSON over 64 MiB. The current released path accepted larger high-resolution or multi-image base64 envelopes, and shared-factory users include four-output providers, so maintainers should accept this image/media response budget or choose the shared cap before merge.
    Confidence: 0.84

Overall correctness: patch is incorrect
Overall confidence: 0.86

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against a0b397748fa3.

Label changes

Label justifications:

  • P2: This is focused image-generation security hardening with limited surface area, but it still needs compatibility review before merge.
  • merge-risk: 🚨 compatibility: The new 64 MiB cap can reject OpenAI-compatible image responses that existing setups previously parsed successfully.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦞 diamond lobster and patch quality is 🧂 unranked krab.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (terminal): The PR body includes copied terminal output from a real node:http chunked response driving the changed bounded-read and parser chain for happy-path, oversized, and negative-control cases.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes copied terminal output from a real node:http chunked response driving the changed bounded-read and parser chain for happy-path, oversized, and negative-control cases.
Evidence reviewed

PR surface:

Source +6, Tests +5. Total +11 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 7 1 +6
Tests 1 5 0 +5
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 12 1 +11

What I checked:

Likely related people:

  • obviyus: GitHub commit metadata for the commit that introduced the current shared image factory in this checkout lists @obviyus as committer. (role: current-tree import committer; confidence: medium; commits: ecd29fe57269; files: src/image-generation/openai-compatible-image-provider.ts, src/plugin-sdk/image-generation.ts, extensions/deepinfra/image-generation-provider.ts)
  • Alix-007: Merged PR fix(agents): bound provider JSON response reads #95218 introduced readProviderJsonResponse, the helper used by this PR. (role: shared bounded helper contributor; confidence: high; commits: 2592f8a51a4e; files: src/agents/provider-http-errors.ts, src/agents/provider-http-errors.test.ts)
  • vincentkoc: Commit eb7a082 recently hardened image response schema handling in the same parser/factory area. (role: recent image hardening contributor; confidence: medium; commits: eb7a082b777d; files: src/image-generation/image-assets.ts, src/image-generation/openai-compatible-image-provider.ts, src/image-generation/image-assets.test.ts)
  • joshavant: Commit 0a14444 recently added successful-provider response bounded read helpers in the provider HTTP surface. (role: recent shared response-reader contributor; confidence: medium; commits: 0a14444924e3; files: src/agents/provider-http-errors.ts, src/plugin-sdk/provider-http.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. labels Jun 25, 2026
P1: Image responses carry base64-encoded image data that can exceed the
default 16 MiB provider JSON cap (4 images × ~7 MiB base64 each = ~28 MiB).
Pass a 64 MiB cap to readProviderJsonResponse.

P3: Use mode-dependent label for bounded-read errors, matching the pattern
already used by assertOkOrThrowHttpError in the same function.

Co-Authored-By: Claude <[email protected]>
@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Addressed both findings:

[P1] Image-sized JSON cap — Changed from default 16 MiB to 64 MiB (mode-dependent label + cap):

const parsedBody = await readProviderJsonResponse(
  response,
  mode === "edit"
    ? `${options.label} image edit`
    : `${options.label} image generation`,
  { maxBytes: 64 * 1024 * 1024 },
);

64 MiB covers worst-case base64 image payloads (4 images × ~7 MiB base64 ≈ ~28 MiB + headroom).

[P3] Mode-dependent label — Now uses "image edit" for edit mode, "image generation" for generate mode, matching the pattern already used by assertOkOrThrowHttpError in the same function.

Pre-flight: oxlint ✅, tsgo ✅, vitest 5/5 ✅
Body updated to 5-section Alix-007 template with new cap/proof values.

@clawsweeper

clawsweeper Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot added status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed proof: sufficient ClawSweeper judged the real behavior proof convincing. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jun 25, 2026
@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Upgraded to 🦞-grade proof with all 6 gaps addressed:

[P1] CLI/real-server product-path proof — Upgraded from 3 PASS to 6 PASS with BEFORE/AFTER/negative control:

  • AFTER (64 MiB cap): overflow capped ✅, happy path ✅, edit mode ✅, malformed JSON ✅
  • BEFORE (no cap): unbounded read consumed 30.0 MiB from same hostile server — OOM vector confirmed ✅
  • Negative control: unbounded .text() consumed 20.0 MiB vs capped stops at 64 MiB — cap is load-bearing ✅
  • 6/6 assertions total over real node:http server

[P1] Threat model — Now documents specific attack vectors: SSRF-redirected target, misconfigured proxy, CDN cache compromise
[Sibling coordination] — Positioned as companion to bounded-read campaign (#96772, #96782, etc.)
[AI disclosure] — Added "AI-assisted; reviewed before submission"

Pre-flight: vitest 5/5 ✅, oxlint ✅, tsgo ✅

@openclaw-barnacle openclaw-barnacle Bot added the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jun 25, 2026
@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Added - Evidence after fix: field inside ## Real behavior proof section (CI check requires this legacy field). All other sections unchanged.

@openclaw-barnacle openclaw-barnacle Bot removed the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jun 25, 2026
@clawsweeper

clawsweeper Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

P1 finding from the previous review: proof now exercises the full production chain (real HTTP server → readProviderJsonResponse → parseOpenAiCompatibleImageResponse), not just the helper in isolation.

7/7 assertions pass:

  • Happy path: valid image JSON returns 1 image with correct mime
  • Oversized (endless chunked stream, 64 MiB cap): throws bounded error at ~66.4 MiB
  • Negative control: unbounded path reads same under-cap body fine, capped path also succeeds

No code changes — proof-only update.

@openclaw-barnacle openclaw-barnacle Bot added the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jun 26, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 26, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. label Jun 26, 2026
@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

After studying Alix-007's #96893, I've come to understand that it covers the same bounded-read surface (image-generation OpenAI-compatible provider) more comprehensively — it patches 3 files (microsoft-foundry, openai, and the shared core) while this PR only covers 2.

One consideration for @Alix-007: image generation responses (base64-encoded images) can exceed the default 16 MiB readProviderJsonResponse cap. This PR used 64 MiB ({ maxBytes: 64 * 1024 * 1024 }) to accommodate large image payloads. You may want to consider whether the default 16 MiB is sufficient for your PR's scope.

Otherwise, your implementation is the better fix. Closing in favor of #96893.

@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

Closed in favor of #96893 (Alix-007)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: XS status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant