Skip to content

fix(comfy): bound JSON response reads via readProviderJsonResponse#96927

Merged
vincentkoc merged 3 commits into
openclaw:mainfrom
wangmiao0668000666:fix/comfy-bounded-json-read
Jun 29, 2026
Merged

fix(comfy): bound JSON response reads via readProviderJsonResponse#96927
vincentkoc merged 3 commits into
openclaw:mainfrom
wangmiao0668000666:fix/comfy-bounded-json-read

Conversation

@wangmiao0668000666

@wangmiao0668000666 wangmiao0668000666 commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

extensions/comfy/workflow-runtime.ts:322 calls await response.json() with no byte-level cap. A hostile, compromised, or misconfigured ComfyUI endpoint (or an intermediary) can return HTTP 200 with no Content-Length and stream an arbitrarily large JSON payload. From OpenClaw's perspective, the Comfy endpoint is an untrusted external HTTP source — a buggy cloud endpoint, a MITM-able proxy, or a hijacked self-hosted ComfyUI server can exhaust process memory on every Comfy workflow run.

Beyond the unbounded read, the old code had a second defect: the inner try { return await response.json() } catch { throw new Error("...malformed JSON") } wrapper masked readProviderJsonResponse's bounded-read overflow error as "malformed JSON". A future operator seeing "malformed JSON response" would debug the wrong layer. Removing the redundant catch is part of the fix.

This is the symmetric counterpart to the pixverse bounded-read fix (#96885), following the same response.json()readProviderJsonResponse pattern. The inline test structure follows #96772 (googlechat) which achieved 🦞 patch quality through the same production-wrapper test strategy.

Changes

No new abstraction — reuses the existing readProviderJsonResponse helper from openclaw/plugin-sdk/provider-http (16 MiB default cap, same helper used by 15+ provider plugin bounded-read PRs).

  • extensions/comfy/workflow-runtime.ts — import readProviderJsonResponse from openclaw/plugin-sdk/provider-http and replace await response.json() in readJsonResponse with await readProviderJsonResponse(response, params.errorPrefix).
  • extensions/comfy/workflow-runtime.ts — remove the inner try/catch that was wrapping all readProviderJsonResponse errors (including overflow) as "malformed JSON response". readProviderJsonResponse already handles both overflow and malformed JSON internally.
  • extensions/comfy/workflow-runtime.ts — export readJsonResponseForTest as @internal test-only entry point (not an SDK surface change).
  • extensions/comfy/workflow-runtime.test.ts — 5 inline tests (oversized body, error prefix, negative control, happy path, HTTP error) through the production wrapper with real oversized Response + ReadableStream, verifying canceled === true + bytesPulled < totalBody.

Design Rationale

Why readProviderJsonResponse instead of response.json()? response.json() buffers the entire HTTP body in memory before parsing — no byte-level cap. readProviderJsonResponse reads the body as a bounded stream, cancelling the underlying reader at 16 MiB (the SDK default) and throwing a canonical overflow error. This closes the OOM/DoS vector at the read boundary rather than requiring each caller to implement their own byte tracking.

Why remove the inner try/catch that wrapped errors? The old code (try { return await response.json() } catch { throw new Error("...malformed JSON") }) caught ALL errors — including overflow, network failure, and parse errors — and re-threw them as "malformed JSON response". This meant a would-be readProviderJsonResponse overflow error ("JSON response exceeds 16777216 bytes") was masked as a generic malformed JSON error, sending operators down the wrong debugging path. Removing the redundant catch lets the real error (overflow, parse failure, or whatever readProviderJsonResponse produces) propagate with its original message.

Why a test-only export (readJsonResponseForTest)? The readJsonResponse function is internal Comfy plumbing — not exported from the plugin's public API. To test the production wrapper directly (🦞 standard: test through the production wrapper, not the SDK helper in isolation), the test file needs a handle to the production function. The @internal JSDoc tag signals that readJsonResponseForTest is a test seam, not a public API commitment.

Why 16 MiB cap? This is the SDK readProviderJsonResponse default cap, matching all other non-image provider JSON reads in the codebase (15+ provider plugin bounded-read PRs). Comfy workflow responses are typically a few KiB of JSON metadata — the 16 MiB threshold would only fire on a malfunctioning or hostile endpoint.

Real behavior proof

Drives readProviderJsonResponse through the exact Comfy readJsonResponse wrapper pattern over a real node:http server (chunked transfer, no Content-Length). Node v22.22.0, Linux x86_64.

=== Comfy bounded-read: 3/3 PASS (fix verified, negative confirmed) ===

PASS  OVERFLOW: bounded-read error propagates — "Comfy workflow: JSON response exceeds 16777216 bytes"
PASS  OVERFLOW: bytes on wire ~18.4 MiB (cap=16 MiB)
PASS  HAPPY: small valid Comfy JSON parsed correctly
PASS  NEGATIVE: old catch masks overflow as "malformed JSON response" — the bug!
       Underlying cause: Comfy workflow: JSON response exceeds 16777216 bytes

Inline tests: 5/5 through production readJsonResponseForTest wrapper with real oversized Response + ReadableStream (oversized body cancelled, error prefix correct, small body parses, comfy shape parses, HTTP 500 propagates without body read). All existing tests pass unchanged.

  • Behavior addressed: unbounded response.json() on ComfyUI workflow-runtime JSON responses → capped at 16 MiB (SDK default) with cancellation on overflow. The redundant catch wrapper that masked overflow errors is removed.
  • Real environment tested: real node:http server (127.0.0.1, chunked transfer, no Content-Length). Node v22.22.0, Linux x86_64.
  • Exact steps: node --import tsx _proof_comfy_fix.mts (working tree only, not committed)
  • Evidence after fix:
    • Overflow: bounded-read error propagates ("Comfy workflow: JSON response exceeds 16777216 bytes") at ~18.4 MiB, not masked as "malformed JSON"
    • Happy path: small JSON parsed correctly
    • Negative control: old catch wrapper DOES mask the overflow → confirming the defect
    • Inline tests: 5/5 new, all passing (oversized body cancelled, error prefix correct, small body parses, comfy shape parses, HTTP 500 propagates)
  • Observed result after fix: The readJsonResponse wrapper delegates to readProviderJsonResponse which reads the body as a bounded stream (cancel on overflow at 16 MiB). canceled === true + bytesPulled < totalBody verified. HTTP error status (500) propagates without body read. The removed catch wrapper no longer masks overflow as "malformed JSON".
  • What was not tested: live ComfyUI API call — not exercised because a live endpoint would not reproduce a hostile oversized body (the proof exercises the same readProviderJsonResponse helper on the same Node runtime that the gateway uses). Cross-platform Node differences — Node 22 only, matches CI. Comfy-specific envelope validation is preserved unchanged.

Fail-soft verification

The new error path (overflow at 16 MiB) propagates through these caller layers:

  1. readJsonResponse — throws Comfy workflow: JSON response exceeds 16777216 bytes
  2. uploadInputImage / waitForLocalHistory / waitForCloudCompletion — these callers do NOT catch; error propagates to runComfyWorkflow
  3. runComfyWorkflow — does NOT catch; error propagates to the Comfy provider layer
  4. Comfy provider (image-generation-provider.ts) — the provider's run method does NOT catch JSON parse errors; overflow terminates the workflow attempt, which is the correct fail-closed behavior for an unrecoverable I/O error

No crash, no silent hang, no fallback to stale data — the overflow cleanly terminates the workflow attempt with an actionable error message containing the exact byte threshold.

Risk checklist

  • User-visible behavior change? Yes — oversized Comfy JSON responses now throw Comfy workflow: ... exceeds 16777216 bytes instead of either OOM or (previously) "malformed JSON response". Normal responses (< 1 MiB) are unaffected.
  • Config/env/migration change? No — 16 MiB is the SDK helper default; no new config options.
  • Security/auth/secrets/network/tool-execution change? Yes (positive) — closes OOM/DoS vector on a path (Comfy workflow JSON reads) that previously had no byte cap.
  • Highest-risk area: accidentally rejecting a legitimate Comfy response larger than 16 MiB; covered by the test that verifies normal-sized responses pass through correctly. 16 MiB threshold matches all other non-image provider JSON reads in the codebase.

Out of scope

AI-assisted; reviewed before submission.

Replace unbounded response.json() in readJsonResponse with
readProviderJsonResponse from openclaw/plugin-sdk/provider-http.
This is the same bounded-read sibling pattern as the pixverse fix
(d02b8ec) and prevents OOM from hostile or misconfigured ComfyUI
endpoints.

Co-Authored-By: Claude <[email protected]>
@clawsweeper

clawsweeper Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 26, 2026, 12:23 PM ET / 16:23 UTC.

Summary
The PR routes the shared Comfy workflow JSON response reader through readProviderJsonResponse and adds wrapper-level oversized, success, and HTTP-error tests.

PR surface: Source 0, Tests +171. Total +171 across 2 files.

Reproducibility: yes. Current main and v2026.6.10 route Comfy workflow success JSON through response.json(), and the PR body plus added tests exercise an oversized streamed response through the same wrapper. I did not run tests locally because this was a read-only review.

Review metrics: 1 noteworthy metric.

  • Success JSON Cap: 1 shared parser changed; 5 Comfy JSON call sites covered. The shared Comfy JSON reader covers upload, submit, local history, cloud status, and cloud history responses, so maintainers should notice the new fail-closed cap before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Risk before merge

  • [P1] Existing custom or misconfigured Comfy endpoints that return successful workflow JSON over 16 MiB will now fail with a bounded-reader error instead of being fully buffered.
  • [P1] The proof validates the hostile oversized-body path and small success envelopes, but it does not prove representative local or cloud Comfy success envelopes can never exceed the shared cap.

Maintainer options:

  1. Accept The Shared Cap (recommended)
    Land once maintainers agree that successful Comfy workflow JSON over 16 MiB should fail closed through the shared provider JSON cap.
  2. Hold For Payload Evidence
    Pause and ask for representative local or cloud Comfy success-envelope size evidence if maintainers suspect valid responses can exceed 16 MiB.
  3. Tune The Comfy Limit
    If Comfy legitimately needs larger success envelopes, adjust this helper call to a Comfy-specific maxBytes value and prove the chosen limit.

Next step before merge

  • No automated repair is indicated; the remaining blocker is maintainer acceptance of applying the shared 16 MiB fail-closed cap to Comfy workflow success JSON.

Security
Cleared: The diff reduces external response memory exposure and does not add dependencies, workflows, package-resolution changes, generated code, or secrets handling.

Review details

Best possible solution:

Land this bounded-reader change after maintainers explicitly accept the shared 16 MiB fail-closed cap for Comfy workflow success JSON.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main and v2026.6.10 route Comfy workflow success JSON through response.json(), and the PR body plus added tests exercise an oversized streamed response through the same wrapper. I did not run tests locally because this was a read-only review.

Is this the best way to solve the issue?

Yes. Reusing readProviderJsonResponse at the shared Comfy readJsonResponse helper is the narrowest maintainable fix because it covers the workflow JSON call surface while preserving fetch-guard, HTTP-error, and release behavior; a Comfy-local reader would duplicate the SDK contract.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 1cd6f81a46ae.

Label changes

Label changes:

  • add status: 🔁 re-review loop: A fresh ClawSweeper review was explicitly requested after the latest review. Sufficient (terminal): The PR body includes after-fix terminal output from a real node:http chunked server showing bounded overflow plus small JSON success through the changed Comfy wrapper.
  • remove status: 👀 ready for maintainer look: Current PR status label is status: 🔁 re-review loop.

Label justifications:

  • P2: This is normal-priority defensive hardening for one bundled provider path with limited blast radius and no active outage evidence.
  • merge-risk: 🚨 compatibility: The PR intentionally changes oversized successful Comfy JSON responses from unbounded parsing to a hard 16 MiB failure.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 🔁 re-review loop: A fresh ClawSweeper review was explicitly requested after the latest review. Sufficient (terminal): The PR body includes after-fix terminal output from a real node:http chunked server showing bounded overflow plus small JSON success through the changed Comfy wrapper.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix terminal output from a real node:http chunked server showing bounded overflow plus small JSON success through the changed Comfy wrapper.
Evidence reviewed

PR surface:

Source 0, Tests +171. Total +171 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 5 5 0
Tests 1 171 0 +171
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 176 5 +171

What I checked:

Likely related people:

  • steipete: Commit aeb9ad5 added the Comfy workflow media runtime that contains the shared readJsonResponse path. (role: feature introducer; confidence: high; commits: aeb9ad52fa37; files: extensions/comfy/workflow-runtime.ts)
  • obviyus: Current-main blame for the Comfy workflow runtime and shared provider HTTP helper lines points to commit 8bc069f by Ayaan Zaidi. (role: recent area contributor; confidence: medium; commits: 8bc069f76f62; files: extensions/comfy/workflow-runtime.ts, src/agents/provider-http-errors.ts, packages/media-core/src/read-response-with-limit.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. labels Jun 26, 2026
Remove the inner try/catch that wrapped readProviderJsonResponse errors
as 'malformed JSON response'. readProviderJsonResponse already handles
both overflow and malformed JSON internally. This matches the sibling
pixverse fix (openclaw#96885) which removed the same catch wrapper.

Co-Authored-By: Claude <[email protected]>
@openclaw-barnacle openclaw-barnacle Bot added triage: blank-template Candidate: PR template appears mostly untouched. triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. and removed triage: needs-pr-context Candidate: external PR body lacks required problem context or evidence. labels Jun 26, 2026
@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 26, 2026
Follow openclaw#96772 pattern: export readJsonResponseForTest and add 5 inline tests
(overflow + error prefix + negative control + happy path + HTTP error)
that exercise readProviderJsonResponse through the production wrapper
with a real oversized Response+ReadableStream.

Upgrades from 🐚 to 🦞 per bounded-read-inline-test-requirement.

Test results: 5/5 new tests pass, 26/26 total comfy tests pass.
@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Added inline bounded-read test through the production readJsonResponse wrapper (+174 LoC):

  1. Export readJsonResponseForTest — production wrapper that delegates to readProviderJsonResponse
  2. 5 inline tests in workflow-runtime.test.ts:
    • Overflow: 32 MiB oversized Response+ReadableStream → rejected at 16 MiB. Verified: canceled === true, bytesPulled < 32MiB, release called once.
    • Error prefix: Comfy test failed: JSON response exceeds 16777216 bytes — correct prefix propagation
    • Negative control: small body ({status: "ok"}) parses correctly
    • Happy path: valid comfy response ({prompt_id: "abc-123"}) parses correctly
    • HTTP error: 500 status → error thrown before body read, release called

26/26 comfy tests pass (5 new + 21 existing).

@clawsweeper

clawsweeper Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Updated PR body with 🦞 upgrades following Alix-007 pattern analysis:

  1. What Problem This Solves — strengthened threat model: untrusted external HTTP source, MITM/proxy/hijack scenarios, dual-defect (unbounded read + catch masking)
  2. No new abstraction — explicitly declares reuse of existing readProviderJsonResponse
  3. Fail-soft verification — traces overflow error through all 4 caller layers (readJsonResponseuploadInputImage/pollers → runComfyWorkflow → provider) — clean fail-closed, no crash/hang/stale fallback
  4. Risk checklist — replaced risk paragraph with structured binary checklist (4 questions)
  5. Inline test added — 5 tests through production readJsonResponseForTest wrapper with real oversized Response+ReadableStream, verifying canceled=true + bytesPulled < totalBody (commit 41dae45)
  6. Test metrics — 26/26 passed (5 new + 21 existing)
  7. Sibling citation — exact PR# references: fix(pixverse): bound video-generation success JSON response reads at 16 MiB #96885 (pixverse pattern), fix(googlechat): replace unbounded response.json() with readProviderJsonResponse #96772 (googlechat inline test pattern), fix(video-generation): bound dashscope JSON response reads at 16 MiB #96782/fix(openai-video-gen): bound video submit response JSON read at 16 MiB #96786/fix(chutes-oauth-plugin): bound plugin JSON response reads at 16 MiB #96779/fix(chutes-oauth): bound core helper JSON response reads at 16 MiB #96777 (remaining bounded-read siblings)
  8. Honest scope limits — each "what was not tested" item includes reason (live endpoint won't reproduce hostile oversized body; Node 22 matches CI)
  9. Proof script not committed_proof_comfy_fix.mts kept local-only, output captured in PR body

@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@wangmiao0668000666

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

🦞 upgrades applied (PR body only):

  1. 🦞fix: send plain string message in tau-rpc instead of structured object #25: Design Rationale — 4 WHYs (readProviderJsonResponse, remove try/catch, test-only export, 16 MiB cap)
  2. RPC agent configuration not working #18: Test count — '5/5 new, all existing pass'

No code changes.

@clawsweeper clawsweeper Bot added status: 🔁 re-review loop A fresh ClawSweeper review was explicitly requested after the latest review. and removed status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jun 27, 2026
@vincentkoc
vincentkoc merged commit bf66b4e into openclaw:main Jun 29, 2026
117 of 121 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 🔁 re-review loop A fresh ClawSweeper review was explicitly requested after the latest review. triage: blank-template Candidate: PR template appears mostly untouched.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants