Skip to content

fix(provider-usage): bound usage JSON response reads#97659

Merged
vincentkoc merged 2 commits into
mainfrom
clownfish/pr-97610-provider-usage-bounded
Jun 29, 2026
Merged

fix(provider-usage): bound usage JSON response reads#97659
vincentkoc merged 2 commits into
mainfrom
clownfish/pr-97610-provider-usage-bounded

Conversation

@vincentkoc

@vincentkoc vincentkoc commented Jun 29, 2026

Copy link
Copy Markdown
Member

Replacement for #97610 because the fork rejected maintainer pushes while the submitted diff was incomplete.

Preserves the original shared usage-reader change and adds the missing bounded reads for MiniMax success responses and Claude usage error bodies.

Validation:

  • ./node_modules/.bin/oxlint --tsconfig config/tsconfig/oxlint.core.json src/infra/provider-usage.fetch.shared.ts src/infra/provider-usage.fetch.claude.ts src/infra/provider-usage.fetch.claude.test.ts src/infra/provider-usage.fetch.minimax.ts src/infra/provider-usage.fetch.minimax.test.ts
  • node scripts/test-projects-serial.mjs src/infra/provider-usage.fetch.shared.test.ts src/infra/provider-usage.fetch.claude.test.ts src/infra/provider-usage.fetch.minimax.test.ts

@openclaw-barnacle openclaw-barnacle Bot added size: S maintainer Maintainer-authored PR labels Jun 29, 2026
@clawsweeper

clawsweeper Bot commented Jun 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 29, 2026, 12:56 AM ET / 04:56 UTC.

Summary
The PR routes shared provider-usage JSON parsing and MiniMax success-body parsing through the existing capped provider JSON reader, reorders Claude’s import, and adds oversized-stream regression tests.

PR surface: Source +5, Tests +73. Total +78 across 5 files.

Reproducibility: yes. Current main and v2026.6.10 still use direct response.json()/res.json() in the shared provider usage and MiniMax success paths, and the bounded-reader contract shows why oversized streams are currently unbounded there.

Review metrics: 1 noteworthy metric.

  • Provider usage response cap: 1 existing 16 MiB cap applied to 2 success-read paths. The cap is the compatibility boundary maintainers should notice before changing oversized provider usage responses from unbounded parsing to fail-closed errors.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🌊 off-meta tidepool
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Risk before merge

  • [P1] Merging intentionally changes oversized provider usage JSON responses from unbounded parsing to capped fail-closed errors, so maintainers should accept the 16 MiB provider-usage contract.
  • [P1] There are two open landing paths for the shared helper, this PR and test(provider-usage): cover bounded shared response reads #96360, so maintainers should consolidate or explicitly choose one before merge.
  • [P1] Exact-head CI still had two in-progress jobs during review; merge should wait for required checks to finish green.

Maintainer options:

  1. Land the fuller branch after checks (recommended)
    Accept the existing 16 MiB provider JSON cap as the provider-usage contract, wait for exact-head CI, and close the overlapping shared-helper-only PR after landing this fuller branch.
  2. Tune the cap before merge
    If maintainers know legitimate provider usage payloads can exceed 16 MiB, adjust the cap or call-site option and update the oversized-response tests before landing.
  3. Fold the missing path elsewhere
    Alternatively, move the MiniMax success-read change and Claude regression coverage into the older proof-positive PR and close this branch after that replacement is ready.

Next step before merge

  • [P2] Protected maintainer PR with a compatibility cap, overlapping open PR, and remaining exact-head CI needs maintainer consolidation and approval rather than an automated repair job.

Security
Cleared: No security or supply-chain regression was found; the diff reduces a provider-response OOM/DoS class and does not change dependencies, workflows, secrets, or package resolution.

Review details

Best possible solution:

Land one consolidated provider-usage hardening PR that uses the shared bounded reader for all usage JSON success/error bodies, keeps focused oversized-stream regression tests, and treats the 16 MiB cap as an accepted provider usage contract.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main and v2026.6.10 still use direct response.json()/res.json() in the shared provider usage and MiniMax success paths, and the bounded-reader contract shows why oversized streams are currently unbounded there.

Is this the best way to solve the issue?

Mostly yes. The shared helper and MiniMax direct success read are the right fix layer, but the best merge path is to reconcile this fuller branch with the older shared-helper PR and have maintainers accept or tune the 16 MiB cap.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 75db48a17522.

Label changes

Label changes:

  • add P2: This is normal-priority provider usage hardening with limited blast radius and no evidence of an active production outage.
  • add merge-risk: 🚨 compatibility: The PR deliberately changes oversized provider usage JSON responses from potentially parseable unbounded bodies to capped fail-closed errors.

Label justifications:

  • P2: This is normal-priority provider usage hardening with limited blast radius and no evidence of an active production outage.
  • merge-risk: 🚨 compatibility: The PR deliberately changes oversized provider usage JSON responses from potentially parseable unbounded bodies to capped fail-closed errors.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: Contributor real-behavior proof is not required for this maintainer-labeled same-repo PR; the PR body still lists targeted oxlint and serial test commands as validation context.
Evidence reviewed

PR surface:

Source +5, Tests +73. Total +78 across 5 files.

View PR surface stats
Area Files Added Removed Net
Source 3 8 3 +5
Tests 2 80 7 +73
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 5 88 10 +78

What I checked:

  • Repository policy read: Root AGENTS.md was read fully, and scoped src/agents/AGENTS.md plus src/plugin-sdk/AGENTS.md were read because the PR reuses the agent bounded-reader helper and affects SDK-exported provider usage helpers. (AGENTS.md:12, 75db48a17522)
  • Current main shared read remains unbounded: Current main still parses shared provider usage success responses through direct response.json(), so providers using readUsageJson can buffer an oversized JSON body before this PR. (src/infra/provider-usage.fetch.shared.ts:70, 75db48a17522)
  • Current main MiniMax success read remains unbounded: Current main still parses MiniMax successful usage responses with direct res.json().catch(...), which is outside the shared helper fixed by the overlapping shared-only PR. (src/infra/provider-usage.fetch.minimax.ts:421, 75db48a17522)
  • Bounded reader contract: readProviderJsonResponse uses the existing 16 MiB provider JSON response cap, and readResponseWithLimit cancels the stream when the next chunk would exceed the limit before throwing. (src/agents/provider-http-errors.ts:312, 75db48a17522)
  • PR diff covers the missing reads: The PR diff replaces the shared helper's response.json() and MiniMax's direct res.json() with readProviderJsonResponse and adds oversized stream cancellation coverage for MiniMax success and Claude error responses. (src/infra/provider-usage.fetch.shared.ts:68, 39e1fc2a9ab4)
  • SDK exposure checked: Provider usage fetchers and shared helpers are exported through the plugin SDK provider-usage subpath, so the behavior change is visible to provider plugins even though the PR does not add or rename SDK exports. (src/plugin-sdk/provider-usage.ts:9, 75db48a17522)

Likely related people:

  • steipete: Peter Steinberger introduced the MiniMax usage snapshot and later deduplicated provider usage fetch logic into the shared helper structure touched by this PR. (role: feature-history owner; confidence: high; commits: 18b4575e4d, badafdc7b3, 14b4c7fd56; files: src/infra/provider-usage.fetch.minimax.ts, src/infra/provider-usage.fetch.shared.ts, src/infra/provider-usage.fetch.claude.ts)
  • cxbAsDev: Recent merged history bounded the Claude usage error response path, and the first commit in this PR preserves the earlier shared-reader hardening attempt from the closed source PR. (role: recent hardening contributor; confidence: high; commits: 615558f6fb3f, a411eed47aca; files: src/infra/provider-usage.fetch.claude.ts, src/infra/provider-usage.fetch.shared.ts)
  • IVY: Recent MiniMax usage parsing work changed the sibling normalization path that this PR now protects from oversized successful responses. (role: recent MiniMax usage contributor; confidence: medium; commits: efd5d5eb20, 30de4337bf, dd9c9dac53; files: src/infra/provider-usage.fetch.minimax.ts, src/infra/provider-usage.fetch.minimax.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

cxbAsDev and others added 2 commits June 28, 2026 21:46
@cxbAsDev @claude @vincentkoc
fix(provider-usage): bound usage JSON response reads to prevent OOM
a411eed 
Replace unbounded response.json() with readProviderJsonResponse in
readUsageJson to cap provider usage API response bodies at 16 MiB,
preventing unbounded buffering from all seven providers (claude,
gemini, codex, z.ai, deepseek, minimax, and openai-compatible).

Co-Authored-By: Claude <[email protected]>
@vincentkoc
fix(provider-usage): bound remaining usage JSON reads
39e1fc2
@vincentkoc
vincentkoc force-pushed the clownfish/pr-97610-provider-usage-bounded branch from afd614a to 39e1fc2 Compare June 29, 2026 04:46
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. labels Jun 29, 2026
@vincentkoc
vincentkoc merged commit db2786b into main Jun 29, 2026
126 of 130 checks passed
@vincentkoc
vincentkoc deleted the clownfish/pr-97610-provider-usage-bounded branch June 29, 2026 04:57
This was referenced Jun 29, 2026
Closed
Merged
This was referenced Jun 29, 2026
Open
Open
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

maintainer Maintainer-authored PR merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P2 Normal backlog priority with limited blast radius. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vincentkoc @cxbAsDev