Skip to content

test(clawrouter): avoid mocked oversized usage fetch#99860

Closed
mushuiyu886 wants to merge 3 commits into
openclaw:mainfrom
mushuiyu886:fix/followup-99605
Closed

test(clawrouter): avoid mocked oversized usage fetch#99860
mushuiyu886 wants to merge 3 commits into
openclaw:mainfrom
mushuiyu886:fix/followup-99605

Conversation

@mushuiyu886

Copy link
Copy Markdown
Contributor

Summary

  • Bound ClawRouter usage success responses with the shared readResponseWithLimit helper before JSON parsing.
  • Add a focused regression test for oversized successful usage payloads.
  • Preserve existing non-OK behavior: HTTP errors still report only the status code and do not expose upstream bodies.

What Problem This Solves

ClawRouter usage is an external provider usage boundary with a configurable baseUrl. The non-OK path already avoids exposing upstream bodies, but the successful path used bare response.json(), allowing a misconfigured or hostile usage endpoint to make OpenClaw buffer an unbounded JSON body before extracting a tiny budget summary.

User Impact

  • Why it matters / User impact: A ClawRouter-compatible endpoint can return an arbitrarily large successful /v1/usage response. Before this patch, a provider usage refresh could spend memory on the full body even though the UI only needs budget and summary fields.
  • User-visible behavior after fix: Normal ClawRouter budget summaries still render the same way, while oversized success bodies fail fast with a bounded-read error.

Origin / follow-up

  • Follows: fix(google): bound OAuth token error response reads #99605, which bounded Google OAuth token error response reads.
  • Gap this fills: The same bounded-read pattern had not been applied to ClawRouter usage successful responses; extensions/clawrouter/usage.ts still used bare response.json() on the success path.
  • Consistency: This uses the shared plugin SDK readResponseWithLimit wrapper, matching the existing extension/provider bounded-response pattern rather than adding a ClawRouter-only reader.

Competition / linked PR analysis

  • Linked issue: N/A — this is a follow-up to a recently merged bounded-read PR, not a closing-issue PR.
  • Related open PR scan: Searched open PRs for clawrouter usage response limit OR bounded read OR response.json; result was [] in open-pr-conflict-search.json.
  • Competition assessment: No open PR was found for this ClawRouter usage success-response gap.

Real behavior proof

  • Behavior or issue addressed: ClawRouter usage success responses are now read through a 1 MiB cap before JSON parsing, so an oversized successful /v1/usage body is rejected instead of being fully parsed by response.json().
  • Real environment tested: Local Node HTTP server on 127.0.0.1 with production fetchClawRouterUsage, real fetch, and configurable baseUrl; no mock fetch and no external credentials.
  • Exact steps or command run after this patch:
source /media/vdb/code/ai/aispace/openclaw-followup-99605-evidence/context.env
cd "$TASK_WORKTREE"
TASK_WORKTREE="$TASK_WORKTREE" node --import tsx "$EVIDENCE_DIR/clawrouter-local-http-proof.mjs"
node scripts/run-vitest.mjs extensions/clawrouter/usage.test.ts
pnpm lint:extensions
pnpm test:extensions:package-boundary
pnpm exec oxfmt --check extensions/clawrouter/usage.ts extensions/clawrouter/usage.test.ts
git diff --check
  • Evidence after fix:
HEAD=fd5e8db719073f0a462c8f7d23f8fb39074c1e3b
baseUrl=http://127.0.0.1:40871
oversizedResponseBytes=1048659
oversizedResult=ClawRouter usage response exceeds 1048576 bytes
normalResult={"provider":"clawrouter","displayName":"ClawRouter","windows":[{"label":"Monthly budget","usedPercent":25,"resetAt":1785542400000}],"summary":"12 requests · 34,567 tokens · $25.00 used","plan":"Managed monthly budget"}
Test Files  1 passed (1)
Tests  4 passed (4)
[test] passed 1 Vitest shard in 14.87s
extension package boundary check passed
mode: all
compiled plugins: 117
canary plugins: 2
Checking formatting...
All matched files use the correct format.
Finished in 108ms on 2 files using 8 threads.
  • Observed result after fix: The oversized successful usage response rejects at 1048576 bytes, and a normal managed monthly budget response still maps to the same ProviderUsageSnapshot fields.
  • What was not tested: No live ClawRouter service account was used; the defect is the client-side bounded read of an untrusted usage HTTP response, and the after-fix proof exercises that boundary with a local HTTP server and real fetch.
  • Fix classification: Root cause fix.

Review findings addressed

  • N/A — this is a new follow-up PR with no existing review findings.

Regression Test Plan

  • Target test file: extensions/clawrouter/usage.test.ts
  • Scenario locked in: A successful /v1/usage response larger than 1 MiB rejects before JSON parsing, while a normal managed monthly budget response still maps to the expected ProviderUsageSnapshot.
  • Why this is the smallest reliable guardrail: The regression test covers the previously unbounded success path, and the local HTTP proof uses production fetchClawRouterUsage with real fetch so the body-size guard is exercised at the HTTP response boundary.
  • node scripts/run-vitest.mjs extensions/clawrouter/usage.test.ts
  • pnpm lint:extensions
  • pnpm test:extensions:package-boundary
  • pnpm exec oxfmt --check extensions/clawrouter/usage.ts extensions/clawrouter/usage.test.ts
  • git diff --check

Risk / Compatibility

  • Risk labels considered: provider-usage, external HTTP boundary, availability/resource-exhaustion.
  • Risk explanation: The success body cap can reject a ClawRouter-compatible endpoint that returns more than 1 MiB from /v1/usage, but expected usage payloads contain only budget and summary fields and are far smaller than the cap.
  • Why acceptable: The configurable baseUrl makes the response untrusted; failing fast on an oversized usage body is safer than buffering unbounded JSON, and normal response mapping is covered by both the regression test and local HTTP proof.
  • Patch quality notes: The test follows the existing ClawRouter test style by passing a typed fetchFn dependency; this is the function's public seam for exercising success and error response paths without changing production API shape.

Merge risk

Low. The diff is limited to ClawRouter usage parsing and one focused regression test. The only behavior change is rejecting unexpectedly large successful usage JSON before parsing.

What was not changed

  • What did NOT change: ClawRouter URL normalization, authorization headers, provider catalog behavior, streaming/chat behavior, non-OK error body redaction, and normal usage snapshot formatting.
  • Scope boundary: This patch only changes the ClawRouter usage success-response read path and its regression coverage; it does not alter provider catalog fetching, chat completion streaming, auth setup, or any shared response-limit helper behavior.

Architecture / source-of-truth check

  • Architecture / source-of-truth check: fetchClawRouterUsage is the source-of-truth owner for ClawRouter usage snapshots. The fix is applied at that external HTTP response boundary before JSON parsing, not downstream in UI formatting or provider catalog code.

Root Cause

  • Root cause: The ClawRouter usage success path trusted response.json() to consume /v1/usage, so a configurable provider usage endpoint could return an unbounded successful JSON body.
  • Why this is root-cause fix: The patch replaces the unbounded body read at the owner boundary with the shared capped reader before JSON parsing, while preserving the existing snapshot mapping and non-OK behavior.

Maintainer-ready confidence

  • Maintainer-ready confidence: High. The patch is small, follows an existing bounded-read pattern, includes a regression test, and has after-fix proof through a real local HTTP response path using production code.

@clawsweeper

clawsweeper Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 4, 2026, 3:05 AM ET / 07:05 UTC.

Summary
The PR changes ClawRouter usage success parsing to read /v1/usage bodies through readResponseWithLimit with a 1 MiB cap and adds an oversized-body regression test.

PR surface: Source +16, Tests +19. Total +35 across 2 files.

Reproducibility: yes. from source and submitted terminal proof. Current main uses response.json() on successful ClawRouter usage responses, and the PR body shows production fetchClawRouterUsage rejecting an oversized local HTTP response after the fix.

Review metrics: 1 noteworthy metric.

  • Usage Response Cap: 1 added: 1 MiB successful ClawRouter /v1/usage body cap. Maintainers should notice the intentional compatibility boundary because endpoints over the cap now surface a usage error instead of being parsed.

Stored data model
Persistent data-model change detected: serialized state: extensions/clawrouter/usage.test.ts, serialized state: extensions/clawrouter/usage.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

  • [P1] The 1 MiB cap intentionally changes behavior for any ClawRouter-compatible /v1/usage endpoint that currently returns a larger successful body; maintainers should accept or adjust that cap before merge.

Maintainer options:

  1. Accept The 1 MiB Usage Cap (recommended)
    Land the focused hardening after explicitly accepting that oversized successful ClawRouter usage responses now fail as bounded provider usage errors.
  2. Raise Or Align The Cap
    If real ClawRouter-compatible usage payloads can exceed 1 MiB, adjust this cap or align it with the broader provider JSON budget before merge.
  3. Pause For Usage Contract Confirmation
    If the expected /v1/usage response size is unsettled, pause until the ClawRouter usage contract names an acceptable maximum body size.

Next step before merge

  • [P2] Maintainers need to accept or adjust the 1 MiB compatibility cap; there is no concrete automated repair needed.

Security
Cleared: The diff reduces external HTTP response materialization risk and does not add dependencies, workflows, permissions, package metadata, or secret-handling changes.

Review details

Best possible solution:

Land the owner-boundary bounded read after maintainer acceptance of the 1 MiB ClawRouter usage cap, or adjust the cap first if real usage payloads can exceed it.

Do we have a high-confidence way to reproduce the issue?

Yes, from source and submitted terminal proof. Current main uses response.json() on successful ClawRouter usage responses, and the PR body shows production fetchClawRouterUsage rejecting an oversized local HTTP response after the fix.

Is this the best way to solve the issue?

Yes, after maintainer acceptance of the cap. The ClawRouter plugin owns this usage boundary, and reading through the existing bounded response helper before JSON parsing is the narrow maintainable fix.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against f48ff25b3be3.

Label changes

Label justifications:

  • P2: This is a focused ClawRouter provider-usage hardening fix with limited blast radius and no evidence of an active outage.
  • merge-risk: 🚨 compatibility: The PR can change behavior for existing ClawRouter-compatible usage endpoints that return successful bodies larger than 1 MiB.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body provides terminal proof from a local HTTP server using production fetchClawRouterUsage and real fetch, showing oversized rejection and normal usage mapping after the fix.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body provides terminal proof from a local HTTP server using production fetchClawRouterUsage and real fetch, showing oversized rejection and normal usage mapping after the fix.
Evidence reviewed

PR surface:

Source +16, Tests +19. Total +35 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 17 1 +16
Tests 1 19 0 +19
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 36 1 +35

What I checked:

  • Root policy read: Root AGENTS.md was read fully and applied to the plugin-boundary, review-depth, proof, and compatibility-risk checks. (AGENTS.md:1, f48ff25b3be3)
  • Scoped extension policy read: The extension guide allows bundled plugin production code to import public openclaw/plugin-sdk/* subpaths, which matches this PR's response-limit import. (extensions/AGENTS.md:1, f48ff25b3be3)
  • Current main behavior: Current main still parses successful ClawRouter usage responses with bare response.json(), so the central unbounded successful-body read remains present. (extensions/clawrouter/usage.ts:81, f48ff25b3be3)
  • Runtime entry point: The ClawRouter provider hook calls fetchClawRouterUsage with the configured base URL, timeout, token, and runtime fetch function. (extensions/clawrouter/index.ts:217, f48ff25b3be3)
  • Shared helper contract: readResponseWithLimit reads under a byte cap, cancels on overflow, and throws the caller-provided overflow error. (packages/media-core/src/read-response-with-limit.ts:129, f48ff25b3be3)
  • Plugin SDK export: openclaw/plugin-sdk/response-limit-runtime is a public package export, so the PR does not cross into core internals from the bundled plugin. (package.json:980, f48ff25b3be3)

Likely related people:

  • steipete: Peter Steinberger authored the current ClawRouter routing and quotas feature that added the usage files, then made a follow-up ClawRouter auth-profile fix. (role: feature owner and recent area contributor; confidence: high; commits: 4a354f76c1bf, 8250b8d02eb6; files: extensions/clawrouter/usage.ts, extensions/clawrouter/usage.test.ts, extensions/clawrouter/index.ts)
  • vincentkoc: Vincent Koc authored earlier ClawRouter managed-proxy work in the same provider area before the current reintroduced quotas implementation. (role: earlier adjacent ClawRouter provider contributor; confidence: medium; commits: 95dafc824edb; files: extensions/clawrouter/index.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. labels Jul 4, 2026
@steipete steipete self-assigned this Jul 5, 2026
@steipete

steipete commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Thank you for the contribution. This fix landed with contributor co-authorship preserved in #100258 (deac98eb7204fdb51589c5e369b95be128215e77).

I consolidated the source fixes into a maintainer takeover because the contributor branches were based on rewritten pre-main history; updating them directly would have pulled unrelated changes into the review surface. Closing this PR as superseded by the landed batch.

@steipete steipete closed this Jul 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

extensions: clawrouter Extension: clawrouter merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. size: XS status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants