Skip to content

fix(openrouter): bound video catalog JSON reads#96505

Merged
sallyom merged 1 commit into
openclaw:mainfrom
mushuiyu886:fix/followup-95420
Jun 25, 2026
Merged

fix(openrouter): bound video catalog JSON reads#96505
sallyom merged 1 commit into
openclaw:mainfrom
mushuiyu886:fix/followup-95420

Conversation

@mushuiyu886

Copy link
Copy Markdown
Contributor

Summary

  • Bound OpenRouter video model catalog success JSON reads through the shared provider JSON reader.
  • Add coverage that exercises the catalog path with real Response bodies, including an oversized stream that must be canceled.

What Problem This Solves

extensions/openrouter/video-model-catalog.ts fetched /videos/models and parsed successful responses with response.json(). That bypassed the provider response byte cap, so a provider endpoint or user-configured OpenRouter-compatible baseUrl could stream an oversized JSON success body and make the runtime buffer it before parsing.

User Impact

  • Why it matters / User impact: Users who enable OpenRouter video model discovery keep the same catalog behavior for normal responses. Oversized or runaway catalog success bodies now fail with a bounded provider error instead of being read without a byte limit.
  • What did NOT change: The patch only changes the OpenRouter video model catalog success-body parser. It does not change auth headers, provider routing, catalog caching, model projection, API key resolution, or other OpenRouter media response paths.
  • Architecture / source-of-truth check: The canonical source-of-truth contract for provider JSON body limits is readProviderJsonResponse, which wraps readResponseWithLimit with the shared 16 MiB cap and overflow cancellation. This extension now uses that SDK contract instead of a one-off parser.

Origin / follow-up

Follow-up to #95420. That PR bounded a sibling OpenRouter catalog response path; this patch applies the same shared provider JSON reader to the OpenRouter video model catalog path.

Competition / linked PR analysis

Related open PR scan: checked for an open PR covering an OpenRouter video model catalog bounded response reader and did not find one. This patch is intentionally limited to extensions/openrouter/video-model-catalog.ts and its existing provider test so it does not overlap with open bounded-reader work in other extensions.

Real behavior proof

  • Behavior or issue addressed: The OpenRouter video model catalog success path now reads JSON through readProviderJsonResponse, which enforces the shared 16 MiB cap and cancels the stream on overflow.
  • Real environment tested: Local OpenClaw worktree on Node v22.22.0 / pnpm v11.2.2, using real Web Response and ReadableStream bodies. No OpenRouter credential was used; the proof targets the response-body boundary after the HTTP result is returned.
  • Exact steps or command run after this patch:
node scripts/run-vitest.mjs extensions/openrouter/video-generation-provider.test.ts
pnpm exec tsx <evidence-only bounded-reader proof script>
  • Evidence after fix:
{
  "proof": "openrouter-video-catalog-bounded-json-reader",
  "boundary": "real Response and ReadableStream body without Content-Length",
  "reader": "src/agents/provider-http-errors.ts readProviderJsonResponse",
  "caller": "extensions/openrouter/video-model-catalog.ts OpenRouter video model catalog success path",
  "normalPathModel": "openrouter/video-ok",
  "overflowMessage": "OpenRouter video models request failed: JSON response exceeds 16777216 bytes",
  "expectedMessage": "OpenRouter video models request failed: JSON response exceeds 16777216 bytes",
  "streamCanceled": true
}
Test Files  1 passed (1)
Tests  17 passed (17)
[test] passed 1 Vitest shard
  • Observed result after fix: Normal catalog JSON still parses, while a no-Content-Length ReadableStream that exceeds 16 MiB throws OpenRouter video models request failed: JSON response exceeds 16777216 bytes; the stream cancellation hook ran, and the guarded fetch release callback was called.
  • What was not tested: A live OpenRouter /videos/models request was not run because the current environment does not provide an OpenRouter API key. Other OpenRouter response paths, including audio transcription, were not changed in this patch.
  • Fix classification: Root cause fix

Review findings addressed

No maintainer review findings are attached to this follow-up yet. The patch addresses the bounded-reader gap found while comparing the already-merged OpenRouter catalog fix with remaining OpenRouter video catalog response reads.

Regression Test Plan

  • Target test file: extensions/openrouter/video-generation-provider.test.ts
  • Scenario locked in: The catalog path receives a successful OpenRouter video model response backed by a real Web Response; normal JSON still maps into catalog entries, and an oversized no-Content-Length stream fails at the shared 16 MiB cap.
  • Why this is the smallest reliable guardrail: The test calls the exported listOpenRouterVideoModelCatalog path rather than only testing the helper, and asserts the overflow stream is canceled and the guarded fetch release callback runs. This directly guards the changed parser call site without adding a broader provider fixture.
node scripts/run-vitest.mjs extensions/openrouter/video-generation-provider.test.ts
pnpm exec tsx <evidence-only bounded-reader proof script>

Merge risk

  • Risk labels considered: no broad merge-risk labels apply; this is a two-file provider-boundary patch with one production import/call-site change and one focused provider test update.
  • Risk explanation: The only behavior change is for successful OpenRouter video model catalog bodies that exceed the shared provider JSON cap or contain malformed JSON. Valid responses under the cap still parse and project through the same catalog code.
  • Why acceptable: The new behavior matches the existing provider JSON reader contract used by sibling provider paths and prevents an unbounded external response body from being buffered.

Risk / Compatibility

Low risk. Valid JSON responses under the shared provider cap follow the same parsing and catalog projection path as before. Responses over the cap now fail earlier with the same provider-reader error shape used by other provider JSON responses.

What was not changed

  • OpenRouter authentication and headers were not changed.
  • Catalog caching and model projection were not changed.
  • Other OpenRouter media or transcription response readers were not changed.
  • The shared 16 MiB provider JSON cap was not changed.

Maintainer-ready confidence

  • Maintainer-ready confidence: High. The diff is small, follows an existing shared helper contract, and has fresh runtime proof plus the targeted provider test.
  • Patch quality notes: The patch does not add a new abstraction or provider-specific limit; it reuses readProviderJsonResponse from openclaw/plugin-sdk/provider-http, keeping extension code on the existing SDK boundary.

Root Cause

  • Root cause: The root cause was that the OpenRouter video model catalog success path used the platform response.json() parser directly at the external response boundary, bypassing the shared provider JSON reader contract that is responsible for bounded response parsing.
  • Why this is root-cause fix: This fixes the source parser invariant because the unbounded read happened at the exact success-body parse site after assertOkOrThrowHttpError. Replacing that call with readProviderJsonResponse moves the byte cap and cancellation behavior before JSON parsing can buffer an oversized body, while leaving endpoint selection, auth, caching, and catalog projection unchanged.

@clawsweeper

clawsweeper Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 24, 2026, 1:00 PM ET / 17:00 UTC.

Summary
The PR replaces the OpenRouter video model catalog success response.json() call with readProviderJsonResponse and adds focused normal/oversized Response coverage.

PR surface: Source +4, Tests +55. Total +59 across 2 files.

Reproducibility: yes. Source inspection shows current main and v2026.6.10 still call response.json() on the successful OpenRouter video catalog body, and the PR body includes terminal proof for the bounded replacement path.

Review metrics: 2 noteworthy metrics.

  • Bounded Response Surface: 1 success JSON read changed. The only runtime behavior change is enforcing the shared provider JSON cap on OpenRouter video catalog discovery.
  • Overlapping Open Fixes: 1 overlapping open PR. Maintainers should choose one landing path or preserve this PR's direct regression test without merging duplicate branches independently.

Stored data model
Persistent data-model change detected: serialized state: extensions/openrouter/video-generation-provider.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Risk before merge

  • [P1] Custom or unusually large OpenRouter-compatible video catalogs that previously parsed if memory allowed will now fail once the shared 16 MiB provider JSON cap is exceeded.
  • [P1] fix(openrouter): bound video model catalog JSON response reads #96500 changes the same production call site, so maintainers should choose one landing path or combine this PR's direct test coverage with the other branch.

Maintainer options:

  1. Accept The Shared Provider Cap (recommended)
    Maintainers can intentionally accept that OpenRouter video catalogs over the shared 16 MiB provider JSON cap now fail fast instead of allowing an unbounded provider-controlled read.
  2. Preserve Direct Changed-Path Coverage
    If maintainers choose fix(openrouter): bound video model catalog JSON response reads #96500 as the landing branch, port this PR's direct overflow test before closing this branch.
  3. Pause For Catalog Limit Policy
    If 16 MiB is not the desired limit for video catalog discovery, pause both overlapping PRs until the provider catalog cap policy is decided.

Next step before merge

  • [P2] Maintainer review should choose between this branch and the overlapping open PR and accept the 16 MiB compatibility tradeoff; there is no narrow automated repair needed.

Security
Cleared: The diff reduces provider-controlled response-body risk by using the existing bounded SDK reader and adds no dependency, workflow, credential, or supply-chain surface.

Review details

Best possible solution:

Land one bounded-reader fix for this call site, preserve direct overflow coverage, accept the shared 16 MiB cap, and then close or supersede the alternate branch.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main and v2026.6.10 still call response.json() on the successful OpenRouter video catalog body, and the PR body includes terminal proof for the bounded replacement path.

Is this the best way to solve the issue?

Yes. Reusing readProviderJsonResponse at the exact success-body parse site is the narrowest owner-boundary fix; the remaining decision is which overlapping branch should land and whether to preserve this direct test.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 4ae0a5d958a0.

Label changes

Label justifications:

  • P2: This is a normal-priority provider hardening fix with limited blast radius to OpenRouter video catalog discovery.
  • merge-risk: 🚨 compatibility: The new shared JSON cap can intentionally reject oversized custom OpenRouter-compatible video catalogs that current users might previously load.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes terminal proof using real Web Response and ReadableStream bodies plus copied test output showing normal parsing, bounded overflow, stream cancellation, and release callback behavior.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes terminal proof using real Web Response and ReadableStream bodies plus copied test output showing normal parsing, bounded overflow, stream cancellation, and release callback behavior.
Evidence reviewed

PR surface:

Source +4, Tests +55. Total +59 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 5 1 +4
Tests 1 57 2 +55
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 62 3 +59

What I checked:

Likely related people:

  • steipete: Commit history shows 311e4608d17326965012881def74c25bfa161cd8 added extensions/openrouter/video-model-catalog.ts, including the catalog fetch path now being hardened. (role: introduced current catalog file; confidence: high; commits: 311e4608d173; files: extensions/openrouter/video-model-catalog.ts)
  • vincentkoc: Commit history shows 4291e3277720b265720671fcc3ab20587c220d11 refactored shared OpenRouter video mode capability handling in the same module. (role: recent area contributor; confidence: medium; commits: 4291e3277720; files: extensions/openrouter/video-model-catalog.ts)
  • Alix-007: History shows this person authored the shared bounded provider JSON helper work and merged sibling OpenRouter bounded-catalog fixes that this PR follows. (role: adjacent hardening contributor; confidence: high; commits: 2592f8a51a4e, 3da4280caf4c, 06ca1235efb7; files: src/agents/provider-http-errors.ts, src/agents/embedded-agent-runner/openrouter-model-capabilities.ts, src/agents/model-scan.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. labels Jun 24, 2026
@mushuiyu886

Copy link
Copy Markdown
Contributor Author

hi @sallyom there is a bounded response pr ready to merge, I'd really appreciate it if you could take a look!

@sallyom sallyom self-assigned this Jun 25, 2026
@sallyom

sallyom commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Merge-ready from maintainer review.

This is the best narrow fix for the OpenRouter video catalog success-body read: it changes the exact response.json() call site to the shared bounded provider JSON reader, keeps the existing OK-status and release paths, and adds direct coverage for normal catalog parsing plus oversized stream cancellation.

Autoreview is clean with no accepted/actionable findings. The targeted OpenRouter provider test passes locally, and current CI is green. I do not see a breaking-change or compatibility concern for supported OpenRouter video catalog behavior; the only changed behavior is the intended fail-fast cap for oversized/runaway catalog success bodies using the existing shared provider JSON limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

extensions: openrouter merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants