fix(voyage): bound embedding-batch status, error, and non-OK responses#96608
Conversation
|
Codex review: needs maintainer review before merge. Reviewed June 24, 2026, 11:22 PM ET / 03:22 UTC. Summary PR surface: Source +35, Tests +217. Total +252 across 2 files. Reproducibility: yes. Current main source clearly shows the Voyage status, error-file, and non-OK diagnostic paths use unbounded Review metrics: 1 noteworthy metric.
Stored data model Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Risk before merge
Maintainer options:
Next step before merge
Security Review detailsBest possible solution: Land the plugin-scoped bounded-reader fix after maintainers accept the 16 MiB compatibility tradeoff and required exact-head checks are green. Do we have a high-confidence way to reproduce the issue? Yes. Current main source clearly shows the Voyage status, error-file, and non-OK diagnostic paths use unbounded Is this the best way to solve the issue? Yes. Reusing the existing plugin SDK bounded response helpers inside the Voyage plugin is the narrow owner-boundary fix; the submit/upload helpers are already bounded and the successful output-file path remains streaming instead of being buffered. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 643410c1f3c0. Label changesLabel justifications:
Evidence reviewedPR surface: Source +35, Tests +217. Total +252 across 2 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
|
@clawsweeper review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
The batch status read (fetchVoyageBatchStatus) parsed its response with an
unbounded await res.json(), and the batch error-file read (readVoyageBatchError)
buffered the whole body via await res.text(). On top of that, the non-OK
(4xx/5xx) diagnostic body was still read unbounded: assertVoyageResponseOk did
await res.text() before throwing, and the non-OK output-file branch in
runVoyageEmbeddingBatches did the same. Voyage base URLs are user-supplied and
reachable via SSRF, so a misbehaving or hostile endpoint could stream an
unbounded body into memory on any of these paths before parsing.
Route the status JSON through the shared readProviderJsonResponse, the error
file through readResponseWithLimit, and now the non-OK diagnostic body through
readResponseWithLimit as well, all under a single 16 MiB cap, cancelling the
stream on overflow before decode/parse. assertVoyageResponseOk preserves its
original "${context}: ${status} ${text}" diagnostic shape for under-cap bodies
and throws a bounded "(error body exceeds <N> bytes)" on overflow; the non-OK
output-file branch now reuses it instead of a duplicate unbounded read. The
existing error-file fail-soft handling (formatUnavailableBatchError) is
preserved, so a capped endpoint degrades gracefully. The submit path already
bounds its body via postJsonWithRetry/maxResponseBytes and is left untouched.
Symmetric counterpart to the openclaw#96027/openclaw#96038 response-limit campaign.
ec26b41 to
dd3def7
Compare
|
Maintainer local review: merge-ready, reviewed as a batch:
Current CI red check is unrelated to this PR change Autoreview is clean, the 16 MiB cap is acceptable for these provider JSON/status/error/usage bodies, and the changes follow the established shared bounded-reader pattern. This is the best narrow fix for these call sites: no new SDK/API/config surface, no upgrade/backward-compatibility concern beyond the intended oversized-body failure mode, and no expected breaking behavior for normal provider responses. |
openclaw#96608) The batch status read (fetchVoyageBatchStatus) parsed its response with an unbounded await res.json(), and the batch error-file read (readVoyageBatchError) buffered the whole body via await res.text(). On top of that, the non-OK (4xx/5xx) diagnostic body was still read unbounded: assertVoyageResponseOk did await res.text() before throwing, and the non-OK output-file branch in runVoyageEmbeddingBatches did the same. Voyage base URLs are user-supplied and reachable via SSRF, so a misbehaving or hostile endpoint could stream an unbounded body into memory on any of these paths before parsing. Route the status JSON through the shared readProviderJsonResponse, the error file through readResponseWithLimit, and now the non-OK diagnostic body through readResponseWithLimit as well, all under a single 16 MiB cap, cancelling the stream on overflow before decode/parse. assertVoyageResponseOk preserves its original "${context}: ${status} ${text}" diagnostic shape for under-cap bodies and throws a bounded "(error body exceeds <N> bytes)" on overflow; the non-OK output-file branch now reuses it instead of a duplicate unbounded read. The existing error-file fail-soft handling (formatUnavailableBatchError) is preserved, so a capped endpoint degrades gracefully. The submit path already bounds its body via postJsonWithRetry/maxResponseBytes and is left untouched. Symmetric counterpart to the openclaw#96027/openclaw#96038 response-limit campaign.
openclaw#96608) The batch status read (fetchVoyageBatchStatus) parsed its response with an unbounded await res.json(), and the batch error-file read (readVoyageBatchError) buffered the whole body via await res.text(). On top of that, the non-OK (4xx/5xx) diagnostic body was still read unbounded: assertVoyageResponseOk did await res.text() before throwing, and the non-OK output-file branch in runVoyageEmbeddingBatches did the same. Voyage base URLs are user-supplied and reachable via SSRF, so a misbehaving or hostile endpoint could stream an unbounded body into memory on any of these paths before parsing. Route the status JSON through the shared readProviderJsonResponse, the error file through readResponseWithLimit, and now the non-OK diagnostic body through readResponseWithLimit as well, all under a single 16 MiB cap, cancelling the stream on overflow before decode/parse. assertVoyageResponseOk preserves its original "${context}: ${status} ${text}" diagnostic shape for under-cap bodies and throws a bounded "(error body exceeds <N> bytes)" on overflow; the non-OK output-file branch now reuses it instead of a duplicate unbounded read. The existing error-file fail-soft handling (formatUnavailableBatchError) is preserved, so a capped endpoint degrades gracefully. The submit path already bounds its body via postJsonWithRetry/maxResponseBytes and is left untouched. Symmetric counterpart to the openclaw#96027/openclaw#96038 response-limit campaign.
What Problem This Solves
extensions/voyage/embedding-batch.tsmakes several unbounded reads ofexternal, untrusted Voyage batch responses. A misconfigured / hostile /
SSRF-reachable endpoint (including a self-hosted
baseUrloverride) can return aContent-Length-less body that streams forever — and each unbounded read buffersthe entire payload before we can act on it, driving the batch worker into
memory pressure / OOM or a hang. Three reads are affected: the batch status
JSON, the batch error-file body, and the non-OK (4xx/5xx) diagnostic
body. This is the same untrusted-body surface the merged response-limit campaign
(#95218 / #96027 / #96038) closes across the other bundled providers.
The two success-body reads were bounded in the first revision of this PR. This
revision closes the last remaining gap: the non-OK diagnostic body.
assertVoyageResponseOkstill read the failure body with an unboundedawait res.text()before throwing (and a duplicate unboundedawait contentRes.text()existed on the non-OK output-file branch inrunVoyageEmbeddingBatches), so a hostile endpoint returning a 500 with anunbounded body was still buffered whole on the error path. Both non-OK reads are
now bounded too.
Changes
assertVoyageResponseOk): now read through theshared
readResponseWithLimit(16 MiB cap,media-corebounded reader,re-exported via
openclaw/plugin-sdk/response-limit-runtime) instead ofawait res.text(). The${context}: ${status} ${text}diagnostic is preservedbyte-for-byte for under-cap bodies; on overflow it throws a bounded
${context}: ${status} (error body exceeds <N> bytes)and cancels the stream.Cap is threaded from each caller so the boundary is testable.
runVoyageEmbeddingBatches): now routes throughthe same
assertVoyageResponseOk(samevoyage batch file content failed: <status> <text>message) instead of its own unboundedawait contentRes.text()— removing the last unbounded read in the file.readProviderJsonResponse, error file viareadResponseWithLimit. Theerror-file path still fail-softs by design — an oversized error body
degrades to
formatUnavailableBatchError(cap value included so truncation isvisible) rather than crashing the worker; the normal NDJSON
trim/split/JSON.parse/extractBatchErrorMessagechain and the empty-file branchare preserved.
media-corebounded reader already usedacross the codebase. Plugin-scoped.
Real behavior proof
oversized / never-ending body whole. The success-status read, the success
error-file read, and the non-OK diagnostic read must each stop at the cap,
cancel the stream, and either throw bounded or fail-soft — while valid small
bodies (status JSON, NDJSON error file, empty error file, small non-OK
diagnostic) still parse / format unchanged.
node:httpserver (createServer) bound to127.0.0.1, streaming bodies in 64 KiB chunks with noContent-Length(~64 MiB per overflow route, 4× the 16 MiB cap) over a real loopback socket,
on Node v22.22.0. The real exported
fetchVoyageBatchStatusandreadVoyageBatchError(testingexports of the real module) were driven againstit, with
withRemoteHttpResponseperforming a realfetch()and feeding thereal
Responseinto the realonResponse— so the productionassertVoyageResponseOk+readResponseWithLimit/readProviderJsonResponserun exactly as in production.
node --import tsx voyage-proof.mts— boot the server →drive the real functions against the streaming overflow routes and the small
routes → after each overflow, wait for the server-side socket-close event and
read the server's observed bytes-sent / abort counters → run a negative control
(raw unbounded read of the same non-OK route).
fetchVoyageBatchStatusthrewvoyage-batch-status: JSON response exceeds 16777216 bytes; the server saw thesocket aborted after ~17 MiB (≪ the ~64 MiB it would have sent) → stream
cancelled, not drained.
readVoyageBatchErrorfail-softedto
error file unavailable: voyage batch error file content exceeds 16777216 bytes(no throw out of the worker); socket aborted after ~17 MiB.500 drove
assertVoyageResponseOkto throw the boundedvoyage batch status failed: 500 (error body exceeds 16777216 bytes); socketaborted after ~18 MiB ≪ ~64 MiB. Before this fix the 500 body was buffered
whole by
await res.text().NDJSON error file still extracted the first error message
(
voyage upstream rejected); the empty error file returnedundefinedvia theempty-body branch; a small non-OK body preserved the original
voyage batch status failed: 503 voyage upstream is downdiagnostic.buffered past the 16 MiB cap (16,790,567 > 16,777,216 bytes) — proving the
cap is load-bearing (without it the body keeps accumulating).
api.voyageai.comendpoint (no key/ would not reproduce a hostile oversized body); the untrusted-body behavior is
fully reproduced with a local streaming server over a real socket, which is the
same transport path. The proof drives the bytes-on-wire + early-socket-close
signal (the load-bearing signal) rather than driving the worker to actual OOM.
The proof script is not committed.
Evidence
Real
node:httpterminal proof (real loopback sockets, real exported functions):In-repo Vitest suite (real
ReadableStreamfixtures, including the new non-OKoverflow + small non-OK diagnostic regression tests):
Checks on the changed files:
oxlint extensions/voyage/embedding-batch.ts extensions/voyage/embedding-batch.test.ts→ exit 0, clean;tsgo -p tsconfig.extensions.json→ exit 0, no errors.Label: security
AI-assisted.