Skip to content

fix: release abandoned provider streams#88500

Merged
steipete merged 5 commits into
openclaw:mainfrom
samzong:fix/gateway-undici-socket-leak
May 31, 2026
Merged

fix: release abandoned provider streams#88500
steipete merged 5 commits into
openclaw:mainfrom
samzong:fix/gateway-undici-socket-leak

Conversation

@samzong

@samzong samzong commented May 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #67461

  • Release guarded provider fetch slots when a streamed response body is abandoned before normal completion or cancel.
  • Keep the existing completion/cancel release path idempotent, and unregister normal releases from the cleanup registry.
  • Add regression coverage for abandoned managed stream cleanup.

Verification

  • git diff --check -> passed.
  • node scripts/run-vitest.mjs src/agents/provider-transport-fetch.test.ts -> passed, 67 tests.
  • .agents/skills/autoreview/scripts/autoreview --mode local -> clean, no accepted/actionable findings.
  • Real stream harness after rebase: node --expose-gc --import tsx --input-type=module with a local HTTP streaming server, buildGuardedModelFetch, first-chunk read, abandoned Response/reader, forced GC loop -> {"requestReceived":true,"firstChunkRead":true,"socketClosed":true,"writes":3}.

Real behavior proof

  • Behavior addressed: abandoned provider streaming responses now release the guarded fetch slot and close the underlying streamed HTTP socket instead of waiting forever for a body that no caller is reading.

  • Real environment tested: local Node 24.14.0 process using the actual buildGuardedModelFetch source path, actual fetch/ReadableStream, and a real local TCP HTTP streaming server on 127.0.0.1.

  • Exact steps or command run after this patch: ran node --expose-gc --import tsx --input-type=module with a local streaming HTTP server, called buildGuardedModelFetch(model, 5000, { sanitizeSse: false }), read the first response body chunk, dropped the Response and body reader, then forced GC until the server observed socket close.

  • Evidence after fix: Console output from the real local stream harness:

    {"requestReceived":true,"firstChunkRead":true,"socketClosed":true,"writes":3}
  • Observed result after fix: The copied live output shows the server received the model request, the client read the first streamed chunk, and the server observed the socket close after the abandoned managed response was garbage-collected.

  • What was not tested: live remote provider credentials, full Gateway agent run, and cross-OS/Testbox proof.

Performance accounting

No latency, throughput, or speedup claim. This moves abandoned-stream cleanup off a leak path; the runtime change adds one FinalizationRegistry registration per managed streamed response and unregisters it on normal release.

AI-assisted

This PR was prepared with Codex.

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: S triage: blank-template Candidate: PR template appears mostly untouched. triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 31, 2026
@clawsweeper

clawsweeper Bot commented May 31, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed May 31, 2026, 12:11 PM ET / 16:11 UTC.

Summary
The branch adds fallback cleanup for abandoned managed provider response bodies, cancels Anthropic and Google SSE readers on early parser exit, and updates focused regression plus live-shard coverage.

PR surface: Source +23, Tests +163, Other +1. Total +187 across 9 files.

Reproducibility: yes. source-level: current main only releases managed provider responses on normal end, read error, or explicit cancel, and the linked report provides production lsof evidence of leaked sockets. I did not run a current-main multi-hour Gateway soak in this read-only review.

Review metrics: 1 noteworthy metric.

  • Abandoned-stream proof scope: 1 local TCP harness, 0 live remote-provider soaks. The fix targets file descriptor exhaustion, so maintainers should decide whether local GC/socket proof is enough before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Maintainer should decide whether to accept the local TCP/GC proof or request a Gateway or live-provider socket/FD soak before landing.

Risk before merge

  • [P1] FinalizationRegistry cleanup for truly abandoned wrappers is GC-timed, so maintainers may still want a longer Gateway or remote-provider socket/FD soak before landing an availability-sensitive fix.
  • [P1] The deterministic cancellation additions cover Anthropic and Google error/parser-exit paths, but the supplied proof does not include live remote provider credentials or a full Gateway agent run.

Maintainer options:

  1. Accept Current Proof
    Maintainers can land the focused cleanup if the local TCP stream harness and regression tests are enough for this P1 leak.
  2. Require Soak Proof
    Ask for a Gateway or live-provider socket/FD soak if GC-timed finalization needs production-confidence evidence before merge.
  3. Pause For Lifecycle Redesign
    Pause the PR if maintainers prefer an explicit timeout or consumer-owned cancellation design over a FinalizationRegistry fallback.

Next step before merge

  • No automated repair is indicated because the branch already contains the focused fix; the remaining action is maintainer landing or proof-depth judgment.

Security
Cleared: No concrete security or supply-chain regression was found; the diff does not change dependencies, lockfiles, workflows, secrets, or SSRF policy.

Review details

Best possible solution:

Provider streams should cancel on parser exit and release guarded slots on abandoned wrappers without adding provider APIs, config, or fallback policy.

Do we have a high-confidence way to reproduce the issue?

Yes, source-level: current main only releases managed provider responses on normal end, read error, or explicit cancel, and the linked report provides production lsof evidence of leaked sockets. I did not run a current-main multi-hour Gateway soak in this read-only review.

Is this the best way to solve the issue?

Yes, with a maintainer-proof caveat: the shared guarded fetch wrapper is the right boundary for returning slots across provider transports. The remaining decision is whether the supplied local TCP/GC harness is enough or a longer live soak is required.

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 63621eead219.

Label changes

Label justifications:

  • P1: The linked bug can exhaust gateway file descriptors and break real agent turns that depend on streamed provider calls.
  • merge-risk: 🚨 availability: The PR changes provider stream cancellation/finalization, where missed or premature cleanup can affect long-running Gateway availability.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes after-fix real local TCP streaming output using the actual buildGuardedModelFetch path and showing the abandoned stream socket closed.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix real local TCP streaming output using the actual buildGuardedModelFetch path and showing the abandoned stream socket closed.
Evidence reviewed

PR surface:

Source +23, Tests +163, Other +1. Total +187 across 9 files.

View PR surface stats
Area Files Added Removed Net
Source 3 23 0 +23
Tests 5 188 25 +163
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 1 1 0 +1
Total 9 212 25 +187

What I checked:

  • Repository policy read: Read the full root AGENTS.md and scoped guides for src/agents, src/gateway, scripts, extensions, and test; provider runtime proof, dependency inspection, and real behavior proof requirements shaped this review. (AGENTS.md:1, 63621eead219)
  • Current main leak path: Current main's managed provider response releases the guarded fetch slot on normal completion, read error, or explicit cancel, but has no release path when the wrapper body is abandoned before those callbacks run. (src/agents/provider-transport-fetch.ts:389, 63621eead219)
  • PR fallback cleanup: The PR head adds a FinalizationRegistry fallback, unregisters on normal finalize, cancels the locked source reader, and keeps release/local-service cleanup idempotent. (src/agents/provider-transport-fetch.ts:373, 766b36cf9ccb)
  • Parser-side deterministic cancellation: The PR also cancels open Anthropic and Google SSE readers when parser iteration exits before a clean end, reducing reliance on GC for known early-exit error paths. (src/agents/anthropic-transport-stream.ts:655, 766b36cf9ccb)
  • Focused regression coverage: The PR adds tests for abandoned managed-stream finalization plus Anthropic and Google malformed/open SSE cancellation paths. (src/agents/provider-transport-fetch.test.ts:186, 766b36cf9ccb)
  • Dependency contract check: TypeScript's FinalizationRegistry contract allows an unregister token and forbids heldValue being the target; DOM stream types define reader cancel as a Promise-returning cleanup path, matching the PR's approach.

Likely related people:

  • steipete: Local blame attributes the current provider transport stream implementation to Peter Steinberger in the grafted main history, and the PR branch includes maintainer follow-up commits from the same area for deterministic reader cancellation and shard/tooling updates. (role: recent area contributor; confidence: medium; commits: 2b61d38a455c, 32f20d552745, 3c8f526d197e; files: src/agents/provider-transport-fetch.ts, src/agents/anthropic-transport-stream.ts, extensions/google/transport-stream.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@samzong
samzong marked this pull request as draft May 31, 2026 02:41
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P1 High-priority user-facing bug, regression, or broken workflow. labels May 31, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label May 31, 2026
@samzong
samzong marked this pull request as ready for review May 31, 2026 07:35
@openclaw-barnacle openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed proof: sufficient ClawSweeper judged the real behavior proof convincing. labels May 31, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 31, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 31, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 31, 2026
@steipete steipete self-assigned this May 31, 2026
@steipete
steipete force-pushed the fix/gateway-undici-socket-leak branch from f7c84df to b9f986b Compare May 31, 2026 15:26
@openclaw-barnacle openclaw-barnacle Bot added scripts Repository scripts app: web-ui App: web-ui and removed proof: sufficient ClawSweeper judged the real behavior proof convincing. labels May 31, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 31, 2026
@steipete
steipete force-pushed the fix/gateway-undici-socket-leak branch from e5f281b to bcea9c0 Compare May 31, 2026 15:48
@openclaw-barnacle openclaw-barnacle Bot removed app: web-ui App: web-ui proof: sufficient ClawSweeper judged the real behavior proof convincing. labels May 31, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 31, 2026
@clawsweeper clawsweeper Bot added the merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. label May 31, 2026
@steipete
steipete force-pushed the fix/gateway-undici-socket-leak branch from e645e90 to 766b36c Compare May 31, 2026 16:04
@openclaw-barnacle openclaw-barnacle Bot added the gateway Gateway runtime label May 31, 2026
@steipete
steipete force-pushed the fix/gateway-undici-socket-leak branch from 766b36c to a1db789 Compare May 31, 2026 16:13
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 31, 2026
@steipete

Copy link
Copy Markdown
Contributor

Land-ready after maintainer fixups.

Summary:

  • Kept the abandoned managed-response finalizer as a last-resort safety net for leaked provider streams.
  • Added deterministic reader cancellation for Anthropic and Gemini SSE parsers when parsing exits early.
  • Added regression coverage for abandoned provider bodies, Anthropic error-event early exit, Gemini malformed-event early exit, and the mocked exec shell-snapshot CI drift.
  • Rebased onto current main and fixed the inherited live-shard/lint expectation drift for gateway-acp-spawn-defaults.live.test.ts.

Local verification:

  • node scripts/run-vitest.mjs src/agents/provider-transport-fetch.test.ts src/agents/anthropic-transport-stream.test.ts extensions/google/transport-stream.test.ts src/agents/bash-tools.test.ts src/agents/bash-tools.exec.path.test.ts test/scripts/test-live-shard.test.ts
  • pnpm check:test-types
  • node scripts/run-oxlint-shards.mjs --threads=8
  • .agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main --parallel-tests "node scripts/run-vitest.mjs src/agents/provider-transport-fetch.test.ts src/agents/anthropic-transport-stream.test.ts extensions/google/transport-stream.test.ts src/agents/bash-tools.test.ts src/agents/bash-tools.exec.path.test.ts test/scripts/test-live-shard.test.ts"
  • git diff --check origin/main...HEAD

CI:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling extensions: google gateway Gateway runtime merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. P1 High-priority user-facing bug, regression, or broken workflow. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. scripts Repository scripts size: M status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. triage: blank-template Candidate: PR template appears mostly untouched.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gateway leaks undici sockets on every streamed Anthropic API call (buildManagedResponse missing finalize on GC)

2 participants