Skip to content

fix(tlon): clear SSE connect timeout after failed openStream#104585

Merged
steipete merged 2 commits into
openclaw:mainfrom
hugenshen:fix/tlon-sse-connect-timeout-cleanup
Jul 15, 2026
Merged

fix(tlon): clear SSE connect timeout after failed openStream#104585
steipete merged 2 commits into
openclaw:mainfrom
hugenshen:fix/tlon-sse-connect-timeout-cleanup

Conversation

@hugenshen

@hugenshen hugenshen commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

Fixes an issue where Tlon Urbit SSE openStream armed a 60s connect setTimeout and only cleared it after a successful urbitFetch. When the connect fetch rejected (DNS/SSRF/network) or returned non-OK, the timer stayed armed until it fired, and reconnect storms could stack dangling timers in the Gateway process.

Why This Change Was Made

Wrap the connect fetch in try/finally and always clearTimeout, matching the zalouser probe timeout cleanup pattern, so failed connects release the timer immediately without changing the connect deadline or stream lifecycle.

User Impact

Flaky Urbit hosts no longer leave dangling connect timers in long-running Tlon monitors during reconnect failures.

Evidence

Unit regression (mocked fetch + fake timers)

node scripts/run-vitest.mjs extensions/tlon/src/urbit/sse-client.test.ts --reporter=verbose
 ✓ ... > openStream > clears the connect timeout when urbitFetch rejects 12ms
 ✓ ... > openStream > clears the connect timeout when the stream response is not ok 10ms
 Test Files  1 passed (1)
      Tests  18 passed (18)

Production-path connect-timeout storm proof

node scripts/run-vitest.mjs extensions/tlon/src/urbit/sse-client.open-stream-timeout.proof.test.ts --reporter=verbose
 ✓ ... > clears connect timers after a real non-OK stream response storm 305ms
 ✓ ... > clears the connect timer when production urbitFetch rejects 10ms
 Test Files  1 passed (1)
      Tests  2 passed (2)

The storm case runs production openStreamurbitFetchfetchWithSsrFGuard against a real loopback node:http server that returns 503 five times; real timers; each armed setTimeout(..., 60000) is cleared. The reject case uses a closed loopback port (127.0.0.1:9) so the production transport fails before headers.

Real behavior proof

  • Behavior or issue addressed: Failed SSE connect attempts no longer leave uncleared 60s AbortController timers, including under repeated failure.
  • Canonical reachability path: Tlon monitor reconnect → UrbitSSEClient.openStream → production urbitFetch / fetchWithSsrFGuard → reject or non-OK → finally { clearTimeout(timeoutId) }.
  • Boundary crossed: local-runtime HTTP; loopback node:http + closed-port TCP failure; production SSRF-guarded transport (no urbitFetch mock, no fake timers).
  • Shared helper / provider constraint check: Same try/finally { clearTimeout } pattern as extensions/zalouser/src/probe.ts; no new config/API.
  • Real environment tested: macOS, Node via node scripts/run-vitest.mjs with real HTTP listeners and real timers.
  • Exact steps or command run after this patch: node scripts/run-vitest.mjs extensions/tlon/src/urbit/sse-client.open-stream-timeout.proof.test.ts extensions/tlon/src/urbit/sse-client.test.ts --reporter=verbose
  • Evidence after fix:
 ✓ |extension-messaging| ... > clears the connect timeout when urbitFetch rejects 12ms
 ✓ |extension-messaging| ... > clears the connect timeout when the stream response is not ok 10ms
 ✓ |extension-messaging| ... > clears connect timers after a real non-OK stream response storm 305ms
 ✓ |extension-messaging| ... > clears the connect timer when production urbitFetch rejects 10ms
 Test Files  2 passed (2)
      Tests  20 passed (20)
[test] passed 1 Vitest shard in 7.90s
  • Observed result after fix: Five sequential non-OK connects each arm then clear a 60s connect timer with matching clearTimeout(handle); a real transport reject also clears its single connect timer.
  • What was not tested: Live Urbit ship reconnect storm against a real Tlon host; SSE stream body processing after a successful connect.
  • Fix classification: Root cause fix.

Risk / Compatibility

Low. Successful connects still clear the timer after headers; only failed connects gain the missing cleanup.

@openclaw-barnacle openclaw-barnacle Bot added channel: tlon Channel integration: tlon size: S labels Jul 11, 2026
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. labels Jul 11, 2026
@clawsweeper

clawsweeper Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 14, 2026, 9:25 PM ET / July 15, 2026, 01:25 UTC.

Summary
The PR always clears the Tlon SSE connection timer after openStream fetch completion or failure and adds mocked plus real loopback-network regression coverage.

PR surface: Source -1, Tests +159. Total +158 across 3 files.

Reproducibility: yes. at the source level: current main leaves the timer armed when urbitFetch rejects, and the PR supplies focused production-path proof for both rejection and non-OK responses. I did not independently execute a current-main failing run during this read-only review.

Review metrics: none identified.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

  • [P1] A live authenticated Urbit ship reconnect storm was not exercised, although the submitted loopback HTTP and closed-port proof crosses the relevant production transport boundary.

Maintainer options:

  1. Decide the mitigation before merge
    Land the try/finally cleanup with both regression suites intact, provided the clean three-way merge result and required checks remain green on the exact head.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • No automated repair is indicated; the patch has no actionable finding and only needs ordinary maintainer landing review.

Security
Cleared: The localized runtime and test changes introduce no dependency, secret, permission, downloaded-code, or supply-chain surface and preserve the existing SSRF-guarded transport.

Review details

Best possible solution:

Land the try/finally cleanup with both regression suites intact, provided the clean three-way merge result and required checks remain green on the exact head.

Do we have a high-confidence way to reproduce the issue?

Yes at the source level: current main leaves the timer armed when urbitFetch rejects, and the PR supplies focused production-path proof for both rejection and non-OK responses. I did not independently execute a current-main failing run during this read-only review.

Is this the best way to solve the issue?

Yes. Moving the existing timer cleanup into finally is the narrowest maintainable fix because it covers every fetch exit path without changing the timeout deadline, reconnect policy, or successful stream lifecycle.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 3c0654c00271.

Label changes

Label justifications:

  • P2: The PR fixes bounded resource retention during repeated Tlon connection failures without changing configuration, APIs, or normal successful-stream behavior.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): After-fix live output demonstrates repeated real HTTP 503 responses and a real TCP connection rejection through the production SSRF-guarded transport, with every 60-second timer handle cleared.
  • proof: sufficient: Contributor real behavior proof is sufficient. After-fix live output demonstrates repeated real HTTP 503 responses and a real TCP connection rejection through the production SSRF-guarded transport, with every 60-second timer handle cleared.
Evidence reviewed

PR surface:

Source -1, Tests +159. Total +158 across 3 files.

View PR surface stats
Area Files Added Removed Net
Source 1 31 32 -1
Tests 2 159 0 +159
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 3 190 32 +158

What I checked:

Likely related people:

  • hclsys: Authored the merged change that added the 60-second SSE connection timer and established the cleanup behavior this PR repairs. (role: introduced timeout behavior; confidence: high; commits: 6b5460c3d18f, 411d5fda587c; files: extensions/tlon/src/urbit/sse-client.ts)
  • Pick-cat: Recently carried substantial merged hardening work in the same Tlon SSE client and its regression suite. (role: recent area contributor; confidence: high; commits: c895c97d66d0, 1710ccad2977; files: extensions/tlon/src/urbit/sse-client.ts, extensions/tlon/src/urbit/sse-client.test.ts)
  • steipete: Merged and integrated the original Tlon timeout change, making this a useful routing candidate for the narrow follow-up. (role: merger and recent integrator; confidence: medium; commits: 411d5fda587c, 13c8b2a21c1a; files: extensions/tlon/src/urbit/sse-client.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (28 earlier review cycles; latest 8 shown)
  • reviewed 2026-07-14T22:31:43.549Z sha 2f3769f :: needs maintainer review before merge. :: none
  • reviewed 2026-07-14T22:49:59.129Z sha 2f3769f :: needs maintainer review before merge. :: none
  • reviewed 2026-07-14T23:06:55.584Z sha 2f3769f :: needs maintainer review before merge. :: none
  • reviewed 2026-07-14T23:24:42.855Z sha 2f3769f :: needs maintainer review before merge. :: none
  • reviewed 2026-07-14T23:42:38.090Z sha 2f3769f :: needs maintainer review before merge. :: none
  • reviewed 2026-07-14T23:59:18.425Z sha 2f3769f :: needs maintainer review before merge. :: none
  • reviewed 2026-07-15T00:17:27.530Z sha 2f3769f :: needs maintainer review before merge. :: none
  • reviewed 2026-07-15T00:40:35.487Z sha 2f3769f :: needs maintainer review before merge. :: none

@hugenshen
hugenshen force-pushed the fix/tlon-sse-connect-timeout-cleanup branch from 1c618ac to a6f2fb7 Compare July 11, 2026 18:19
@hugenshen

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 11, 2026
@hugenshen
hugenshen force-pushed the fix/tlon-sse-connect-timeout-cleanup branch from a6f2fb7 to f79d2ee Compare July 12, 2026 00:07
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Jul 12, 2026
@hugenshen
hugenshen force-pushed the fix/tlon-sse-connect-timeout-cleanup branch from f79d2ee to 8bdd6d4 Compare July 13, 2026 15:06
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jul 13, 2026
@hugenshen
hugenshen force-pushed the fix/tlon-sse-connect-timeout-cleanup branch 2 times, most recently from c597d5c to 2e5a551 Compare July 14, 2026 00:52
@clawsweeper clawsweeper Bot removed the rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. label Jul 14, 2026
@clawsweeper clawsweeper Bot added rating: 🦀 challenger crab Exceptional PR readiness: strong proof, clean patch, and convincing validation. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. and removed rating: 🦀 challenger crab Exceptional PR readiness: strong proof, clean patch, and convincing validation. labels Jul 14, 2026
@hugenshen
hugenshen force-pushed the fix/tlon-sse-connect-timeout-cleanup branch 2 times, most recently from 3ee883a to 5cf6f37 Compare July 14, 2026 14:04
@hugenshen
hugenshen force-pushed the fix/tlon-sse-connect-timeout-cleanup branch from 5cf6f37 to 2f3769f Compare July 14, 2026 14:50
@clawsweeper clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jul 14, 2026
@steipete steipete self-assigned this Jul 15, 2026
@steipete

Copy link
Copy Markdown
Contributor

Land-ready proof for exact head fbcb0124b88a833c1b69a0ee655c32b68a869fee:

  • Refactored the cleanup boundary so the 60-second deadline owns only the response-header wait. Rejected fetches always disarm it; non-OK responses clear it before releasing the guarded response; successful SSE body ownership is unchanged.
  • Testbox tbx_01kxm147at2gxpgxffdspz25z2 (run 29458087968): pnpm test extensions/tlon/src/urbit/sse-client.test.ts extensions/tlon/src/urbit/sse-client.open-stream-timeout.proof.test.ts -- --reporter=verbose — 26/26 passed.
  • That proof includes five sequential real loopback HTTP 503 connections and a real socket rejection through openStream -> urbitFetch -> fetchWithSsrFGuard; every armed connect timer was cleared.
  • Same Testbox: touched-file oxfmt --check and pnpm check:test-types passed.
  • Fresh Codex autoreview: no accepted/actionable findings (0.91 confidence).
  • Hosted exact-head CI is green: run 29458383743.
  • Repo-native OPENCLAW_TESTBOX=1 scripts/pr prepare-run 104585 passed and verified the hosted head.

Known gap: no live reconnect storm was induced against a real Tlon ship. The exercised network path is the production SSRF-guarded transport with real sockets and timers; no fetch mock or fake timer is used in the integration proof.

@steipete
steipete merged commit b18f1a8 into openclaw:main Jul 15, 2026
84 checks passed
@steipete

Copy link
Copy Markdown
Contributor

Merged via squash.

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 16, 2026
…w#104585)

* fix(tlon): clear SSE connect timeout after failed openStream

* refactor(tlon): scope SSE connect timeout

---------

Co-authored-by: Peter Steinberger <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: tlon Channel integration: tlon P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. size: M status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants