Skip to content

fix(codex): backoff startup connection-close retries and surface exhaustion (#83959) [AI-assisted]#94137

Open
ml12580 wants to merge 1 commit into
openclaw:mainfrom
ml12580:fix/vuln-83959-codex-startup-retry-backoff
Open

fix(codex): backoff startup connection-close retries and surface exhaustion (#83959) [AI-assisted]#94137
ml12580 wants to merge 1 commit into
openclaw:mainfrom
ml12580:fix/vuln-83959-codex-startup-retry-backoff

Conversation

@ml12580

@ml12580 ml12580 commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

[AI-assisted] This PR was generated using Claude Code.

Summary

  • Problem: When the Codex app-server connection closes during startup, startCodexAttemptThread cleared the shared client and retried immediately — before the replacement app-server process had bound/become ready — under a tight 3-attempt budget. All retries landed in the same failing window and exhausted, so a scheduled/background turn failed even though the Codex route became viable moments later. Cron automation reported job failure with no task-level problem. (Issue Codex app-server startup retries can exhaust before replacement server is ready #83959)
  • Why now: P1 for background/cron Codex turns; the issue's own local mitigation confirmed the direction (raise the startup-close retry budget + add a bounded backoff).
  • Intended outcome: Startup tolerates a short replacement-server warm-up window before declaring failure, and when it does give up it says so distinctly instead of surfacing a misleading "client is closed".
  • Out of scope: The separate per-run replay retry gate in embedded-agent-runner/run.ts (already correct, untouched) and the shared-client eviction work tracked under Codex harness app-server shared client is evicted across agents because cache key includes agentDir #79495.
  • Success: A transient startup connection-close no longer fails the turn when the server recovers within the bounded window; persistent closes fail with a distinguishable error.
  • Reviewer focus: The retry-loop change in attempt-startup.ts (budget 3→5, added computeBackoff+sleepWithAbort backoff, new CodexAppServerStartupExhaustedError) and the two new regression tests.

Linked context

Closes #83959

Related #79495 (shared Codex app-server client eviction across agents — broader, separate).

Was this requested by a maintainer or owner? No — external contributor fix for an open P1.

Real behavior proof (required for external PRs)

  • Behavior or issue addressed: Codex app-server startup connection-close retry loop exhausted before the replacement server was ready (Codex app-server startup retries can exhaust before replacement server is ready #83959).
  • Real environment tested: Windows 11, Node v22.22.3, OpenClaw checkout. The capture drives the REAL startCodexAttemptThread (the production function this PR changes) against a REAL local app-server subprocess spawned by the REAL CodexAppServerClient.start -> createStdioTransport -> child_process.spawn. The subprocess speaks the same newline-delimited JSON-RPC framing as the real Codex app-server; on the first N attempts it closes its connection during the initialize handshake (real child exit -> real codex app-server exited: error -> real isCodexAppServerConnectionClosedError -> the real startup retry loop), and completes the handshake on attempt N+1. It is a stand-in for the codex binary (no Codex account/backend is available on this host); it is NOT a reimplementation of the retry loop and does not stub startCodexAttemptThread.
  • Exact steps or command run after this patch: pnpm exec tsx .tmp/capture-83959-real.ts with CLOSES_BEFORE_SUCCESS=4 (recovery) and CLOSES_BEFORE_SUCCESS=99 (persistent-close exhaustion), run on unpatched main and on this PR branch. Driver: .tmp/capture-83959-real.ts; stand-in app-server subprocess: .tmp/fake-codex-app-server.mjs.
  • Evidence after fix (terminal capture):
$ CLOSES_BEFORE_SUCCESS=4 pnpm exec tsx .tmp/capture-83959-real.ts   # this PR (budget=5, bounded backoff)
codex app-server connection closed during startup; restarting app-server and retrying
codex app-server connection closed during startup; restarting app-server and retrying
codex app-server connection closed during startup; restarting app-server and retrying
codex app-server connection closed during startup; restarting app-server and retrying
{
  "outcome": "recovered",
  "closesBeforeSuccess": 4,
  "spawnAttempts": 5,
  "spawnModes": ["exit", "exit", "exit", "exit", "serve"],
  "observedWallClockGapsMs": [1962, 2531, 3399, 5340],
  "backoffPolicyComputedMs": [532, 1179, 2019, 4000],
  "elapsedMs": 14856
}

$ CLOSES_BEFORE_SUCCESS=99 pnpm exec tsx .tmp/capture-83959-real.ts  # this PR, persistent close
codex app-server connection closed during startup; restarting app-server and retrying
codex app-server connection closed during startup; restarting app-server and retrying
codex app-server connection closed during startup; restarting app-server and retrying
codex app-server connection closed during startup; restarting app-server and retrying
codex app-server connection closed during startup; retries exhausted
{
  "outcome": "exhausted",
  "closesBeforeSuccess": 99,
  "spawnAttempts": 5,
  "spawnModes": ["exit", "exit", "exit", "exit", "exit"],
  "observedWallClockGapsMs": [1836, 2417, 3459, 5278],
  "backoffPolicyComputedMs": [575, 1183, 2332, 4000],
  "elapsedMs": 14385,
  "error": {
    "message": "codex app-server startup exhausted after 5 connection-close retries",
    "name": "CodexAppServerStartupExhaustedError",
    "cause": "codex app-server exited: code=1 signal=null"
  }
}
  • Observed result after fix: BEFORE (unpatched, budget 3) — startup exhausted at 3 attempts with 2 flat ~1.3s gaps (no backoff sleep; the gaps are spawn/cleanup overhead only) and the turn failed with the raw codex app-server exited: code=1 signal=null (plain Error, no distinct exhaustion type). AFTER (this PR, budget 5 + bounded abortable backoff) — the same 4 leading closes were survived: startup recovered on the 5th attempt with 4 monotonically increasing gaps [1962, 2531, 3399, 5340] ms that track the real computeBackoff policy [532, 1179, 2019, 4000] ms (policy value + spawn/cleanup overhead), and the turn succeeded. Persistent-close AFTER — exhausted at 5 with the distinct CodexAppServerStartupExhaustedError ("codex app-server startup exhausted after 5 connection-close retries"), the original codex app-server exited: preserved as cause, instead of the raw close error.
  • What was not tested: A live codex app-server process backed by a real Codex/LLM account (no local Codex harness or API key on this Windows host). The stand-in subprocess reproduces the connection-close-during-startup condition and the JSON-RPC initialize/thread-start handshake but does not execute Codex model turns. macOS/Linux runtime not exercised locally.
  • Proof limitations or environment constraints: The stand-in app-server subprocess is a real local process exercising the real production startup retry path (real spawn, real stdio JSON-RPC, real child-exit close detection, real computeBackoff+sleepWithAbort wall-clock delays, real CodexAppServerStartupExhaustedError); it is not the real codex binary because no Codex backend is available on this host.
  • Before evidence (optional but encouraged):
$ pnpm exec tsx .tmp/capture-83959-real.ts        # unpatched main (budget=3, no backoff)
codex app-server connection closed during startup; restarting app-server and retrying
codex app-server connection closed during startup; restarting app-server and retrying
codex app-server connection closed during startup; retries exhausted
{
  "outcome": "exhausted",
  "closesBeforeSuccess": 4,
  "spawnAttempts": 3,
  "spawnModes": ["exit", "exit", "exit"],
  "observedWallClockGapsMs": [1329, 1231],
  "backoffPolicyComputedMs": [551, 1078, 2252, 4000],
  "elapsedMs": 3917,
  "error": { "message": "codex app-server exited: code=1 signal=null", "name": "Error" }
}

Tests and validation

  • Commands run: pnpm exec oxfmt --check (changed files, clean); pnpm tsgo:extensions (clean); pnpm tsgo:extensions:test (clean); pnpm exec oxlint --type-aware (changed files, clean); node scripts/run-vitest.mjs extensions/codex/src/app-server/attempt-startup.test.ts -t "83959" (2 passed).
  • Regression coverage added: Two new tests in attempt-startup.test.ts covering the previously-untested startup connection-close retry loop — (1) survives N transient closes with bounded backoff and recovers, asserting sleepWithAbort is awaited between attempts with bounded delays; (2) persistent closes produce the distinct CodexAppServerStartupExhaustedError with the original error as cause, and the loop is bounded (no infinite retry).
  • Failed before this fix: Both new tests fail on unpatched main (budget=3 exhausts before the 4th-close recovery; raw "client is closed" rethrown with no distinct exhaustion error).
  • Note: pnpm check:host-env-policy:swift skipped on Windows (no Swift).

Risk checklist

Did user-visible behavior change? Yes — a Codex startup that previously failed after 3 immediate retries now retries up to 5 times with a bounded backoff (worst case a few extra seconds before a transient-close recovery or a cleaner failure).

Did config, environment, or migration behavior change? No.

Did security, auth, secrets, network, or tool execution behavior change? No — the change only affects the Codex app-server startup retry timing/error shape; no auth, secret, network, or tool-execution surface touched. extensions/codex/src/** is not a CODEOWNERS secops path.

What is the highest-risk area? Added latency on a Codex startup that closes repeatedly during warm-up (bounded to ~8s worst case across 5 attempts).

How is that risk mitigated? Backoff only fires after a connection-close failure (never on the success path); sleepWithAbort is abortable so a cancelled/aborted run fails fast instead of stalling; the budget is still bounded (5) so a permanently-broken server fails rather than retrying forever; the distinct exhaustion error preserves the original cause for diagnostics.

Current review state

What is the next action? Awaiting maintainer review. The real behavior proof was upgraded (see above) from a synthetic retry-loop model to a real local app-server subprocess driving the actual startCodexAttemptThread; @clawsweeper re-review requested.

What is still waiting on author, maintainer, CI, or external proof? External-fork heavy CI shards require maintainer workflow approval to run. A maintainer proof/latency override is also an acceptable path per ClawSweeper's option 2 if the stand-in-subprocess proof is deemed sufficient.

Which bot or reviewer comments were addressed? Upgraded the real behavior proof in response to ClawSweeper's status: 📣 needs proof ("supplies tests and a synthetic terminal model, but not after-fix live OpenClaw/Codex app-server startup proof").

@openclaw-barnacle openclaw-barnacle Bot added extensions: codex size: S proof: supplied External PR includes structured after-fix real behavior proof. labels Jun 17, 2026
@ml12580

ml12580 commented Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

CI status note (round 1):

  • Real behavior proof — pass

  • checks-node-agentic-agents-embedded — pass (this is the shard that runs extensions/codex/src/app-server/attempt-startup.test.ts, including the two new Codex app-server startup retries can exhaust before replacement server is ready #83959 regression tests, on Linux)

  • ✅ 129/163 checks pass; check-prod-types, check-test-types, build-artifacts, actionlint, all check-additional-extension-*, all checks-node-agentic-* green.

  • ⚠️ 2 failures, both unrelated to this PR's changed code:

    • checks-node-core-runtime-cron-isolated-agent — runs only src/cron/isolated-agent* tests (per scripts/lib/ci-node-test-plan.mjs); does not execute any extensions/codex/** test.
    • checks-node-core-src-security — runs src/security tests; likewise does not touch the Codex app-server startup path changed here.

    Both failed at ~11m01s (job timeout signature), not on a test assertion from this PR's diff. I don't have admin rights to rerun failed jobs on an external fork; flagging for a maintainer re-run if these don't clear on their own. No code change is warranted — the changed file (extensions/codex/src/app-server/attempt-startup.ts) is not exercised by either failing shard.

  • The 32 skipping checks are the external-fork workflow-approval-gated shards (Critical Quality boundaries, native Telegram proof, etc.) — same gate as other external PRs.

Nothing actionable from my side until maintainer review/approval. Happy to address any review feedback.

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P1 High-priority user-facing bug, regression, or broken workflow. labels Jun 19, 2026
@clawsweeper

clawsweeper Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 9, 2026, 12:09 PM ET / 16:09 UTC.

Summary
The PR changes Codex app-server startup handling to tolerate transient connection-close failures with bounded retry backoff, a larger retry window, a distinct exhaustion error, and focused regression tests.

PR surface: Source +45, Tests +85. Total +130 across 2 files.

Reproducibility: yes. at source level. Current main catches Codex app-server closed-client and exit errors in startCodexAttemptThread, clears the shared client, and spends a 3-attempt retry budget without backoff, matching the field logs in #83959.

Review metrics: 1 noteworthy metric.

  • Startup Retry Policy: 1 policy changed: attempts 3->5; 4 retry sleeps capped at 4,000 ms. This quantified timing change is the main availability and latency tradeoff maintainers need to accept before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #83959
Summary: This PR is the open candidate fix for the canonical Codex app-server startup-close retry exhaustion issue; related reports are triggers, downstream symptoms, umbrella stability tracking, or adjacent startup diagnostics.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Risk before merge

  • [P1] Merging intentionally adds up to roughly eight seconds of backoff, plus spawn and cleanup overhead, before a persistently closing Codex app-server reports terminal failure.
  • [P2] Adjacent open work in fix(codex): bound shared app-server startup waits #89442 touches Codex startup timeout attribution, so maintainers should do a final current-base sequencing check before landing both startup changes.

Maintainer options:

  1. Accept the bounded retry window (recommended)
    Merge after maintainer review if the added persistent-close wait is an acceptable availability tradeoff for recovering transient replacement-server warm-up races.
  2. Request a readiness-gate revision
    Have the author replace or supplement the sleep-based retry admission with an explicit replacement-server readiness signal before merge.
  3. Pause for startup sequencing
    Hold this PR until maintainers decide how it should compose with the adjacent initialize-timeout PR and broader shared-client startup work.

Next step before merge

  • No automated repair is indicated because the patch has no discrete blocking defect; the next action is maintainer review of the availability-latency tradeoff and current-base sequencing.

Maintainer decision needed

  • Question: Should this Codex startup fix land as the bounded 5-attempt backoff now, or should the author replace it with an explicit replacement-server readiness gate before merge?
  • Rationale: The patch is narrow and source/proof-backed, but it deliberately trades faster persistent failure for better transient recovery in a P1 startup path, and issue discussion raised readiness-gate preferences that require maintainer intent.
  • Likely owner: steipete — The retry loop and recent Codex startup seam history are most directly tied to steipete's commits.
  • Options:
    • Land bounded backoff (recommended): Accept the small added persistent-failure latency because the PR fixes the reported transient startup-close window with focused tests and real subprocess proof.
    • Require readiness gate: Ask for a follow-up patch that admits retries only after a concrete replacement-server readiness signal instead of time-based backoff.
    • Sequence with adjacent startup PR: Pause until maintainers decide how this PR should compose with the initialize-timeout work in fix(codex): bound shared app-server startup waits #89442.

Security
Cleared: Cleared: the diff only changes Codex startup retry timing/error handling and tests, with no dependency, workflow, secret, install, or package-resolution changes.

Review details

Best possible solution:

Land this PR after maintainer acceptance of the bounded backoff/error-shape tradeoff and a final current-base check, or ask for a readiness-gate variant if maintainers prefer explicit server readiness over time-based retry admission.

Do we have a high-confidence way to reproduce the issue?

Yes, at source level. Current main catches Codex app-server closed-client and exit errors in startCodexAttemptThread, clears the shared client, and spends a 3-attempt retry budget without backoff, matching the field logs in #83959.

Is this the best way to solve the issue?

Yes, with one maintainer tradeoff. The fix lives in the Codex app-server startup owner and reuses the shared backoff/sleep helper, while maintainers still need to decide whether bounded sleep is preferable to an explicit readiness gate.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 4f9a371c9530.

Label changes

Label justifications:

  • P1: The PR targets a real scheduled/background Codex startup failure that can break user work before a task begins.
  • merge-risk: 🚨 availability: Changing retry timing in the Codex app-server startup loop can improve transient recovery but also delays terminal failure for persistently closing startup paths.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): Sufficient: the PR includes before/after terminal captures driving the production startup path through a real spawned stand-in app-server process; it is not a live Codex backend run, but it directly proves this retry-loop behavior.
  • proof: sufficient: Contributor real behavior proof is sufficient. Sufficient: the PR includes before/after terminal captures driving the production startup path through a real spawned stand-in app-server process; it is not a live Codex backend run, but it directly proves this retry-loop behavior.
Evidence reviewed

PR surface:

Source +45, Tests +85. Total +130 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 47 2 +45
Tests 1 85 0 +85
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 132 2 +130

What I checked:

Likely related people:

  • steipete: Blame and history tie the current startup-close retry loop and earlier retry/attempt-seam work to Peter Steinberger commits, including the split app-server attempt seam and original startup-close retry handling. (role: introduced behavior and recent area contributor; confidence: high; commits: e5dc3f712e5c, a4c2e7f5cf1b, 554d772c1a83; files: extensions/codex/src/app-server/attempt-startup.ts, extensions/codex/src/app-server/run-attempt.ts, extensions/codex/src/app-server/client.ts)
  • vincentkoc: Recent commits by Vincent Koc changed Codex app-server startup cleanup, abandoned startup retirement, and related shared-client cleanup in the same startup lifecycle path. (role: recent area contributor; confidence: high; commits: f47277871791, 5a0d9d6326f7, ff3ebb1c24ca; files: extensions/codex/src/app-server/attempt-startup.ts, extensions/codex/src/app-server/shared-client.ts)
  • kevinslin: Kevin Lin recently worked on remote app-server plugin support across attempt-startup.ts, client.ts, and config.ts, making him relevant for adjacent app-server startup/runtime behavior. (role: recent adjacent contributor; confidence: medium; commits: bc5081c58730, c5d34c8376f8; files: extensions/codex/src/app-server/attempt-startup.ts, extensions/codex/src/app-server/client.ts, extensions/codex/src/app-server/config.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (2 earlier review cycles)
  • reviewed 2026-06-21T21:39:51.068Z sha 2746c6f :: needs real behavior proof before merge. :: none
  • reviewed 2026-07-01T19:04:45.891Z sha 2746c6f :: needs maintainer review before merge. :: none

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Jun 20, 2026
@ml12580

ml12580 commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

Real behavior proof upgraded in response to status: 📣 needs proof ("supplies tests and a synthetic terminal model, but not after-fix live OpenClaw/Codex app-server startup proof").

The previous capture reimplemented the retry loop (simulateRecovery) and only imported computeBackoff. The new proof drives the real startCodexAttemptThread (the production function this PR changes) against a real local app-server subprocess spawned by the real CodexAppServerClient.startcreateStdioTransportchild_process.spawn. The subprocess speaks the same newline-delimited JSON-RPC framing as the real Codex app-server and closes its connection during the initialize handshake on the first N attempts (real child exit → real codex app-server exited: → real isCodexAppServerConnectionClosedError → the real startup retry loop), then completes the handshake on attempt N+1. It is a stand-in for the codex binary (no Codex backend is available on this host), not a reimplementation of the retry loop.

Observed on the same CLOSES_BEFORE_SUCCESS=4 scenario (the issue's "closes on first 4, recovers on 5th" shape):

  • BEFORE (unpatched main, budget 3): exhausted at 3 attempts, 2 flat ~1.3s gaps (no backoff; spawn/cleanup overhead only), turn failed with the raw codex app-server exited: code=1 signal=null (plain Error, no distinct exhaustion type).
  • AFTER (this PR, budget 5 + bounded abortable backoff): recovered on the 5th attempt with 4 monotonically increasing gaps [1962, 2531, 3399, 5340] ms tracking the real computeBackoff policy [532, 1179, 2019, 4000] ms (policy value + spawn/cleanup overhead); turn succeeded.
  • AFTER persistent close (CLOSES_BEFORE_SUCCESS=99): exhausted at 5 with the distinct CodexAppServerStartupExhaustedError ("codex app-server startup exhausted after 5 connection-close retries"), the original codex app-server exited: preserved as cause, instead of the raw close error.

Full terminal captures are in the ## Real behavior proof section of the PR body. Driver: .tmp/capture-83959-real.ts; stand-in app-server subprocess: .tmp/fake-codex-app-server.mjs.

@clawsweeper re-review

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 🛠️ actively grinding The PR author has acted after the latest ClawSweeper review and work remains. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jun 23, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 🛠️ actively grinding The PR author has acted after the latest ClawSweeper review and work remains. labels Jul 7, 2026
@clawsweeper clawsweeper Bot added the status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. label Jul 7, 2026
…ustion (openclaw#83959)

The Codex app-server startup retry loop re-acquired the shared client
immediately after a connection-close, before the replacement app-server
process had time to bind, and exhausted a 3-attempt budget in the same
failing window. Scheduled/background turns then failed even though the
route became viable moments later.

Add a bounded, abortable backoff between startup retries, raise the
budget from 3 to 5, and throw a distinct CodexAppServerStartupExhausted
Error (original error preserved as cause) when the window is exhausted,
so callers can distinguish startup lifecycle exhaustion from a mid-turn
client close.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
@ml12580
ml12580 force-pushed the fix/vuln-83959-codex-startup-retry-backoff branch from 2746c6f to 40c0d9f Compare July 9, 2026 15:46
@openclaw-barnacle

Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Jul 24, 2026
@clawsweeper

clawsweeper Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper status: review started.

I am starting a fresh review of this pull request: fix(codex): backoff startup connection-close retries and surface exhaustion (#83959) [AI-assisted] This is item 1/1 in the current shard. Shard 1/4.

This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking.

Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

extensions: codex merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. P1 High-priority user-facing bug, regression, or broken workflow. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S stale Marked as stale due to inactivity status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Codex app-server startup retries can exhaust before replacement server is ready

1 participant