Skip to content

fix(codex): release timed-out app-server lanes#94833

Open
clawsweeper[bot] wants to merge 4 commits into
mainfrom
clawsweeper/automerge-openclaw-openclaw-94685
Open

fix(codex): release timed-out app-server lanes#94833
clawsweeper[bot] wants to merge 4 commits into
mainfrom
clawsweeper/automerge-openclaw-openclaw-94685

Conversation

@clawsweeper

@clawsweeper clawsweeper Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Makes #94685 merge-ready for the ClawSweeper automerge loop.
The edit pass should inspect the live PR diff, review comments, and failing checks; rebase if needed; keep the contributor branch credited; and stop only when validation is green or an external blocker is proven.
Known failing checks:

Source

Replacement for #94685 because the source branch could not be updated.

Inherited issue-closing references from the source PR:
Fixes #84569

Original contributor: @hannesrudolph.

@clawsweeper clawsweeper Bot added agents Agent runtime and tooling extensions: codex clawsweeper:automerge Maintainer opted this PR into bounded ClawSweeper-reviewed automerge clownfish:automerge Maintainer opted this Clownfish PR into bounded ClawSweeper-reviewed automerge labels Jun 19, 2026
@clawsweeper clawsweeper Bot added the clawsweeper Tracked by ClawSweeper automation label Jun 19, 2026
@clawsweeper clawsweeper Bot mentioned this pull request Jun 19, 2026
Closed
@clawsweeper

clawsweeper Bot commented Jun 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Codex review: needs maintainer review before merge. Reviewed July 7, 2026, 8:20 AM ET / 12:20 UTC.

Summary
This PR wires embedded-runner lane-release callbacks into Codex app-server attempts, adds a focused same-session timeout regression, and includes small test/tooling cleanup changes.

PR surface: Source +2, Tests +64, Other 0. Total +66 across 5 files.

Reproducibility: yes. Source inspection shows current main still withholds plugin-owned Codex timeout release hooks, and the PR regression models a Codex timeout that never settles followed by a second same-session run.

Review metrics: 1 noteworthy metric.

  • Plugin Lane-Release Callbacks: 3 enabled in PR. The diff changes timeout-armed heartbeat, timeout release, and explicit-abort release behavior for plugin-owned Codex attempts, which is the main merge-risk boundary.

Stored data model
Persistent data-model change detected: migration/backfill/repair: src/commands/doctor-gateway-health.test.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #84569
Summary: This PR is the writable candidate fix for the Codex/plugin-owned lane-release slice of the canonical WhatsApp/Codex stall report; related PRs cover separate delivery and recovery surfaces.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🌊 off-meta tidepool
Patch quality: 🦐 gold shrimp
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Port the change onto current main while preserving current abort ownership behavior.
  • [P2] Rerun focused Codex/runner tests and required checks on the exact mergeable head.
  • [P2] Record whether focused proof is enough or live wedge proof is required.

Risk before merge

  • [P1] The live PR head is CONFLICTING/DIRTY against current main, so the fix must be ported rather than merged as-is.
  • [P2] The port must preserve current-main abort ownership behavior, including cancellation bookkeeping and replyOperation abort handling, while enabling plugin-owned timeout release.
  • [P1] The behavior intentionally releases same-session lane ownership while a timed-out Codex app-server cleanup may still be unwinding, so session-state and delivery overlap remain maintainer-visible merge risks.
  • [P1] The evidence is focused runner/Codex proof; there is no live wedged Codex app-server or live WhatsApp transport proof for the exact production failure.

Maintainer options:

  1. Port The Current-Main Fix Shape (recommended)
    Resolve the conflict by adding plugin-owned timeout release while preserving current-main abort/cancellation side effects, then rerun required checks on the exact mergeable head.
  2. Accept The Overlap Risk
    After a clean port, maintainers may intentionally accept focused proof for this runner-owned lane-release contract without live transport proof.
  3. Pause For Live Proof
    Hold the PR until a live or Crabbox Codex app-server wedge shows that releasing the lane during late cleanup does not mis-associate session state or user-visible replies.

Next step before merge

  • [P2] Manual handling is needed because the live PR head is conflicting and already carries human-review/proof-decision status; the next step is a maintainer-approved port and proof choice, not a narrow autofix finding.

Maintainer decision needed

  • Question: Should maintainers port this conflicting replacement PR to current main and accept focused runner/Codex proof for the Codex lane-release fix, or require live wedge proof before merge?
  • Rationale: The code path is a valid bug-fix candidate, but the current head conflicts with newer abort handling and the remaining question is maintainer acceptance of proof and session/message overlap risk rather than a discrete line-level defect.
  • Likely owner: vincentkoc — vincentkoc merged the adjacent lane-release contract and was the maintainer who drove the source PR automation path.
  • Options:
    • Port And Accept Focused Proof (recommended): Refresh the branch on current main, preserve current abort semantics, rerun focused and required checks, and accept the existing runner/Codex proof for this bug slice.
    • Require Live Wedge Proof: Pause merge until a live or Crabbox Codex app-server wedge demonstrates that late cleanup does not corrupt session state or channel-visible delivery.
    • Pause Or Replace: Close or replace this branch if maintainers prefer to fold the Codex lane-release fix into a broader current-main recovery effort.

Security
Cleared: No concrete security or supply-chain concern was found; the diff changes existing TypeScript timeout plumbing, tests, and small script/test cleanup without workflows, dependencies, lockfiles, permissions, or secret handling.

Review details

Best possible solution:

Refresh the PR onto current main, preserve the newer abort/cancellation semantics, keep the Codex lane-release hooks and regression coverage, and merge only after maintainers accept the focused proof and overlap risk.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main still withholds plugin-owned Codex timeout release hooks, and the PR regression models a Codex timeout that never settles followed by a second same-session run.

Is this the best way to solve the issue?

Yes conceptually, but not as the current conflicting head. Reusing the runner-owned bounded lane-release contract is the narrow owner-boundary fix; the branch must be ported onto current main without dropping newer abort semantics.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 2ba622ca3019.

Label changes

Label changes:

  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.
  • remove rating: 🐚 platinum hermit: Current PR rating is rating: 🦐 gold shrimp, so this older rating label is no longer current.

Label justifications:

  • P1: The PR targets a broken agent/channel workflow where same-session messages can remain queued behind a timed-out Codex app-server attempt.
  • merge-risk: 🚨 session-state: The diff changes when a timed-out Codex app-server attempt releases lane ownership while its cleanup may still be unresolved.
  • merge-risk: 🚨 message-delivery: The linked production report includes queued same-session work and missing user-visible channel replies.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.
  • status: 🚀 automerge armed: This PR is in ClawSweeper's automerge lane. Not applicable: This is a ClawSweeper bot replacement PR, so the external-contributor proof gate does not apply; the inherited source PR supplies focused terminal/test proof but no live wedge proof.
Evidence reviewed

PR surface:

Source +2, Tests +64, Other 0. Total +66 across 5 files.

View PR surface stats
Area Files Added Removed Net
Source 2 16 14 +2
Tests 2 68 4 +64
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 1 1 1 0
Total 5 85 19 +66

What I checked:

  • Root policy read: The root repository policy and scoped guides for extensions, agents, and scripts were read; the Codex source-inspection gate and plugin/agent boundary rules affected this review. (AGENTS.md:1, 2ba622ca3019)
  • Current main still gates plugin timeout release callbacks: Current main passes undefined timeout-armed and timeout callbacks when pluginHarnessOwnsTransport is true, so Codex plugin-owned attempts still cannot arm the runner-owned lane timeout release through this path. (src/agents/embedded-agent-runner/run.ts:2239, 2ba622ca3019)
  • Current Codex timeout records local state only: The Codex app-server turn-watch timeout handler records timeout state and message text but does not call params.onAttemptTimeout on current main. (extensions/codex/src/app-server/run-attempt.ts:1823, 2ba622ca3019)
  • Runner lane-release contract exists: CommandLaneTaskTimeoutError is the intended caller-owned release mechanism: the underlying task may still unwind, but the lane is released so queued work is not blocked forever. (src/process/command-queue.ts:23, 2ba622ca3019)
  • Native sibling already uses these callbacks: The native embedded attempt calls onAttemptTimeout when its timeout fires and onAttemptTimeoutArmed after the abort timer is scheduled, which is the sibling behavior this PR extends to Codex app-server attempts. (src/agents/embedded-agent-runner/run/attempt.ts:3538, 2ba622ca3019)
  • PR callback wiring: The PR diff removes the plugin-owned transport gating and passes startLaneProgressHeartbeat, armAttemptTimeoutRelease, and an abort lane-release callback into all harness attempts. (src/agents/embedded-agent-runner/run.ts:2006, 87a6a4b125eb)

Likely related people:

  • vincentkoc: Merged the adjacent native lane-release PR and left maintainer automation comments on the source PR that led to this replacement. (role: recent area merger and automerge reviewer; confidence: high; commits: fa4f1abb290c, dc3f6629d282; files: src/agents/embedded-agent-runner/run.ts, src/process/command-queue.ts, src/agents/embedded-agent-runner/run.compaction-loop-guard.test.ts)
  • ajwan8998: Authored the native embedded-runner heartbeat and lane-timeout release work in the merged adjacent PR that this Codex PR builds on. (role: adjacent feature author; confidence: medium; commits: 29c509571ddd, fd3a5014ad7b, 0ced1d37f031; files: src/agents/embedded-agent-runner/run.ts, src/agents/embedded-agent-runner/run.compaction-loop-guard.test.ts, src/process/command-queue.ts)
  • jalehman: Recently authored merged Codex app-server run-attempt changes near the same runtime surface. (role: recent Codex app-server area contributor; confidence: medium; commits: 81dc8805b6c9, 004825a10041, 55b494610c9b; files: extensions/codex/src/app-server/run-attempt.ts, extensions/codex/src/app-server/run-attempt.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (1 earlier review cycle)
  • reviewed 2026-06-28T21:58:17.242Z sha 87a6a4b :: needs maintainer review before merge. :: none

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 🚀 automerge armed This PR is in ClawSweeper's automerge lane. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. labels Jun 19, 2026
@clawsweeper

clawsweeper Bot commented Jun 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

🦞🔧
ClawSweeper applied a repair to this PR branch.

Repair: kept the fix on this contributor branch instead of opening a replacement PR.
Validation: pnpm check:changed; pnpm lint; pnpm check:test-types
Updated head: 87a6a4b125eb
Run: https://github.com/openclaw/clawsweeper/actions/runs/27819208129

Current state: exact-head review queued immediately; GitHub checks and the review verdict gate final merge.

Automerge progress:

  • 2026-06-19 07:30:27 UTC review queued cb09a2732cc1 (after repair)
  • 2026-06-19 07:54:52 UTC review passed cb09a2732cc1 (structured ClawSweeper verdict: pass (sha=cb09a2732cc1c93a6a9d435e2513dfa268317...)
  • 2026-06-19 11:25:27 UTC review queued cb09a2732cc1 (queued)
  • 2026-06-19 11:36:05 UTC review queued 87a6a4b125eb (after repair)

@blacksmith-sh

This comment has been minimized.

Sign in to view
@clawsweeper
clawsweeper Bot force-pushed the clawsweeper/automerge-openclaw-openclaw-94685 branch from a2edc98 to cb09a27 Compare June 19, 2026 07:30
@clawsweeper clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jun 19, 2026
@clawsweeper
clawsweeper Bot force-pushed the clawsweeper/automerge-openclaw-openclaw-94685 branch from cb09a27 to 87a6a4b Compare June 19, 2026 11:36
@openclaw-barnacle openclaw-barnacle Bot added scripts Repository scripts commands Command implementations labels Jun 19, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. clawsweeper:human-review Needs maintainer review before ClawSweeper can continue labels Jun 19, 2026
@clawsweeper

clawsweeper Bot commented Jun 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

🦞✅
ClawSweeper is pausing this repair loop for human review.

Source: clawsweeper[bot]
Reason: - [P2] Manual handling is needed because the live PR head is conflicting and already carries human-review/proof-decision status; the next step is a maintainer-approved port and proof choice, not a narrow autofix finding.; Cleared: No concrete security or supply-chain concern was found; the diff changes existing TypeScript timeout plumbing, tests, and small script/test cleanup without workflows, dependencies, lockfil... (sha=87a6a4b125eb72777c0662f4705ff3403fd449eb)

Why human review is needed:
This item has security-sensitive risk. ClawSweeper is pausing instead of making an autonomous change that could affect trust, credentials, permissions, or exposure.

What the maintainer can do as a next step:
If the maintainer accepts the current risk and wants ClawSweeper to continue merge gates, comment @clawsweeper approve. If the security-sensitive detail still needs changes, describe the safe path or push the fix, then comment @clawsweeper automerge. If the risk should not be automated, keep the PR paused for manual review or comment @clawsweeper stop.

I added clawsweeper:human-review and left the final call with a maintainer.

@clawsweeper clawsweeper Bot mentioned this pull request Jun 20, 2026
Closed
@clawsweeper clawsweeper Bot removed the rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. label Jun 21, 2026
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. and removed proof: sufficient ClawSweeper judged the real behavior proof convincing. labels Jun 21, 2026
@github-actions github-actions Bot mentioned this pull request Jun 21, 2026
Open
@clawsweeper clawsweeper Bot mentioned this pull request Jun 21, 2026
Closed
@clawsweeper clawsweeper Bot added status: needs maintainer proof decision A ClawSweeper-authored PR needs a maintainer proof capture or override decision. and removed status: 🚀 automerge armed This PR is in ClawSweeper's automerge lane. labels Jun 23, 2026
@clawsweeper

clawsweeper Bot commented Jun 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

ClawSweeper status: this ClawSweeper-authored replacement PR is blocked on real behavior proof.

Reviewed head: 87a6a4b125eb72777c0662f4705ff3403fd449eb
Proof status: not_applicable
Updated: 2026-07-01T11:06:17.643Z

Maintainer decision needed:

  • capture real behavior proof for this head
  • apply proof: override if the proof requirement should be waived
  • pause or close the replacement if proof should not be pursued

@clawsweeper clawsweeper Bot added status: 🚀 automerge armed This PR is in ClawSweeper's automerge lane. status: needs maintainer proof decision A ClawSweeper-authored PR needs a maintainer proof capture or override decision. and removed status: needs maintainer proof decision A ClawSweeper-authored PR needs a maintainer proof capture or override decision. status: 🚀 automerge armed This PR is in ClawSweeper's automerge lane. labels Jun 26, 2026
@clawsweeper clawsweeper Bot mentioned this pull request Jun 30, 2026
Closed
@clawsweeper clawsweeper Bot added status: 🚀 automerge armed This PR is in ClawSweeper's automerge lane. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed status: needs maintainer proof decision A ClawSweeper-authored PR needs a maintainer proof capture or override decision. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jul 1, 2026
@barnacle-openclaw

barnacle-openclaw Bot commented Jul 22, 2026

Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@barnacle-openclaw barnacle-openclaw Bot added the stale Marked as stale due to inactivity label Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

agents Agent runtime and tooling clawsweeper:automerge Maintainer opted this PR into bounded ClawSweeper-reviewed automerge clawsweeper:human-review Needs maintainer review before ClawSweeper can continue clawsweeper Tracked by ClawSweeper automation clownfish:automerge Maintainer opted this Clownfish PR into bounded ClawSweeper-reviewed automerge commands Command implementations extensions: codex merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. merge-risk: 🚨 session-state 🚨 May lose, corrupt, stale, or mis-associate session, agent, or context state. P1 High-priority user-facing bug, regression, or broken workflow. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. scripts Repository scripts size: S stale Marked as stale due to inactivity status: 🚀 automerge armed This PR is in ClawSweeper's automerge lane.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

WhatsApp session stalls on long model_call: incomplete turn with payloads=0, reply never delivered

0 participants