Skip to content

[Bug]: Codex thread stays "running" forever after the Codex app-server process dies; only a server restart recovers it #22422

Closed

Description

@blueworkslabs

Before submitting

  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.

Area

apps/server

Steps to reproduce

  1. Run the t3 server (v0.1.0) with the Codex provider.
  2. Start a Codex thread with a multi-step task that takes a few minutes, e.g. "Run these 12 shell commands one at a time (sleep 15 && echo step-N) and report each".
  3. While the turn is mid-work, kill the Codex app-server process (kill -9 <pid>).
  4. Watch the thread, then send it another message, then try Interrupt and a fresh message.

Observed 2 of 2 times: once with a message steered into the turn 30 s before the kill, once with no input at all after the turn started. In the same runs, Claude threads on the same server whose CLI process was killed failed their run and recovered as expected.

Expected behavior

The run fails with a provider runtime error, the thread goes idle, and the next message starts a new Codex app-server (the same path a Claude thread takes when its CLI dies).

Actual behavior

  • The run stays running with no Codex process behind it. We watched for 40 minutes; it never failed on its own.
  • New messages to the thread are accepted as steers and then fail against the dead transport (server log: steer failures). Nothing progresses.
  • Interrupt marks the run interrupted and clears active/pending state, but the next turn on the same thread fails immediately at provider start and no new app-server is spawned.
  • Only restarting the t3 server recovers it: the same thread then runs on a new app-server and completes normally.

Where it seems to come from (v0.1.0, source reading):

  • The protocol layer does notice the exit: handleTermination fails pending requests and calls an optional onTermination hook (packages/effect-codex-app-server/src/protocol.ts:196-217).
  • But CodexAppServerClientOptions has no onTermination (packages/effect-codex-app-server/src/client.ts:22-28), the client never passes one to the protocol (client.ts:198-205), and CodexAdapterV2 opens the client without one.
  • The adapter's event stream is Stream.fromEffectRepeat(Queue.take(events)) (apps/server/src/orchestration-v2/Adapters/CodexAdapterV2.ts:6518) and nothing ends or fails that queue when the process dies.
  • ProviderSessionManager only releases a session (and terminalizes its runs) when the event stream ends or fails (ProviderSessionManager.ts:2080-2124), so the dead session stays registered and the turn waits for a turn/completed that never comes.

Possible fix: thread an onTermination option through the Codex client to the protocol, and in the adapter fail the session's event queue on termination, so the session manager releases the session with a runtime error the way it does for Claude. A regression test could kill the app-server mid-turn and assert the run fails and the next turn spawns a new process.

Related: #15708, #15358, #21040.

Impact

Minor bug or occasional failure

Version or commit

v0.1.0 (81493ef); relevant code unchanged on main as of 2026-10-20.

Environment

Ubuntu 24.04 (VM), Node 24.22, Codex CLI 0.163.0, Claude Code 2.1.304, headless t3 server driven over its MCP server.

Activity

  1. t3dotgg commented on Oct 31, 2026

    @t3dotgg
    Member

    Note

    🤖 Claude Opus 5.5 responding on behalf of Theo

    Closing: this is covered by #26188, which landed. Reopen if an important part is still missing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions