Zombie running job blocks all subsequent task calls when process crashes

Summary

When a Codex task process crashes or is killed externally, its job record remains in status: "running" permanently. All subsequent task calls in the same Claude session are rejected with:

Task {job-id} is still running. Use /codex:status before continuing it.

This effectively blocks all Codex usage in that session until the user manually fixes the state file.

Steps to Reproduce

Start a Codex task via /codex:rescue (foreground or background)
The task process crashes or is killed (e.g., OOM, signal, Codex app-server timeout)
Job record in state.json remains status: "running" with a dead PID
All subsequent task calls in the same session fail immediately with "Task is still running"

Root Cause

In codex-companion.mjs, resolveLatestTrackedTaskThread() checks for active tasks:

const activeTask = visibleJobs.find(
  (job) => job.jobClass === "task" && 
  (job.status === "queued" || job.status === "running")
);
if (activeTask) {
  throw new Error(`Task ${activeTask.id} is still running.`);
}

This only checks the status field in the job record. It does not verify whether the process (stored in job.pid) is actually alive.

Evidence from Production

Observed in plugin version 1.0.3. A session had 6 consecutive failed task attempts, all blocked by the same zombie job:

task-mntm7f2j  status=running  pid=82311  (process DEAD)
task-mntmbrox  status=failed   error="Task task-mntm7f2j is still running"
task-mntmbvx4  status=failed   error="Task task-mntm7f2j is still running"
task-mntmclwl  status=failed   error="Task task-mntm7f2j is still running"
task-mntme0qz  status=failed   error="Task task-mntm7f2j is still running"
task-mntmg2an  status=failed   error="Task task-mntm7f2j is still running"
task-mntnft5z  status=failed   error="Task task-mntm7f2j is still running"

The PID was confirmed dead via kill(82311, 0) → ProcessLookupError.

Suggested Fix

In resolveLatestTrackedTaskThread(), before throwing, verify the PID is alive. If the process is dead, mark the job as failed and continue:

const activeTask = visibleJobs.find(
  (job) => job.jobClass === "task" && 
  (job.status === "queued" || job.status === "running")
);

if (activeTask) {
  // Check if the process is actually alive
  const pid = activeTask.pid;
  let processAlive = false;
  if (pid) {
    try {
      process.kill(pid, 0);
      processAlive = true;
    } catch (e) {
      processAlive = (e.code === 'EPERM'); // exists but no permission
    }
  }

  if (processAlive) {
    throw new Error(`Task ${activeTask.id} is still running.`);
  }

  // Process is dead — mark as failed (zombie cleanup)
  upsertJob(workspaceRoot, {
    id: activeTask.id,
    status: "failed",
    phase: "failed",
    pid: null,
    errorMessage: "Process exited without updating job status.",
    completedAt: new Date().toISOString()
  });
}

Environment

Plugin version: 1.0.3
Codex CLI: @openai/codex (latest)
Claude Code: 2.1.89
Platform: macOS (Darwin 24.6.0, arm64)
Node.js: v22.12.0

Fix openai#202: zombie job blocks subsequent task calls when process …

fix: handle zombie jobs when process crashes but status remains "runn…

Fix implemented in PR #218. The root cause was that resolveLatestTrackedTaskThread() only checked job.status without verifying the process was actually alive.

Additional Root Cause: Foreground `await state.completion` hangs forever on IPC disconnect

The zombie job state file (described in the original issue) is one symptom, but there's a more fundamental bug that causes foreground tasks to hang indefinitely — the Claude Code session becomes completely unresponsive with no way to recover.

Root Cause

In scripts/lib/codex.mjs, captureTurn() (line ~600):

return await state.completion;

state.completion is a standalone Promise created in createTurnCaptureState(). It is only resolved when a turn/completed notification arrives via the IPC notification handler.

When the app-server process dies (OOM, SIGKILL, broker socket disconnect, etc.):

handleExit() fires in app-server.mjs → rejects all client.pending RPC requests ✅
client.exitPromise resolves ✅
state.completion is never resolved or rejected ❌ — it's a completely separate Promise that nobody touches on connection death
await state.completion hangs forever → Node.js process never exits → Bash call never returns → Claude Code is stuck

captureTurn()
  ├── client.setNotificationHandler(...)   ← listens for turn/completed
  ├── await startRequest()                 ← RPC request (has pending reject protection)
  └── return await state.completion        ← standalone Promise, no reject on disconnect
                                              ↑
                                     handleExit() only rejects client.pending (RPCs),
                                     does NOT touch state.completion

Suggested Fix

Race state.completion against client.exitPromise so that IPC disconnect immediately unblocks the caller:

--- a/scripts/lib/codex.mjs
+++ b/scripts/lib/codex.mjs
@@ -597,7 +597,17 @@ async function captureTurn(client, threadId, startRequest, options = {}) {
       completeTurn(state, response.turn);
     }

-    return await state.completion;
+    // Race completion against client death to avoid hanging forever
+    // when the app-server process crashes (IPC drops but state.completion
+    // is never resolved/rejected — see #202, #222).
+    const clientDeath = client.exitPromise.then(() => {
+      if (!state.completed) {
+        const err = client.exitError ?? new Error(
+          "codex app-server connection closed unexpectedly during turn."
+        );
+        state.rejectCompletion(err);
+      }
+    });
+
+    return await Promise.race([state.completion, clientDeath]);
   } finally {
     clearCompletionTimer(state);
     client.setNotificationHandler(previousHandler ?? null);

This is complementary to the PID liveness check proposed in the original issue — that fix handles background zombie jobs (stale state file), while this fix handles foreground hangs (Promise never settling). Both are needed for full coverage.

Impact

Without this fix, any foreground Codex task (/codex:rescue, /codex:review) will permanently freeze the Claude Code session if the app-server dies mid-turn. The user has no way to recover except killing the entire Claude Code process.

fix: reconcile stale 'running' jobs when tracked PID is dead

fix(broker): P0 stability — idle timeout + PID liveness + shutdown ti…

fix(state): reap stale running jobs via OS-level PID liveness check (o…

Duplicate of #222.

Navigation Menu

Repository navigation

Zombie running job blocks all subsequent task calls when process crashes #202

Summary

Steps to Reproduce

Root Cause

Evidence from Production

Suggested Fix

Environment

Additional Root Cause: Foreground `await state.completion` hangs forever on IPC disconnect

Root Cause

Suggested Fix

Impact

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Zombie running job blocks all subsequent task calls when process crashes #202

Description

Summary

Steps to Reproduce

Root Cause

Evidence from Production

Suggested Fix

Environment

Activity

D2758695161 commented on Apr 14, 2026

2488583886 commented on Apr 14, 2026

Additional Root Cause: Foreground await state.completion hangs forever on IPC disconnect

Root Cause

Suggested Fix

Impact

dkundel-openai commented on Aug 11, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Additional Root Cause: Foreground `await state.completion` hangs forever on IPC disconnect