Codex CLI 0.150.1 still hangs after final output

🤖 I think #150 may have been closed too early. I upgraded Buzz's security review workflow to Codex CLI 0.150.1, but we're still seeing the same post-output hang: Codex prints the final JSON and token count, then the action never exits or emits its end marker.

Our setup is:

GitHub-hosted ubuntu-latest
openai/codex-action@86365089eb2b84e0a8fb0717b304f8bdcb13b20e (v1.12)
codex-version: '0.150.1'
gpt-5.6-sol with model_reasoning_effort=max
safety-strategy: drop-sudo
permission-profile: ':read-only'

The public workflow is here.

Since upgrading, 33 of 39 real review executions have stayed stuck in the Codex action until the job timeout cancelled them. Two light reviews completed and four were superseded by newer pushes. The behavior seems workload-sensitive rather than completely deterministic: the same branch and pinned versions had a clean run that finished in about three minutes, followed by a run that printed its final output but never returned.

A heavier example is run 33114428326. The Codex step started at 20:45:38Z; its completed JSON and 295,198 token count were visible by 21:14:24Z, but the action still did not exit. Run 33276058348 shows the same overall shape.

I also isolated the wrapper lifecycle in this GitHub-hosted run. It invoked the exact v1.12 production run-codex-exec path with a deterministic fake Codex process. The process wrote valid output to the wrapper-provided --output-last-message file and then kept inherited stdio open. The wrapper stayed blocked until the caller timed out the step, after which the next step could immediately read the completed file. This is synthetic, but it reproduces the lifecycle seam independently of the API/model behavior.

The v1.12 source still spawns with inherited stdout/stderr and waits on the child's close event, so a descendant retaining those descriptors can keep the action alive after the direct child is done. #151 looks like the wrapper-side fix: it uses private pipes, completes on the direct child's exit, tears down the streams, and adds a regression test for a descendant holding stdio open.

Could someone take another look at the post-0.150.0 behavior and review #151, or an equivalent wrapper fix?

fix(ci): salvage Codex review output on PTY-shutdown hang (#7042)

Second organization seeing this, on effectively the same configuration: GitHub-hosted ubuntu-latest, openai/codex-action@86365089eb2b84e0a8fb0717b304f8bdcb13b20e (v1.12), gpt-5.6-sol, safety-strategy: drop-sudo, sandbox: read-only, with output-schema-file. We hit it on codex-version 0.147.0 and 0.150.0 alike, so it does not look specific to the CLI version.

At least five hangs under our current 50 minute job timeout, plus two earlier ones under a shorter timeout that look the same. Every one was cancelled at exactly 55 minutes, which is our job timeout-minutes: 50 plus the runner's 5 minute cancellation grace, with the Codex step still in_progress and no conclusion recorded.

Two things made this expensive to diagnose, and both are worth knowing if you are trying to collect evidence:

A step-level timeout-minutes on the action does nothing. timeout-minutes does not apply to a step that uses: a composite action, so the job timeout is the only backstop. We had 40 minutes set on the step and watched it sail past 44 minutes.
Because the job is force-killed rather than completing, the runner never uploads the step log. gh run view --log returns log not found for every hung job, so there is no archived evidence at all. We only saw the tail by opening the live log in a browser while the step was still hanging: the final JSON, tokens used, the token count, and then nothing, with no ##[end-action] marker. That matches v1.12: Linux run never returns after the turn completes; job dies on timeout with the output file already written #150 exactly.

It is workload-sensitive for us too, but not in the way we first assumed. Our smallest hang was a one file, roughly 90 line diff, on a pull request that an earlier run had reviewed successfully in 2m26s. Bigger reviews hang more often, but nothing is immune.

One data point in support of #151. I checked what the innermost process actually sees through the v1.12 chain: running sudo -> sh -> setpriv -> env -> <cmd> in a container with the outer stdout on a pipe, the innermost process has fd 1 and fd 2 on the runner's pipes, not on a pty. So a surviving descendant does retain the runner's log transport descriptors, which is consistent with the root cause described in #151. I also ruled out sudo's use_pty default, which is on in Ubuntu 24.04's sudoers: sudo returns immediately even when its direct child exits leaving a grandchild holding the inherited stdio.

Our mitigation until #151 lands, in case it helps anyone else. A background watchdog armed in the step before the action records what Codex's fd 1 points at while Codex is running. Once the --output-last-message file is non-empty and Codex has had a short grace period to exit on its own, it kills every process still holding that pipe, which lets the step complete. We also read the answer back from that output file, so even a step that had to be killed still publishes its result instead of discarding a finished review.

Happy to test #151 against our workload if that would be useful.

@addo47

Third organization, and the useful part is that our configuration is leaner than either report above — which narrows the repro surface.

Every config in this thread and in #150 sets output-schema-file, and #150 also sets output-file and prompt-file. We set none of them. Our entire step is:

uses: openai/codex-action@v1          # floating -> 86365089 (v1.12)
with:
  openai-api-key: ${{ secrets.OPENAI_API_KEY }}
  responses-api-endpoint: ${{ secrets.OPENAI_BASE_URL }}/v1/responses
  model: gpt-5.5
  sandbox: read-only
  prompt: |                            # inline, not prompt-file

No safety-strategy (so the drop-sudo default), no permission-profile, no output plumbing at all. So the hang does not require the --output-last-message path, and it is not specific to gpt-5.6-* or to max/xhigh effort. That's consistent with the wrapper-lifecycle diagnosis in #151 rather than anything output- or model-shaped.

One difference worth flagging in case it matters: we go through a responses-api-endpoint proxy, which is the setup in #108.

Our numbers

287 runs of this workflow since 2026-04-28.

window	runs	hangs (>15 min)	max
before `@v1` moved (< 2026-08-20T23:38:51Z)	186	0	10 min
v1.12 live, all-light runs	20	0	3 min
from 2026-08-24T22:25Z	82	18	64 min

Same shape as everyone else: final message and token count print, no ##[end-action], step idles until the job is killed.

A tight natural experiment on the intermittency

On 2026-08-31, the same PR on the same pinned everything:

time (UTC)	result
14:38	success, 2 min
14:45	hung, 63 min
15:05	success, 3 min

So a green run clears nothing, and anyone bisecting this needs many runs per cell rather than one.

A confound worth warning others about

We nearly shipped the wrong fix, and the trap is easy to fall into given codex-version defaults to empty.

@openai/[email protected] published 2026-08-24T00:32Z, 22 hours before our first hang. That fit our data cleanly. It is also unfalsifiable from our data alone: our 20 pre-boundary runs on v1.12 were all light (max 3 min), and since the failure is workload-sensitive, the CLI-version boundary and the action-version boundary are perfectly confounded in our history. Both explain our evidence equally well.

What breaks the tie is evidence we don't own — @addo47's hang on 0.147.0 above, which predates the release we suspected, and @uilton-oliveira's 145/145 on v1.11 vs 69/74 on v1.12 in #150 with model and effort held fixed. Posting this so the next person who spots the npm timing doesn't chase it as far as we did.

We've pinned to v1.11 (52fe01ec) and added a job-level timeout-minutes. Seconding @addo47's warning on that: step-level timeout-minutes does nothing for a step that uses: a composite action.

Happy to test #151 against our workload — our config exercises a path the other two reporters don't.

Upstream sync: block/buzz → main (13 commits) (#11)

chore(ci): lower Codex security review effort (#7179)

Duplicate of #160 ?

fix(ai-review): pin the action to v1.11, and stop relying on the defa…

ci: add an AI code review job (Codex + GLM-5.3) (#37)

fix(ci): salvage Codex review output on PTY-shutdown hang (block#7042)

chore(ci): lower Codex security review effort (block#7179)

Navigation Menu

Repository navigation

Codex CLI 0.150.1 still hangs after final output #169

Our numbers

A tight natural experiment on the intermittency

A confound worth warning others about

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Codex CLI 0.150.1 still hangs after final output #169

Description

Activity

addo47 commented on Aug 31, 2026

guohai commented on Sep 1, 2026

Our numbers

A tight natural experiment on the intermittency

A confound worth warning others about

isolomatov-cfa commented on Sep 2, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions