Skip to content

openai-codex Connection Reliability Issues #4945

Open

Description

@liushuaiiu

Summary

openai-codex / gpt-5.5 sometimes leaves the interactive TUI stuck on Working... with no streamed text, no tool call, and no visible error. The only way to recover is pressing Escape, which records an aborted assistant turn.

This has happened repeatedly over the last couple of days in normal interactive use.

Environment

  • pi: 0.75.5
  • Node: v22.22.1
  • Provider/model: openai-codex / gpt-5.5
  • Thinking level: xhigh
  • No explicit transport setting in user settings
  • Default tools/extensions enabled

Observed behavior

When the issue occurs:

  • TUI keeps showing Working... for minutes.
  • No assistant text is streamed.
  • No tool call is emitted.
  • Pressing Escape aborts the turn.
  • The saved session entry for that assistant turn has:
{
  "role": "assistant",
  "stopReason": "aborted",
  "content": [],
  "usage": {
    "input": 0,
    "output": 0,
    "cacheRead": 0,
    "cacheWrite": 0,
    "totalTokens": 0
  }
}

I saw this pattern multiple times, including cases where the previous turn had completed normally and the next user message then hung before any provider usage was recorded.

This looks different from a long reasoning turn: there is no usage, no partial reasoning/text, and no tool call.

Expected behavior

If the provider/transport stalls before the first event, pi should eventually surface a timeout/transport error or retry in a bounded way, instead of keeping the TUI in Working... indefinitely until the user manually aborts.

Suspected area

From the installed 0.75.5 package:

  • SettingsManager.getTransport() returns "auto" when no setting is present.
  • The docs/settings table says the default is "sse", so there may also be a docs/runtime default mismatch.
  • For openai-codex-responses, transport=auto attempts WebSocket first.
  • retry.provider.timeoutMs appears to be passed into streamSimple(), but the openai-codex-responses implementation does not seem to apply it to the Codex fetch/WebSocket wait path in the same way as SDK-based providers.
  • The WebSocket event loop can wait for the first message/completion without an obvious idle timer.

So the likely failure mode is: Codex WebSocket/transport waits before the first event; no assistant event is produced; interactive UI keeps showing Working...; Escape finally records an aborted zero-usage turn.

Suggested fix / mitigation

Possible fixes:

  1. Add a hard idle timeout for openai-codex-responses WebSocket and SSE stream waits, especially before the first event.
  2. Ensure retry.provider.timeoutMs or httpIdleTimeoutMs applies consistently to this provider path.
  3. If auto is intended as the runtime default, update docs; if sse is intended, adjust SettingsManager.getTransport().
  4. Optionally show a clearer status/error when a provider turn has produced zero events for a long time.

Local workaround I am considering, but have not applied yet:

{
  "transport": "sse",
  "httpIdleTimeoutMs": 120000
}

I can provide more sanitized session metadata if helpful, but I avoided attaching raw session logs because they contain private conversation/tool context.

Activity

  1. github-actions commented on May 24, 2026

    @github-actions
    Contributor

    This issue was auto-closed. All issues from new contributors are auto-closed by default.

    Maintainers review auto-closed issues daily and reopen worthwhile ones. Issues that do not meet the quality bar in CONTRIBUTING.md will not be reopened or receive a reply.

    If a maintainer replies lgtmi on one of your issues, your future issues will stay open. If a maintainer replies lgtm, your future issues and PRs will stay open.

    See CONTRIBUTING.md.

  2. liushuaiiu commented on May 24, 2026

    @liushuaiiu
    Author

    Additional repro from the same environment/session.

    It happened again after this issue was filed.

    Environment is unchanged:

    • pi: 0.75.5
    • Node: v22.22.1
    • provider/model: openai-codex / gpt-5.5
    • thinking: xhigh
    • no explicit transport setting, so runtime default is auto

    Timeline:

    • User submitted a normal public GitHub repo lookup prompt.
    • Pi showed Working... for about 2 minutes.
    • No assistant text streamed.
    • No tool call was emitted.
    • User pressed Escape.
    • Session recorded an aborted assistant turn with zero usage.

    Sanitized session evidence:

    {
      "role": "assistant",
      "content": [],
      "stopReason": "aborted",
      "usage": {
        "input": 0,
        "output": 0,
        "cacheRead": 0,
        "cacheWrite": 0,
        "totalTokens": 0
      }
    }

    The immediately following retry of essentially the same user prompt succeeded and emitted tool calls, so this was not caused by the prompt itself or a long-running tool.

    This supports the narrower failure mode described above: a pre-stream / zero-event hang where no assistant event, tool call, or usage has been recorded yet. This seems different from the previously discussed after_message_stream_start WebSocket close case, where retry may be unsafe because partial assistant state may already exist.

    For this zero-event case, a bounded idle timeout or visible transport error should be safe because no assistant content/tool call has entered the session state yet.

  3. mitsuhiko commented on May 24, 2026

    @mitsuhiko
    Member

    Can you /share an effected session with us?

  4. liushuaiiu commented on May 24, 2026

    @liushuaiiu
    Author

    I cannot share the raw affected session because it contains private local/project context, but I created a sanitized excerpt from a real affected session:

    https://gist.github.com/liushuaiiu/48f8b4b7a3298932e90f71ceb986cd83

    What it includes:

    • README.md: timeline, environment, and sanitization notes
    • pi-issue-4945-affected-session.redacted.jsonl: redacted JSONL excerpt
    • pi-issue-4945-affected-session.redacted.html: redacted HTML export

    This excerpt preserves the relevant failure shape:

    • first user turn: normal public GitHub repo lookup prompt
    • assistant turn after Escape: stopReason: "aborted", empty content, zero usage
    • immediate retry in the same session succeeds and emits normal tool calls / final response

    Redactions applied before upload:

    • local paths replaced with placeholders (<HOME>, <PI_AGENT_DIR>, <TMP_REPO>, etc.)
    • provider response IDs and encrypted thinking signatures removed
    • assistant thinking blocks removed
    • long tool outputs truncated
    • scanned for obvious local paths and credential-like patterns before upload
  5. self-assigned this
    on May 24, 2026
  6. mitsuhiko commented on May 24, 2026

    @mitsuhiko
    Member

    Did you validate that setting the mode to sse resolves the issue for you? Would be good to understand better what is going on here.

  7. liushuaiiu commented on May 24, 2026

    @liushuaiiu
    Author

    Not yet, sorry.

    I have not validated sse yet. In the current setup/session state I cannot safely switch the transport/mode without disturbing the affected environment I am using to collect evidence, and I had intentionally kept the local config unchanged while waiting for upstream feedback.

    So the current repro data is still from the default/current transport path only.

    I can run a separate clean test with sse and report back whether it suppresses the zero-event / zero-usage Working... hang, if that is the preferred next data point.

  8. mitsuhiko commented on May 24, 2026

    @mitsuhiko
    Member

    The issue is that I cannot reproduce this at all, so I'm entirely in the dark. Anything that can narrow this down would help.

  9. liushuaiiu commented on May 24, 2026

    @liushuaiiu
    Author

    Additional narrowing data. I now have two related but distinct observations.

    1. Long-context interactive sessions: the tools are usually not the slow part

    In another affected interactive session, the config was again:

    • openai-codex / gpt-5.5
    • thinking level: xhigh
    • large session context, roughly 2.46MB JSONL

    The repeated ~5 minute stalls happened after tool results had already returned, before the next assistant action.

    A representative example:

    11:57:22 assistant called recap_project(pi-memory)
    11:57:27 toolResult returned
    12:02:37 assistant called recap_project(pi-shelf)
    

    That is ~309s where no tool was running.

    Similar gaps also occurred after other tool results, not only recap_project:

    edit result -> ~313s before next assistant action
    bash result -> ~325s before next assistant action
    read result -> ~313s before next assistant action
    

    Tool durations in that session were much smaller:

    recap_project     max ~10.1s, avg ~2.7s
    memory_recall     max ~5.9s
    memory_save       max ~5.6s
    shelf_get/search  <1s
    read/edit         essentially immediate
    subagent start    immediate; child work completed in background
    

    So this part looks less like a specific Pi tool/local extension hanging, and more like model/provider waiting after tool completion. The pattern is amplified by gpt-5.5 + xhigh + long context.

    2. Transport test: forcing sse did not eliminate pre-output stalls

    I also ran a separate clean print-mode batch test with a tiny prompt:

    请只回答 OK,不要使用工具。
    

    Environment:

    • pi: 0.75.5
    • Node: v22.22.1
    • provider/model: openai-codex / gpt-5.5
    • thinking: xhigh
    • mode: pi -p
    • each attempt wrapped with timeout 150s

    Default / auto transport

    Command shape:

    timeout 150s pi -p --model openai-codex/gpt-5.5 --thinking xhigh "请只回答 OK,不要使用工具。"

    Result:

    attempts 1-14: returned OK, exit 0, usually ~6-9s
    attempt 15: no output before timeout; killed at 150s, exit 124
    

    Excerpt:

    [pi-4945] default print attempt 14 start 2026-05-24T12:22:06+00:00
    OK
    [pi-4945] default print attempt 14 exit=0 end 2026-05-24T12:22:14+00:00
    [pi-4945] default print attempt 15 start 2026-05-24T12:22:16+00:00
    [pi-4945] default print attempt 15 exit=124 end 2026-05-24T12:24:46+00:00
    [pi-4945] TIMEOUT/HANG suspected on default attempt 15
    

    Explicit sse

    I used a separate working directory with project settings:

    { "transport": "sse" }

    I also verified locally that Pi's settings manager sees transport = sse for that directory.

    Same command shape:

    timeout 150s pi -p --model openai-codex/gpt-5.5 --thinking xhigh "请只回答 OK,不要使用工具。"

    Result:

    attempts 1-11: returned OK, exit 0, usually ~4-7s
    attempt 12: no output before timeout; killed at 150s, exit 124
    

    Excerpt:

    [pi-4945] sse print attempt 11 start 2026-05-24T12:32:02+00:00
    OK
    [pi-4945] sse print attempt 11 exit=0 end 2026-05-24T12:32:07+00:00
    [pi-4945] sse print attempt 12 start 2026-05-24T12:32:09+00:00
    [pi-4945] sse print attempt 12 exit=124 end 2026-05-24T12:34:39+00:00
    [pi-4945] TIMEOUT/HANG suspected on sse attempt 12
    

    Caveat: this print-mode test is not exactly the same as the interactive /share failure shape. The timed-out print-mode attempts did not create a completed session file, whereas the original interactive repro records an aborted assistant turn with empty content and zero usage after pressing Escape.

    Still, this suggests that simply forcing transport: "sse" does not fully avoid the pre-output / pre-first-event stall in my environment.

    Current hypothesis

    There may be two related symptoms:

    1. Long-context + xhigh thinking can produce multi-minute model-side waits after tool results.
    2. Even tiny prompts can intermittently stall before any assistant output/event, under both default/auto and explicit sse.

    This makes a specific local Pi tool/extension hang less likely. It seems more like provider/model-side waiting or pre-first-event streaming behavior with openai-codex/gpt-5.5, with Pi currently only showing generic Working....

    Potentially useful Pi-side diagnostics would be to distinguish these states in logs/UI/session data:

    • waiting for provider first event
    • provider stream started
    • tool execution running
    • tool finished, waiting for next model response
  10. mitsuhiko commented on May 24, 2026

    @mitsuhiko
    Member

    I'm wondering if you are connecting through some proxy that is misbehaving. Could you share more details how you connect?

  11. liushuaiiu commented on May 25, 2026

    @liushuaiiu
    Author

    Thanks. Here are more details about my connection path:

    • My Pi instance is running on an AWS server located in Japan.
    • I authenticate/use the model through OpenAI Codex login.
    • I access the AWS server from my home computer over SSH and run Pi there.
    • My home/local computer has Tailscale installed, but I only use it to reach other local devices for separate workflows. I am not using a Tailscale route/path for SSH into the AWS server.
    • I have not intentionally configured an HTTP/HTTPS proxy for Pi or for the OpenAI Codex traffic.

    This issue has started happening frequently in the last 3–4 days. There have been no major changes to my network environment during that period.

  12. 51616 commented on May 25, 2026

    @51616

    Thanks. Here are more details about my connection path:

    * My Pi instance is running on an AWS server located in Japan.
    
    * I authenticate/use the model through OpenAI Codex login.
    
    * I access the AWS server from my home computer over SSH and run Pi there.
    
    * My home/local computer has Tailscale installed, but I only use it to reach other local devices for separate workflows. I am not using a Tailscale route/path for SSH into the AWS server.
    
    * I have not intentionally configured an HTTP/HTTPS proxy for Pi or for the OpenAI Codex traffic.
    

    This issue has started happening frequently in the last 3–4 days. There have been no major changes to my network environment during that period.

    I'm also facing the same problem running pi on my vps in Tokyo (non-aws local provider). Wonder if it has something to do with the region.

    To add some anecdotes, the same problem occurs even gpt-5.5. with low reasoning or 5.3-codex-spark model. Using codex has been smooth in the same period that I've been having this problem in pi. Spawning a new pi instance while another pi instance is hanging does somewhat consistently avoid this problem on the new pi process.

    Plus, the "hang" doesn't only happen during the zero-even phase. Many times pi hangs after the first few tool calls.

    Edit2: another anecdote to add is that this also occurred with EU ip as well. By the time i'm posting this I no longer encounter the problem on new sessions. Old sessions with this "slow" behavior is still slow. I suspect that this has something to do with recent cache validation bug https://x.com/thsottiaux/status/2058280452851638313

    Edit: the problem seems to go away now?

  13. 70 remaining items

  14. Blue-B commented on Jul 15, 2026

    @Blue-B

    I am seeing a separate failure mode on the current release.

    • Pi 0.80.7
    • provider/model: openai-codex/gpt-5.6-terra
    • transport: auto
    • 2026-07-15 05:11:19Z and 2026-07-15 05:13:02Z
    • chatgpt.com/backend-api/codex/responses returned the Cloudflare HTML 520 page
    • Ray IDs: a1b63e92080180a1, a1b64591d9f188f8
    • Cloudflare Location: Seoul

    This was not a persistent local session failure. I copied the persisted session, kept its session ID, and sent a follow-up with a fresh npm-installed 0.80.7 using no extensions or settings. It completed in 8 seconds after the incident had cleared.

    There is an independent report 41 minutes later of Unexpected status 520 on the same endpoint:
    https://x.com/JacobRothfield/status/2077268771547574412

    Pi's Codex low-level retry list currently includes 429, 500, 502, 503, and 504, but not 520. Could 520 be handled as a bounded transient failure, perhaps one or two retries? That would not cure the origin problem, but it would stop a brief Cloudflare failure from ending the turn.

  15. hcxcomeon commented on Sep 7, 2026

    @hcxcomeon

    它已经变得完全无法使用了。

    图像

    I encountered the same problem

  16. chigkim commented on Sep 12, 2026

    @chigkim

    It seems it's broken again? Now it stalls pretty often.

  17. jparradog commented on Sep 17, 2026

    @jparradog

    Additional Pi 0.85.1 data point from WSL2 with Node.js 24.19.0, using openai-codex with gpt-6-astra and gpt-5.6-sol.

    A local session-log scan found 82 provider_transport_failure diagnostics across 64 sessions and 18 working directories:

    • 69 occurred after message_stream_start, 13 before it.
    • Post-start request sizes ranged from 97,632 to 2,767,402 bytes, median 607,917 bytes.
    • The post-start records report configuredTransport: "auto", eventsEmitted: true, and phase: "after_message_stream_start".

    One long session at about 85% context displayed repeated WebSocket error messages, followed by Auto-compaction failed: Turn prefix summarization failed: This operation was aborted, then Operation aborted. The terminal was hosted in Herdr, but the persisted diagnostics and stack originate in Pi's openai-codex-responses transport. No proxy environment variables were configured.

    I also ran three small core-only checks with extensions, tools, skills, context files, and session storage disabled. All three returned OK in 6 to 7 seconds, so I could not reproduce this with a small context. This is recurrence evidence, not a deterministic minimal reproduction. Large requests or long streams appear correlated, but the physical source of the disconnect remains unknown.

    One observability gap: the retry and compaction failures visible in the TUI were not persisted with a WebSocket close code or reason, which makes provider, network, and runtime failures difficult to distinguish.

    AI-assisted diagnosis; evidence and wording reviewed before posting.

  18. j57n-3co-x-5735 commented on Sep 26, 2026

    @j57n-3co-x-5735

    I was seeing repeated Codex connection failures in Pi, particularly during long conversations containing images. The recurring error was “Codex SSE response headers timed out after 20000ms.” This investigation and fix targeted my local v0.80.2 checkout, not necessarily current upstream; newer versions may already differ.

    Investigation of the saved session diagnostics identified this sequence in two affected sessions:

    • Accumulated images made the outgoing requests approximately 19–24 MB.
    • The WebSocket connection rejected those requests as too large.
    • Pi switched to its alternative HTTP streaming connection (SSE).
    • That fallback was cut off after 20 seconds, even though the configured timeout was five minutes. The cutoff included time spent uploading the request.

    Retries could then repeat the same failure because the request remained large and the session continued using the fallback connection.

    A local fix removed that independent 20-second cutoff and made connection attempts and retries share the configured time budget. It also stopped repeatedly sending the same rejected WebSocket request, allowed WebSocket recovery after temporary failures, and prevented automatic replay after a response had already started. Images and conversation history were preserved rather than silently removed.

    For anyone tracing this in the codebase, the useful starting points in that checkout are:

    • packages/ai/src/api/openai-codex-responses.ts: the old DEFAULT_SSE_HEADER_TIMEOUT_MS = 20_000 and createSSEHeaderTimeout() imposed a deadline independently of the supplied timeoutMs. The timer ran from immediately before fetch() until response headers arrived—not during the streamed response body. The same adapter owns WebSocket-to-SSE fallback; look for close code 1009, recordWebSocketFailure, and the old websocketSseFallbackSessions set.
    • packages/coding-agent/src/core/sdk.ts: the stream wrapper supplies timeoutMs from caller options, provider retry settings, or httpIdleTimeoutMs. This is why increasing the configured timeout alone did not fix the old adapter's separate timer.
    • packages/coding-agent/src/core/agent-session.ts, particularly _isRetryableError: stopping fallback inside the provider was not sufficient; agent-level retries also needed to reject replay after output. The fix checks whether any provider_transport_failure diagnostic has eventsEmitted: true, since an earlier WebSocket failure can say false while a later SSE failure says true.

    A useful regression case is to delay mocked SSE headers beyond 20 seconds while supplying a longer timeout: the old implementation still times out at 20 seconds. Also test a pre-output WebSocket 1009 followed by an SSE failure after output, ensuring the agent does not replay the turn. The existing packages/ai/test/openai-codex-stream.test.ts is a starting point for transport tests.

    After rebuilding the compiled dist files used by my launcher and using the patched version, I haven’t experienced any further issues.

    The premature timeout was reproduced locally, but the reason the HTTP response needed more than 20 seconds wasn’t established. Slow upload, network conditions, or backend latency remain possibilities. This appears to have resolved my case, but it doesn’t establish that every Codex connection failure has the same cause.

  19. NachoVazquez commented on Oct 6, 2026

    @NachoVazquez

    Adding size data from Pi 1.0.4 (macOS arm64, openai-codex / gpt-6.1-sol, ChatGPT OAuth, no proxy or gateway).

    Two failure shapes, both ending in a zero-usage error turn:

    1. WebSocket error before the stream starts, which surfaces as fetch failed. The diagnostics read configuredTransport: "auto", fallbackTransport: "sse", eventsEmitted: false, phase: "before_message_stream_start", requestBytes: 3514964. A second run failed the same way with requestBytes: 146917, so small bodies fail on this transport too.
    2. HTTP 503 upstream connect error or disconnect/reset before headers, with reset reason: remote connection failure and once connection timeout. CF-Ray location EZE.

    Under transport: "auto" the Codex path uses WebSocket, and that frame is uncompressed JSON. The SSE path compresses the same body with zstd. I probed both with one small request. With transport: "sse" it posts to chatgpt.com/backend-api/codex/responses with content-encoding: zstd and returns HTTP 200. With transport: "auto" the fetch function is never called at all, so nothing is compressed.

    On the plain HTTP provider (openai, api.openai.com/v1/responses, same model, OAuth) I can reproduce a size boundary with a change that means nothing to the model: leading whitespace inside the JSON body, which parsers ignore.

    • 1,048,575 bytes: HTTP 200
    • 1,048,577 bytes: HTTP 503, the same upstream connect error text

    A 1.61 MB conversation failed. Compressing it losslessly recovered it: 1,611,984 bytes became 993,986 bytes with zstd and 861,554 bytes with gzip, both accepted with images and history intact.

    This matches @j57n-3co-x-5735, who measured 19-24 MB requests with accumulated images and saw the WebSocket reject them as too large, and @jparradog, whose diagnostics ranged from 97,632 to 2,767,402 bytes with a median of 607,917.

    Is the uncompressed WebSocket frame intentional, to match the official Codex client? If so, would you consider compressing the WS frame, or preferring SSE above a size threshold? I set transport: "sse" locally, though this thread suggests SSE has its own regional problems.


    AI-assisted: an agent pulled these diagnostics from my sessions and ran the two transport probes. I reviewed the numbers and the conclusion before posting.

  20. dpetrou-continua commented on Oct 11, 2026

    @dpetrou-continua

    hey guys -- i've also been seeing this on Pi 1.1.0 with Codex/Sol on macOS. (tweeted about it a few days ago.) failures sometimes persist across turns, but restarting Pi with the same saved session restores operation without removing history. since it's 1.1.0, the SSE timeout fix is already present. i basically can't work anymore (maybe a good thing? :│. blocks unattended agents. happy to help debug.

  21. petrroll commented on Oct 19, 2026

    @petrroll
    Collaborator

    @dpetrou-continua could you try to instrument your pi to add logging proxy and check what exactly those failures are for you? Is it the tcp connection just dying, explicit http status code, or something else?

    Does it seem to be heavily correlated with request size (always fails bellow / always succeeds bellow)?

    Does forcing sse vs wss help? Does it change anything? Is there a session that always fails? Or is it (high probability in some cases) random?


    Happy to help with setting up the proxy if needed.

  22. petrroll commented on Oct 19, 2026

    @petrroll
    Collaborator

    @NachoVazquez coudl you instrument your pi with http / tcp proxy and ask it to check few things:

    WebSocket error before the stream starts, which surfaces as fetch failed. The diagnostics read configuredTransport: "auto", fallbackTransport: "sse", eventsEmitted: false, phase: "before_message_stream_start", requestBytes: 3514964. A second run failed the same way with requestBytes: 146917, so small bodies fail on this transport too.

    How does the wss connect fails? Does it fail before upgrade? is it just tcp connection dies, is there http status code? (I know js standard wss api doesn't let you see that, but proxy should).

    HTTP 503 upstream connect error or disconnect/reset before headers, with reset reason: remote connection failure and once connection timeout. CF-Ray location EZE.

    I'm a bit puzzled by this one. You mention http 503 (so likely status code was replied) but then the error message mentions error before headers but I'd expect that if you get http code back you'd also get response headers (or is this about sending headers)? Could you ask your pi to instrument itself to disambiguate what exactly fails here and what exactly the server returns?

    If needed I can help with instrumented fork, but just asking it to instrument itself (unless of course it always fails, then it'd not work 😅) should be enough and might work better.

    On the plain HTTP provider (openai, api.openai.com/v1/responses, same model, OAuth) I can reproduce a size boundary with a change that means nothing to the model: leading whitespace inside the JSON body, which parsers ignore.

    Do I get that you're effectively reporting that in addition to the two codex issues, there's also additional "standard openai / chatgpt http endpoint" where anything above 1mb uncompressed fails?

  23. NachoVazquez commented on Oct 20, 2026

    @NachoVazquez

    @petrroll Here is my agent's assessment and test results.

    I ran fresh checks with Pi 1.3.0 / Node 24.15.0 on macOS arm64, using direct TCP/TLS and Undici hooks. The prompt was synthetic, with no tools or saved conversation.

    1. I could not reproduce the original pre-stream WebSocket failure. Both 296-byte and 1,049,600-byte application messages completed. TCP/TLS succeeded; Undici handled unsupported HTTP/2 extended CONNECT by opening the WebSocket over HTTP/1.1. Both negotiated permessage-deflate. My earlier claim about uncompressed WebSocket traffic was too broad: application JSON size does not establish compressed wire size.

    2. The earlier 503 was a received HTTP response, with headers and a CF-Ray value. “Upstream connect error or disconnect/reset before headers” was text in its error body. That appears to describe a gateway's upstream connection; the failed hop remains unknown. Today's HTTP checks all returned 200, so I have no fresh failing response to trace.

    3. Yes, the earlier measurements included api.openai.com/v1/responses with ChatGPT OAuth. A paired October 8 test also observed size-dependent 503s on the Codex HTTP endpoint. Today, Codex accepted uncompressed bodies of 1,047,552 and 1,049,600 bytes over both HTTP/1.1 and HTTP/2. The larger body also passed with zstd. Padding changed only whitespace outside the JSON value. I did not retest the separate API endpoint because its saved OAuth login expired. These results do not establish a current universal 1 MiB limit.

    I excluded one initial control because my instrumentation prevented HTTP/1 fallback. I corrected that before collecting the results above.

    I also have a small local recovery patch working through Pi 1.3.0. recordWebSocketFailure() puts the session into persistent SSE fallback even after streaming starts. My change is:

    recordWebSocketFailure(cacheSessionId, error, !websocketStarted);

    The helper adds the session to the fallback set only when that flag is true, while retaining failure counts and accurate diagnostics. The interrupted response still fails; the next request can reconnect over WebSocket. Pre-stream fallback stays unchanged. The patch adds no replay of the interrupted response.

    The 1.3.0 port passed 37 offline tests and actual bundled-CLI fixtures for disconnect, reconnect, cancellation, reuse, and cleanup, with zero HTTP attempts in those CLI fixtures. An earlier bounded live SDK test recovered after forced in-stream TCP termination. A longer Pi 1.1 pilot completed 242 model steps, recovered from two natural interruptions, and crossed compaction; its real traffic was not instrumented. The original network failure cause remains unresolved.

    Would this narrower recovery change be useful upstream?

  24. petrroll commented on Oct 23, 2026

    @petrroll
    Collaborator

    @NachoVazquez If I read that correctly it seems that it's fine now, no?

    1. success
    2. success
    3. success

    We can think about adding recovery, but the tl;dr is that the issue was likely temporal and we can't no longer repro it to know what to fix :).

    As for the narrover recovery, feel free to link patch I'll think :)

  25. NachoVazquez commented on Oct 24, 2026

    @NachoVazquez

    @petrroll Patch and regression test, based on main at 136b87e8a7f5.

    The test closes the socket after text starts streaming. Stock Pi sends the next turn to HTTP. With the patch, that turn reconnects over WebSocket with the full history. The interrupted response still fails, without replay. Pre-stream fallback stays unchanged.

    npm run check and 41 focused tests pass. The new test fails on stock and when the recovery condition is inverted.

    The original network failures remain unexplained. This patch covers the persistent fallback after a disconnect.

    This comment and patch were prepared by my coding agent.

  26. marandaneto commented on Oct 24, 2026

    @marandaneto

    openai-codex with gpt-6.1-sol is giving some many

    Error: no healthy upstream
    Error: terminated
    Error: upstream connect error or disconnect/reset before headers. reset reason: connection termination

    and other variants

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

inprogressIssue is being worked on

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions