Skip to content

Add buffered Responses for native agents - #5589

Merged
markstuart-oai merged 14 commits into
feat/codex-harnessfrom
codex/buffered-native-responses
Nov 8, 2026
Merged

markstuart-oai merged 14 commits into
feat/codex-harnessfrom
codex/buffered-native-responses

Conversation

@markstuart-oai

@markstuart-oai markstuart-oai commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

For large responses, handling thousands of SSE fragments adds substantial native SDK overhead. NativeProvider(response_mode="buffered") requests JSON over HTTP and passes the complete response through Core's existing tool, policy, history and observation handling. Streaming remains the default.

Buffered mode requires API-key authentication and a standard Responses model. The current bundled default uses Responses Lite and is unsupported, as are WebSockets, CodexLogin and native child agents declared with NativeAgents. Model output and tool-call start wait for the complete body; tool-execution events remain available while tools run.

Each body is limited to 64 MiB and one million structural characters ({}[],:) outside quoted strings. Both limits apply to successful and HTTP-error bodies before JSON tree allocation or retry handling. Limit failures do not retry; ordinary HTTP 429 errors can still retry. Cancellation and teardown retain their existing owners.

This adds one SDK-local Core patch with 409 net production lines. It also narrows certificate-wrapper caching in the existing HTTP-pool patch: ordinary client construction can reuse parsed certificate wrappers, while pooled requests keep fresh certificate snapshots and never consult that cache. No Codex revision change is required. User documentation is separate in #5590.

Measurements

These tests use fixed local responses on Linux and independent fresh conversations. They measure SDK/client costs, not model inference or WAN latency. CPU includes client construction, one fresh prefill, 20 fresh calls and joined teardown, divided by all 21 conversations. Median latency excludes the prefill. Peer CPU is excluded.

The original 125-worker comparison completed 2,625 fresh conversations with zero errors. For the same 64 KiB answer:

SSE fragments in the streaming baseline v0 CPU v1 streaming CPU Buffered CPU v0 median v1 streaming median Buffered median
256 15.79 ms 51.20 ms 22.78 ms 4.06 ms 22.29 ms 13.46 ms
4,096 15.67 ms 394.99 ms 23.09 ms 4.98 ms 142.34 ms 14.97 ms

That is 55.5–94.2% less CPU and 39.6–89.5% lower median latency than streaming v1. The paired CPU-ratio 95% intervals are 0.384–0.503 and 0.0572–0.0588. Short text still took 12.37 ms at the buffered median versus 3.50 ms in v0: this does not establish full v0 parity.

Those measurements belong to 678ec3a2, against native baseline e283d9a5 and v0 0.24.0. A separate 40-worker comparison after the HTTP-error correction at ab01182b completed 840 fresh conversations with zero errors; all four CPU and median-latency point ratios stayed within 1.05. This point guard does not establish equivalence, and the original results are not relabelled as measurements of later commits.

A separate comparison of cf6dada05 against 297663b1 completed 120 workers and 2,520 fresh conversations with zero errors. The selective certificate cache reduced CPU by 45.2% over HTTP and 41.2% over HTTPS, and median latency by 57.8% and 54.6%, using explicit bundles of 121 and 122 certificates. These are geometric-mean reductions across text and one-tool cases in normal SSE mode. Both groups passed the preset retention thresholds. With the default trust store, CPU and median latency changed by +0.3% and +0.4%.

All six pooled cases stayed within a 1.021 candidate/reference ratio for both CPU and median latency, passing the preset 1.05 limit. This is a point-estimate regression check, not an equivalence result. The cache remains limited to ordinary client construction.

The certificate-cache comparison isolates ordinary client construction by disabling sharing in both arms. Supported model paths also use ordinary construction when a Session owns its provider or a route cannot use the shared pool. These timings are not literal default-Agent results or macOS/Windows measurements. The paired pooled comparison checks the normal shared-provider path separately.

The benchmark and reproduction guidance check actual wire mode, fresh histories, complete cleanup and uniform compiled Python caches. Conditional cache comparisons do not update the historical buffered-versus-v0 numbers above.

Test plan

The following checks passed on 43ffe300e9, which preserves the target branch's Linux proc-mount fix and the unchanged buffered-response patch. Hosted PR checks passed after the merge.

  • The normal release wheel built successfully; all 141 native build inputs stayed unchanged. Its five native payloads match the installed SDK and test checkout.
  • The required repository verification script passed formatting, lint, mypy, Pyright, 5,716 parallel tests and 54 serial tests. The parallel run reported 253 skips. This was a Linux run with OPENAI_AGENTS_TEST_IN_CODEX_SANDBOX=1; source files and all five native payloads remained unchanged.
  • All 87 focused installed-package tests passed without skips, including buffered responses, both provider-selection paths, MCP resources, voice and transport authority.
  • Seven proc-mount classifier tests passed using the exact current source. The full Codex Rust crate suite remains unverified because of the existing RMCP test-workspace dependency mismatch. The earlier 33 focused Rust tests passed at cf6dada05 under the recorded, restored test-only dependency overlay; the current release wheel uses the normal dependency graph.
  • 24 preliminary workers checked 48 fresh conversations before the separate 120-worker, 2,520-conversation scored comparison. Both cohorts had zero errors; connection reuse, fresh histories and joined cleanup were verified. Measurements retain the original commit attribution above.
  • Two independent final reviews and the hosted review are clean on 43ffe300e9. Human approval is current, with no unresolved review threads. All 29 PR checks are terminal: 24 passed, five were conditionally skipped, and none failed.

Issue number

None linked.

Checks

  • I've added new tests, if relevant
  • I've run the applicable checks in AGENTS.md, including .agents/skills/code-change-verification/scripts/run.sh when required
  • I've confirmed all required checks pass
  • I've completed the review required by AGENTS.md

@markstuart-oai
markstuart-oai requested review from a team, rm-openai and seratch as code owners November 7, 2026 19:25
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-08T01:14:10.542945Z 43ffe30 New commits
🔒 Security Review ✅ Completed 2026-11-08T01:14:55.633169Z 43ffe30 New commits

Security findings

Advisory findings (1)

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1aca01e66c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the full diff with two independent static reviewers. No actionable findings. Buffered ingress preserves ordered Core tool/history/policy handling, validates all output before effects, retains terminal metadata while withholding rejected output, and keeps body ownership/cancellation/idle timeout/retry behavior on the existing mapper. Reviewed provider selection and unsupported-mode guards, restore selection, observation transport, source manifests, and benchmark actual-wire receipts/full-owner accounting. Benchmark claims are limited to the documented local HTTP workloads; they do not establish live-streaming or short-response parity. The existing field-order comment is nonblocking: the published v0.24.0-alpha.6 provider has no auth field, and this constructor is keyword-only. Hosted CI: 4 passed, 2 skipped, 1 neutral, 4 running; qualification retained. No candidate code/tests executed locally.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1369f2ee01

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the three-file update with an independent static reviewer. Moving response_mode after auth follows the append convention; the matching API snapshot and explicit asdict key-order assertion are consistent. No runtime behavior change or new actionable findings. The existing field-order feedback is addressed. Hosted CI: 3 passed, 2 skipped, 1 neutral, 5 running; follow-up retained. No candidate code or tests executed locally.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reassessed the existing body-bound feedback and am superseding my earlier approval. The finding at #5589 (comment) is actionable: response_stream appends every chunk to an unbounded Vec, and a peer that keeps sending within the per-chunk idle timeout can exhaust the SDK process before JSON validation. Enforce a cumulative response-body limit before extending the buffer, and make the oversize failure non-retryable; cover both declared and chunked/unknown-length responses. This is the existing finding, not a duplicate inline comment. Independent static review confirms the issue. Hosted CI remains pending. No candidate code or tests executed locally.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛡️ Codex Security Review · Automatically triggered

Here are some automated security review suggestions for this pull request.

Reviewed commit: a006551934

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the body-limit fix with an independent static reviewer. The JSON request now sets the existing 64 MiB transport limit; declared and cumulative sizes are checked before delivery to the buffer. ResponseTooLarge stays typed through the JSON mapper and becomes non-retryable InvalidRequest in Core. Added declared/chunked regressions assert one request and cleanup; exact-once custom-tool coverage is consistent with the existing consumer. Patch digest matches the manifest. This resolves my request-changes decision. Hosted CI remains pending. No candidate code or tests executed locally.

@markstuart-oai
markstuart-oai force-pushed the codex/buffered-native-responses branch from a006551 to 4059b9c Compare November 7, 2026 20:17

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the rebased head with an independent static reviewer. Of 14 changed blobs, 13 exactly match the current base; the remaining API-contract fixture integrates the base metadata. The buffered implementation is byte-identical to approved a006551, including the body bound and non-retryable oversize handling. The new JSON-allocation feedback identifies real parsing overhead, but the documented limit is on body bytes, not total heap; no unbounded-body bypass is demonstrated. Complexity limits are nonblocking hardening. Hosted CI: 4 passed, 1 neutral, 4 running. Static review only; no candidate code or tests executed locally. Hosted CI follow-up retained.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed 678ec3a with an independent static reviewer. No new blockers. The pre-deserialization scan bounds dense JSON structures while correctly excluding escaped string payloads; serde retains syntax/depth validation and the existing 64 MiB transport bound remains. Structural-limit errors are non-retryable. Tests cover dense objects/scalars, escaped large text and one-request native failure. Benchmark workers and peers consistently disable bytecode writes, record and validate runtime cache settings, and document the separate requirement to inventory valid caches. Patch SHA-256 matches the series manifest. Hosted CI: 3 passed, 1 neutral, 5 running; platform qualification follow-up retained. No candidate code or tests executed locally.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up to my approval: new hosted CI confirms a release-contract integration failure.

[P2] Refresh the reviewed API snapshot against the integrated base. packages/codex/tests/fixtures/released_api_contract.json:2028 records only three SandboxArchiveLimits fields. The tested merge 1716714 contains five additional fields from base 2a09c45: max_tar_metadata_bytes, max_tar_stream_bytes, max_zip_directory_bytes, max_tar_lzma_memory_bytes, and max_zip_lzma_dictionary_bytes, with their referenced defaults. Five completed Linux/Windows jobs fail the same four released API contract assertions. Regenerate and review the contract against the integrated source, preserving response_mode and the intended public API, then rerun the matrix without weakening contract checks.

Hosted evidence: https://github.com/openai/openai-agents-python/actions/runs/42412090275/job/128529419971 and https://github.com/openai/openai-agents-python/actions/runs/42412090275/job/128528865983 . This is stale snapshot integration, not evidence of a buffered-response behavior regression. Independent static inspection confirmed the missing fields. This decision supersedes my approval based on new CI evidence; remaining hosted checks are still pending. No candidate code or tests executed locally.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed a5a93fb with an independent static reviewer. The contract finding is addressed: the snapshot now records all eight SandboxArchiveLimits fields and the four added default constants, matching the integrated source. Deep comparison shows all other public API entries, including response_mode, are unchanged. The five archive implementation/test files match the merged base; buffered-response implementation, byte limits and structural limits are unchanged from the reviewed commit. No remaining actionable findings.

Hosted CI has restarted and remains pending; successful current-head qualification is still required before merge and is tracked. No candidate code or tests executed locally.

@markstuart-oai

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: a5a93fb0d8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: a5a93fb0d8

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed ab01182 with an independent static reviewer. The three-file delta applies the structural complexity limit to non-2xx buffered HTTP bodies before retry/error parsing. ResponseTooComplex propagates as a typed non-retryable transport error and maps to InvalidRequest in Core; ordinary HTTP errors keep their existing retry behavior. Tests cover dense payloads at 200/400/403/429/503 and a normal 429 followed by a successful retry. The existing byte bound and successful-response validation remain intact; patch checksum matches the manifest. No actionable findings.

Hosted CI has restarted and remains pending; follow-up retained. No candidate code or tests executed locally.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed 93c0065 with an independent static reviewer. This update incorporates base-branch changes: 47 of the 49 changed files match the current base exactly; session.py retains the buffered-mode additions while adopting the base checkpoint-policy check, and the API snapshot incorporates base declarations while retaining response_mode/http_json. The previously reviewed buffered transport, size/complexity bounds, provider and regression tests are unchanged. No new actionable findings. Hosted CI has restarted on this head and remains pending; the earlier green result applied to ab01182. No candidate code or tests executed locally.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two static reviews of the current integration found no new actionable correctness/security issues. Ordinary client construction retains the bounded public-certificate wrapper cache; pooled clients use freshly read and parsed CA bytes and a digest, retire changed roots before building, and validate certificate wrappers on construction. Invalid-DER retirement is covered. The buffered patch is unchanged apart from its series number; MCP/voice integration and the provider override regression preserve the feature boundaries. Approval is a code decision, not a green-CI claim. Python 3.10 hosted CI failed in test_rpc_timeout_exception_chain_does_not_expose_bearer[idle]: its 50 ms timeout elapsed with no request received by the fixture. Logs suggest pre-request timing, but do not prove flakiness. This check still needs resolution: https://github.com/openai/openai-agents-python/actions/runs/42428914157/job/128580220536 . No candidate code or tests run locally.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed this exact commit after the rebase. The buffered Responses patch is byte-identical after renumbering; the additional sandbox diagnostic patch matches the merged base, and benchmark documentation accurately clarifies its scope. No new actionable findings. Hosted CI is still running on this head; the earlier timeout failure is not established as resolved. Static review only; no local code or tests executed.

@markstuart-oai
markstuart-oai merged commit a77c04a into feat/codex-harness Nov 8, 2026
22 checks passed
@markstuart-oai
markstuart-oai deleted the codex/buffered-native-responses branch November 8, 2026 01:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants