Skip to content

Add observed budgets for native agent runs - #5555

Merged
markstuart-oai merged 3 commits into
feat/codex-harnessfrom
mark/req76-observed-budgets
Nov 6, 2026
Merged

markstuart-oai merged 3 commits into
feat/codex-harnessfrom
mark/req76-observed-budgets

Conversation

@markstuart-oai

@markstuart-oai markstuart-oai commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This pull request adds optional budgets for direct native agent runs and Session submissions. Applications can limit admitted root requests, reported tokens, and estimated cost in integer micro-US dollars.

The SDK records reported usage before deciding whether to admit another request. A denial happens before the step hook and provider request. If a measurement required by the budget is missing, the default policy stops the next request. Applications can opt to continue with the measurements they have. An admitted response can use more than the limit and still return a valid final result.

Live manual replies, retained steering, and output repair share the current budget. New root operations start with a fresh budget. Active budgets prevent portable checkpoint capture and preserve the live reply.

Test plan

  • The repository verification wrapper passed format, lint, type checking, and tests on Python 3.14. The parallel suite passed 4,937 tests with 176 skips. The serial suite passed 41 tests.
  • Fresh wheel installs on Python 3.10, 3.13, and 3.14 passed repository qualification, 41 budget cases each, public annotation checks, and the live checkpoint probe.
  • The generated API contract passed its standard check. Independent runtime and public API reviewers verified the feature implementation.
  • The Python 3.10 MCP traceback tests passed in hosted Linux CI before the rebase. This branch now uses the shared correction from the target branch.
  • The Windows child-stop test now waits for native turn handles before taking its snapshot. The corrected test passed five times on the qualified Python 3.10 wheel. Hosted CI is running on the rebased head.
  • An earlier editable test run crashed while another install wrote to the same native addon. The cause remains unknown. The final verification completed with the same addon hash and file metadata before and after the run.

Checks

  • Added native and public API regression tests
  • Ran the applicable repository verification wrapper
  • Completed independent runtime and public API review
  • All required checks pass

@markstuart-oai
markstuart-oai requested review from a team, rm-openai and seratch as code owners November 6, 2026 03:48
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-06T05:15:06.262517Z e2b5d42 New commits
🔒 Security Review ✅ Completed 2026-11-06T05:15:34.978939Z e2b5d42 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@markstuart-oai
markstuart-oai force-pushed the mark/req76-observed-budgets branch from 71c4354 to 1aa8809 Compare November 6, 2026 04:02

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed request/token/estimated-cost accounting, missing usage, root event ordering, live replies, steering, output repair, and checkpoint restrictions. No actionable introduced issue found.

Static review only; no local code or tests executed. Hosted Python 3.10 failed three deferred-MCP cancellation traceback tests; the new budget tests passed in that log. Other observed checks passed. CI failure remains open; baseline causation is unverified.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the exact tree delta from my prior approval: only deferred-MCP test traceback capture changed. Capturing frames within the cancelled task avoids Python 3.10 losing them across the task boundary; sensitive-data and credential-release assertions remain. The runtime budget implementation is unchanged. No actionable introduced issue found. Hosted checks are newly running; the previous Python 3.10 failure is not yet verified resolved.

Static review only; no local code or tests executed.

@markstuart-oai
markstuart-oai force-pushed the mark/req76-observed-budgets branch from 911adc0 to e2b5d42 Compare November 6, 2026 05:13

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approved. The child-turn test now waits for published handles before asserting them, addressing the observed Windows observation race while preserving the policy-stop, sibling and successor assertions and the enclosing timeout. Budget runtime code is unchanged. Also checked the merged CI/base reconciliation and retained budget annotation checks. No new actionable finding. The Windows fix still needs hosted confirmation.

Hosted CI remains pending with no observed current-head failures. Static coordinator and independent review only; no candidate code or local tests executed.

@markstuart-oai
markstuart-oai merged commit ca827f8 into feat/codex-harness Nov 6, 2026
13 checks passed
@markstuart-oai
markstuart-oai deleted the mark/req76-observed-budgets branch November 6, 2026 05:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants