Skip to content

Expose typed reasons for native output failures - #5520

Merged
markstuart-oai merged 53 commits into
feat/codex-harnessfrom
codex/failure-reasons-current-v1-20261105
Nov 5, 2026
Merged

markstuart-oai merged 53 commits into
feat/codex-harnessfrom
codex/failure-reasons-current-v1-20261105

Conversation

@markstuart-oai

@markstuart-oai markstuart-oai commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Native output failures report a fixed message, leaving callers unable to distinguish why output was rejected. Add read-only NativeError.response_integrity_reason with four typed values: an undecodable output item, a provider-owned tool receipt, an unfinished assistant item, or missing supported terminal output.

The reason follows the selected error through existing native settlement. It preserves error categories, provider metadata and retry behavior. Missing or ambiguous ownership leaves the reason unknown. The reason does not imply that earlier operations had no effects.

Test plan

The complete formatting, lint, type and test sequence passed on the exact integrated tree for public head df3969cb and target 6d319860: 4,323 parallel tests passed with 177 platform/sandbox skips, followed by 41 serial passes. Documentation checks and the site build also passed. Source, native package and review fingerprints matched before and after. Two fresh independent reviewers accepted the complete feature and upstream configuration integration with no findings.

The authenticated Linux package matches all 111 native inputs and 40 packaged Python/stub files. Installed checks passed 109 diagnostics cases (four host-sandbox skips), 43 trace-context cases, 44 native tracing/transport cases and all three archive checks.

Two test fixtures now wait for their required state: cancel a pending observation read, and poll a running release command before requiring terminal output. Existing privacy, completion, fork-refusal and side-effect assertions remain intact. Hosted Python 3.12 passed the repaired native Linux fork case: 4,437 passed and 13 skipped. The local sandbox cannot execute that case.

Current human and automated reviews are clean. All reported hosted checks passed or were intentionally skipped on public head df3969cb; no checks remain pending.

Issue number

Supports the diagnostic portion of REQ751. This does not establish incident resolution.

Checks

  • I've added new tests, if relevant
  • I've run the applicable checks in AGENTS.md, including .agents/skills/code-change-verification/scripts/run.sh when required
  • I've confirmed all required checks pass
  • I've completed the review required by AGENTS.md

@markstuart-oai
markstuart-oai marked this pull request as ready for review November 5, 2026 05:23
@markstuart-oai
markstuart-oai requested review from a team, rm-openai and seratch as code owners November 5, 2026 05:23
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-05T17:18:41.488551Z df3969c New commits
🔒 Security Review ✅ Completed 2026-11-05T17:19:18.946488Z df3969c New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved after coordinator and independent static review. Content-free integrity reasons follow the selected originating failure across native error propagation and Python conversion. Same-ID ambiguity stays unknown; ordinary provider metadata, retry behavior and tool-effect settlement remain intact. Reviewed HTTP/WebSocket, child ownership, delivery-order and already-running-effect coverage. Linux and Windows hosted snapshots passed; macOS and Rust CodeQL remain pending. No candidate code or local tests ran.

@markstuart-oai

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: 874d792d16

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 874d792d16

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static re-review of fe8029b56cc454f2fc7dafca24089549deab999f: approved. The child-question regression now waits for a successor snapshot within the existing deadline. The final rebase adopts the base worker test that captures the actual shell job directory, addressing the earlier TMPDIR assumption. Exact old/new tree comparison: 45 changed files, 44 identical to current base; the remaining API fixture preserves the integrity-reason additions while integrating the base fixture update. Hosted native CI remains pending; no candidate code or local tests run.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4e7cb7e722

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/test_native_provider_errors.py Resolved

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved after static re-review. Saved-login integration preserves the reviewed integrity-reason behavior in overlapping source and API fixtures. Hosted CI/native builds remain pending. No candidate code or tests executed.

@markstuart-oai

Copy link
Copy Markdown
Contributor Author

@codex review

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved after static review and independent review. The test-only update addresses existing feedback about fabricated enum values while preserving absent/null coverage. Runtime and native inputs are unchanged. Hosted CI remains pending; no candidate code or tests executed.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved the integration at d9da489. Child admission, tool callbacks and step selection retain the creation-owned tracing context, including explicit absence, without replacing an existing turn carrier. The original feature changes remain intact. The fixture changes correctly wait for child observation before release and accept either winner of competing manual replies while requiring exactly one success.

Static review with an independent reviewer; no candidate code or tests were run. Fresh hosted CI remains pending. The unchanged shared tracing patch does not clear the separately tracked base security finding.

@markstuart-oai
markstuart-oai changed the base branch from feat/codex-harness to codex/native-trace-context November 5, 2026 15:49
Base automatically changed from codex/native-trace-context to feat/codex-harness November 5, 2026 15:55

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at 70eaa3c. The tracing security fix is incorporated unchanged while response-integrity reasons retain originating error metadata and completed effects. All four stack heads contain the exact reviewed W3C security patch from 5451026; the inherited tracing blocker is addressed here. Final ancestry commits do not change the reviewed trees.

Hosted checks: 19 SUCCESS, 4 IN_PROGRESS, 3 SKIPPED, 1 NEUTRAL. Coordinator and independent static review found no actionable integration regressions. No candidate code or local tests executed; no CI reruns requested.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at 21f29bb. The sole incremental change drains buffered observations before cancelling a pending reader and asserts cancellation succeeds. The blocked policy, independent-reader assertions, privacy checks and enclosing timeout remain intact. No runtime changes.

Coordinator and independent static review found no actionable issues. Hosted CI: {'SUCCESS': 4, 'IN_PROGRESS': 4, 'NEUTRAL': 1}. CI follow-up remains open. No candidate code or local tests executed; no CI reruns requested.

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at df3969c. Re-reviewed the single-file incremental fix with an independent static review. It polls the existing release-command session instead of restarting the command, preserves accumulated output, and requires terminal exit 0. The ten-poll limit and existing 40-second timeout bound execution. Both fork-refusal/no-side-effect checks and saved-child linkage remain intact; runtime code is unchanged. No actionable findings.

This addresses the observed premature-completion test assumption in source. Fresh hosted runtime verification is still pending; the previous-head failure is not evidence that this commit has passed. No candidate code, local tests, or CI reruns were executed.

@markstuart-oai
markstuart-oai merged commit b03f556 into feat/codex-harness Nov 5, 2026
29 checks passed
@markstuart-oai
markstuart-oai deleted the codex/failure-reasons-current-v1-20261105 branch November 5, 2026 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants