Skip to content

docs(parity): (GPT 5.4 Parity vs. Opus Agentic) catch up 10-PR program state with architecture diagrams#64837

Closed
100yenadmin wants to merge 10 commits into
openclaw:mainfrom
electricsheephq:docs/gpt54-parity-catchup
Closed

docs(parity): (GPT 5.4 Parity vs. Opus Agentic) catch up 10-PR program state with architecture diagrams#64837
100yenadmin wants to merge 10 commits into
openclaw:mainfrom
electricsheephq:docs/gpt54-parity-catchup

Conversation

@100yenadmin

@100yenadmin 100yenadmin commented Apr 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Documentation-only catch-up. Brings docs/help/gpt54-codex-agentic-parity.md and docs/help/gpt54-codex-agentic-parity-maintainers.md up to the full ten-PR state of the parity program, adds three new architecture diagrams, an end-to-end runbook, and an updated goal-to-evidence matrix.

Zero runtime, test, scenario registry, CLI, or architecture changes. Part of #64227.

Why this exists

The earlier parity docs were written for the first-wave 4-PR plan (A, B, C, D). Once the refinement wave started landing on branches (E, H, J, K, L), the docs were stuck describing a much smaller program than what was actually in flight. Reviewers couldn't tell from a single page what was merged, what was open, or how the pieces fit together — so I'm treating "the full story is legible from the docs" as part of the criterion 5 deliverable and splitting it out here so it doesn't have to wait on whichever wave-2 PR is the last to merge.

What changed

docs/help/gpt54-codex-agentic-parity.md

  • Rewrote the intro to explain the two waves and the 10-PR cap backstory honestly.
  • Added a Where things stand table listing the full program with per-PR merge status.
  • Restructured What changed around "the runtime contract" (PRs A, B, C, H), "the parity harness" (PRs D, E, K, L), and "tool-call enforcement" (PR J) instead of a flat list of ten subsections.
  • Kept the before/after table but extended it to cover the wave-2 symptoms (prose-only scenarios, baseline credentials, provenance).
  • Added three new mermaid diagrams: the dual-provider mock architecture, the parity run orchestration, and the tool-call assertion seam.
  • Added a Running the harness end-to-end section with the three-step flow using the real CLI flags (--model / --alt-model on qa suite, --candidate-summary / --baseline-summary on qa parity-report), plus a clear note that the baseline lane only runs fully offline after PR K lands.
  • Updated Release gate and Goal-to-evidence matrix so the five criteria map across all ten PRs.
  • Rewrote the strict-agentic section around the three real configurations (auto-activated default, explicit opt-out, explicit opt-in for other providers) instead of a bulleted "who should enable" list.
  • Kept every section that describes unmerged behavior tagged inline with "after PR X lands" so nobody tries to run commands that depend on code that isn't on main yet.

docs/help/gpt54-codex-agentic-parity-maintainers.md

  • Rewrote the intro to explain the two-wave shape.
  • Added per-PR Owns / Does not own subsections for E, F, H, J, K, L, and M.
  • Updated Mapping back to the original six contracts so "Same-turn execution" is PR A + PR H, "Replay/continuation/liveness correctness" is PR C + PR H, and "Benchmark/release gate" spans PR D + PR E + PR J + PR K + PR L + PR M.
  • Rewrote Review order around the real dependency graph (H independent; E depends on D; K independent; J depends on E; L independent; M last) instead of a flat enumerated list.
  • Added What to look for subsections for each new PR.
  • Updated Release gate to require PR H merged, PR D + PR E clean on both providers, PR J tool-call assertions passing, PR K's dual-provider mock exercised in CI, and PR L's run metadata present on both summaries.
  • Updated the goal-to-evidence map and reviewer shorthand table to match the ten-PR program.

Test plan

  • wc -l docs/help/gpt54-codex-agentic-parity.md — consolidated from the earlier 401-line draft into a tighter rewrite
  • wc -l docs/help/gpt54-codex-agentic-parity-maintainers.md — similar consolidation
  • rg 'flowchart' docs/help/gpt54-codex-agentic-parity.md — 5 flowcharts (2 original architecture/release diagrams + 3 new)
  • Every mermaid node label that contains a leading / (for example /v1/responses, /debug/requests) is wrapped in quotes so it doesn't collide with mermaid's parallelogram shape
  • Runbook commands use the real CLI flag names (--model / --alt-model), not the earlier draft's --primary-model
  • Maintainer spot-check that each new PR subsection matches its owning PR's scope (depends on review)

Scope the way a reviewer should see it

  • What PR M owns: prose, tables, mermaid, runbook wording, matrix wording, program-status disclaimer.
  • What PR M does not own: anything runtime/test/scenario/CLI/architecture — if the doc describes a behavior, the owning PR is the one that implements it. The CLI flag reference matches what's on main today (--model), not a speculative future flag.

@greptile-apps

greptile-apps Bot commented Apr 11, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This docs-only PR expands gpt54-codex-agentic-parity.md and gpt54-codex-agentic-parity-maintainers.md from 4-PR coverage (A–D) to the full 10-PR program, adding three new Mermaid diagrams, an end-to-end runbook, and an updated goal-to-evidence matrix. The runtime, test, and config surfaces are untouched.

Confidence Score: 5/5

Safe to merge; all remaining findings are P2 documentation-completeness or clarity suggestions with no runtime impact.

No code changes, no logic errors, and no broken invariants. The only items are a stale table column header that undersells the full 10-PR program, a diagram cycle whose intent is inferrable, and a missing "Does not own" block in one PR section.

docs/help/gpt54-codex-agentic-parity.md (stale "After PR A-D" summary table; diagram cycle in Dual-provider mock), docs/help/gpt54-codex-agentic-parity-maintainers.md (PR M section missing "Does not own" block)

Comments Outside Diff (1)

  1. docs/help/gpt54-codex-agentic-parity.md, line 128-135 (link)

    P2 "Before vs after" table not updated to cover the full 10-PR program

    The column header still reads After PR A-D and the five rows only cover PRs A–D. A reader landing on this table after reading the expanded intro gets an incomplete picture — the second-wave improvements from PRs E, H, J, K, and L (auto-activation, doubled scenario pack, tool-call assertions, offline dual-provider mock, run metadata) are silently absent. At minimum the column header should be updated; ideally five new rows should be added.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: docs/help/gpt54-codex-agentic-parity.md
    Line: 128-135
    
    Comment:
    **"Before vs after" table not updated to cover the full 10-PR program**
    
    The column header still reads `After PR A-D` and the five rows only cover PRs A–D. A reader landing on this table after reading the expanded intro gets an incomplete picture — the second-wave improvements from PRs E, H, J, K, and L (auto-activation, doubled scenario pack, tool-call assertions, offline dual-provider mock, run metadata) are silently absent. At minimum the column header should be updated; ideally five new rows should be added.
    
    
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: docs/help/gpt54-codex-agentic-parity.md
Line: 128-135

Comment:
**"Before vs after" table not updated to cover the full 10-PR program**

The column header still reads `After PR A-D` and the five rows only cover PRs A–D. A reader landing on this table after reading the expanded intro gets an incomplete picture — the second-wave improvements from PRs E, H, J, K, and L (auto-activation, doubled scenario pack, tool-call assertions, offline dual-provider mock, run metadata) are silently absent. At minimum the column header should be updated; ideally five new rows should be added.

```suggestion
| Before this program                                                                            | After PRs A–L                                                                             |
| ---------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
| GPT-5.4 could stop after a reasonable plan without taking the next tool step                   | PR A turns "plan only" into "act now or surface a blocked state"                          |
| Strict tool schemas could reject parameter-free or OpenAI/Codex-shaped tools in confusing ways | PR C makes provider-owned tool registration and invocation more predictable               |
| `/elevated full` guidance could be vague or wrong in blocked runtimes                          | PR B gives GPT-5.4 and the user truthful runtime and permission hints                    |
| Replay or compaction failures could feel like the task silently disappeared                    | PR C surfaces paused, blocked, abandoned, and replay-invalid outcomes explicitly          |
| "GPT-5.4 feels worse than Opus" was mostly anecdotal                                           | PR D turns that into the same scenario pack, the same metrics, and a hard pass/fail gate  |
| Strict-agentic had to be opted in per-agent                                                    | PR H auto-activates the contract for all unconfigured GPT-5 / OpenAI-Codex runs          |
| Parity pack covered only five scenarios                                                        | PR E doubles the pack to ten, and any required-scenario fail on either side fails the gate|
| Scenario assertions accepted prose without a real tool call                                    | PR J adds `/debug/requests` tool-call gating on tool-mediated scenarios                  |
| Baseline parity required live Anthropic credentials                                            | PR K routes both providers through the same mock dispatcher — gate runs fully offline     |
| Parity consumers had to trust file paths to identify which summary came from which provider    | PR L records `run.primaryProvider` / `primaryModel` / `providerMode` in each summary     |
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: docs/help/gpt54-codex-agentic-parity.md
Line: 174-186

Comment:
**Diagram cycles leave the response destination implicit**

`H --> B` and `H --> C` route "Scenario-specific response" back into the two POST-endpoint nodes rather than back to the QA suite runner (`A`). Both B and C are labeled as request paths, so a reader following the arrows left-to-right loses track of where the response ultimately lands. Adding the return arrows directly to `A` (or labeling the back-edges) makes the round-trip explicit:

```
    H -- "OpenAI response" --> A
    H -- "Anthropic response" --> A
```

Minor diagram clarity — does not affect correctness of the other docs.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: docs/help/gpt54-codex-agentic-parity-maintainers.md
Line: 135-143

Comment:
**PR M section is missing a "Does not own" block**

Every other PR section (A through L) has matching "Owns" / "Does not own" bullets. PR M only has "Owns". For consistency and so reviewers can quickly verify scope boundaries, a brief "Does not own" block would complete the pattern, e.g.:

```markdown
Does not own:

- runtime behavior changes
- test coverage
- CI configuration
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "docs(parity): drop stale first-wave word..." | Re-trigger Greptile

Comment thread docs/help/gpt54-codex-agentic-parity.md
Comment thread docs/help/gpt54-codex-agentic-parity-maintainers.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the GPT-5.4/Codex agentic parity documentation to reflect the expanded parity program state (beyond the original A–D slices), adding additional PR coverage, new architecture diagrams, an end-to-end runbook, and an updated goal-to-evidence mapping.

Changes:

  • Expands parity program coverage from the initial PRs (A–D) to include follow-up slices (E/F/H/J/K/L) and updates scenario descriptions/matrix accordingly.
  • Adds new mermaid diagrams describing mock routing, parity orchestration, and tool-call assertion seams.
  • Adds an end-to-end parity gate runbook and updates release-gate / goal-to-evidence matrices.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
docs/help/gpt54-codex-agentic-parity.md Expands parity narrative, adds diagrams, adds end-to-end runbook, updates scenario/matrix and release-gate mapping.
docs/help/gpt54-codex-agentic-parity-maintainers.md Updates maintainer review guidance to the expanded program and adds a documentation (PR M) merge-unit section.
Comments suppressed due to low confidence (1)

docs/help/gpt54-codex-agentic-parity.md:223

  • This section says the parity pack is now 10 scenarios, but the current QA_AGENTIC_PARITY_SCENARIOS constant still enumerates 5 scenario IDs. Unless the code is updated elsewhere in this PR/branch, the docs will mislead readers about what --parity-pack agentic actually runs and what the parity gate covers. Either update the docs to reflect the current 5-scenario pack or land the scenario-pack expansion first and then update the docs.
## Scenario pack

The parity pack covers ten scenarios after the second-wave expansion:

### `approval-turn-tool-followthrough`

Checks that the model does not stop at “I’ll do that” after a short approval. It should take the first concrete action in the same turn.

Comment thread docs/help/gpt54-codex-agentic-parity.md Outdated
Comment thread docs/help/gpt54-codex-agentic-parity.md Outdated
Comment thread docs/help/gpt54-codex-agentic-parity.md Outdated
Comment thread docs/help/gpt54-codex-agentic-parity-maintainers.md Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 94e76728c1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/help/gpt54-codex-agentic-parity.md Outdated
Comment thread docs/help/gpt54-codex-agentic-parity.md Outdated
Comment thread docs/help/gpt54-codex-agentic-parity.md Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e3c0f63d2b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/help/gpt54-codex-agentic-parity.md Outdated
Comment thread docs/help/gpt54-codex-agentic-parity.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

Comment thread docs/help/gpt54-codex-agentic-parity.md Outdated
Comment thread docs/help/gpt54-codex-agentic-parity.md Outdated
Comment thread docs/help/gpt54-codex-agentic-parity.md Outdated
Comment thread docs/help/gpt54-codex-agentic-parity-maintainers.md Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b46c51fded

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/help/gpt54-codex-agentic-parity.md
@100yenadmin
100yenadmin force-pushed the docs/gpt54-parity-catchup branch from b46c51f to 60e5ee3 Compare April 11, 2026 16:50

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 60e5ee3d9a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/parity-gate.yml

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cc30b85de6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/parity-gate.yml
@100yenadmin
100yenadmin force-pushed the docs/gpt54-parity-catchup branch from cc30b85 to 8a81016 Compare April 11, 2026 18:41

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8a810160ee

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/parity-gate.yml
Comment thread docs/help/gpt54-codex-agentic-parity.md

Copy link
Copy Markdown
Contributor Author

Current head c284723f1a is rebased on latest main and closes the remaining branch-owned workflow/runbook gaps from this round.

What changed on this head:

  • parity-gate workflow now uses a supported Blacksmith runner label
  • parity-gate path filters now include src/gateway/**
  • the mock structural gate stays intentionally on --provider-mode mock-openai, but the workflow now pins explicit --alt-model values so model-switch-tool-continuity stays inside the intended provider lane
  • the runbook now separates the mock structural gate from the live-frontier proof run and also pins explicit live-frontier alternate models

Local validation passed:

  • npx github-actionlint .github/workflows/parity-gate.yml

If any remaining GitHub red is still the shared current-main type/lint churn outside this docs/workflow diff, please treat that as shared CI debt rather than a PR-M-specific failure.

@100yenadmin
100yenadmin force-pushed the docs/gpt54-parity-catchup branch from c284723 to 2c01bdd Compare April 11, 2026 19:52

Copy link
Copy Markdown
Contributor Author

Current head 2c01bddfc2 is rebased onto latest main.

Local validation after the rebase:

  • npx github-actionlint .github/workflows/parity-gate.yml

The workflow/runbook changes remain intentionally mock-openai for CI, with live-frontier documented as the separate release-proof lane. The remaining pnpm build failure reproduces on fresh main and is tracked by shared blocker PR #64972 rather than this docs/workflow branch.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2c01bddfc2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/parity-gate.yml

Copy link
Copy Markdown
Contributor Author

Current head: ed74e3b8e1

Maintainer update after the docs/workflow cleanup:

  • Re-ran:
    • npx github-actionlint .github/workflows/parity-gate.yml
    • passing
  • Current head keeps the workflow/doc split honest:
    • CI is explicitly the mock structural gate
    • the runbook keeps live-frontier GPT-5.4 vs Opus as a separate release-evidence step
    • the workflow now watches the parity-related gateway / qa-channel paths and pins explicit alt-models
  • Program-level proof update:
    • live GPT-5.4 agentic harness passed 10/10 on the integrated patched stack
    • the offline structural parity rerun is now green end-to-end after the public qa-lab follow-ups

From a branch-owned perspective this is ready for review/merge.

@100yenadmin

Copy link
Copy Markdown
Contributor Author

@steipete remaining GPT 5.4 fixes are ready for review. Built and completed live harness test (see PR for it + you can test as well if you like or use in future for tuning for models to openclaw by provider/model). Passes 10/10 parity scenario test side by side vs Opus 4.6 in Openclaw.

@100yenadmin

Copy link
Copy Markdown
Contributor Author

Superseded by #65216. The code, tests, and proof work from this slice were carried forward into the parity proof rollup so maintainers only need to review one proof/release-certification PR. Closing this preserves the discussion history while avoiding review sprawl.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Improvements or additions to documentation size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants