Skip to content

benchmarks: add GPT-5.4 vs Opus 4.6 agentic parity harness and release gate #64233

Description

@100yenadmin

Parent: #64227

Summary

Add the shared benchmark harness and release gate for GPT-5.4 vs Opus 4.6 so parity claims are evidence-backed.

Scope

  • shared task suite
  • completion / unintended-stop / valid-tool-call metrics
  • targeted auth/proxy mislabel regressions
  • release gate document and CI entrypoint if feasible

Acceptance

  • same prompts, tool surface, sandbox policy, and continuation policy for both models
  • GPT-5.4 must match or beat Opus 4.6 on completion rate
  • equal or lower unintended-stop rate
  • equal or higher valid-tool-call rate
  • zero fake-success cases
  • zero mislabeled auth/proxy/DNS failures

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions