Skip to content

[QA harness] mock Pi provider id and apply_patch failure fixture distort parity #80320

Description

@100yenadmin

Correction TLDR

Status: QA harness bug, not a Codex user-facing apply_patch bug.

The original report conflated two harness problems with product behavior:

  1. Provider/profile setup bug: the Pi cell stayed on mock-openai/<model>, but Pi only exposes apply_patch for OpenAI-compatible provider ids such as openai / openai-codex. The mock server already clones the endpoint under openai, so the parity harness needed to remap forced runtime cells correctly.
  2. Fixture fault-injection boundary: synthetic failure-path inputs must be treated as QA fixture/fault-injection behavior, not proof that Codex users lose patching.

Product impact if OpenClaw moved fully to Codex today: P4. Product patching is not proven broken.

QA impact: resolved for the beta.5 mock hard gate. The corrected tool-defaults direct lane now passes 20/20 with zero skips and zero failures.

Latest Beta.5 Evidence

OpenClaw baseline: v2026.5.10-beta.5
PR: #80323
PR head: 3336dec6419c9cc9a87dc7cfa6f48118ca2d838e
Remote proof run: https://github.com/electricsheephq/openclaw-local-test/actions/runs/25719383976
Confidence tracker: #80936

Relevant artifact result:

{
  "tool-defaults-direct": { "total": 20, "passed": 20, "skipped": 0, "failed": 0 },
  "confidence-report": { "pass": true, "zeroUnknowns": true }
}

Correct Fix

  • Remap mock runtime-pair provider/model ids so the Pi and Codex cells exercise the intended OpenAI-compatible tool surface.
  • Keep apply_patch in default-tool coverage.
  • Do not file this as a product/Codex runtime bug unless a live/native Codex run reproduces patch failure outside this mock fixture.

Superseded Original Report

The earlier wording described this as Pi omitting apply_patch while Codex exposed it. That was true only inside an invalid QA mock provider setup and should not be read as a user-facing runtime regression.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions