qa-lab: (GPT 5.4 Parity vs. Opus Agentic) stage mock auth profiles so the parity gate runs without real credentials by 100yenadmin · Pull Request #64909 · openclaw/openclaw

100yenadmin · 2026-04-11T16:36:49Z

Summary

Makes the mock structural parity gate actually runnable without real provider credentials.

After the current rebase, this PR is intentionally narrow. The legacy runtime seam work that was in earlier drafts has been absorbed elsewhere; what this branch still owns is the auth-staging path that the mock parity gate needs in order to reach the mock providers at all.

Current scope:

stage placeholder auth profiles per provider / agent dir in mock-openai mode
apply the staged auth profile config once per provider after the profile files exist
default the gateway-child provider mode through the shared helper so omitted providerMode still stages mock auth correctly

Part of #64227.

Why this still matters

Without this branch, the mock parity lane can still fail before the mock server ever sees a request because the auth resolver refuses to route through the mock provider base URL until a matching auth profile exists.

This branch makes the structural gate self-contained:

openai and anthropic placeholder API-key profiles are staged for the agents the suite uses
the child config is patched to reference those profiles
callers that omit provider mode still get the mock-openai auth staging path through the default-provider-mode helper

The mock server does not validate the key contents. The placeholder is just enough to satisfy the auth resolver so the parity harness can actually hit the local mock server.

What changed

`extensions/qa-lab/src/gateway-child.ts`

stages placeholder mock auth profiles for openai and anthropic
applies the auth-profile config once per provider after staging
resolves the effective provider mode through the shared helper before deciding whether mock auth staging is needed

`extensions/qa-lab/src/gateway-child.test.ts`

covers the staged mock-auth profile path
covers the provider / agent override path
covers the default-provider-mode path so omitted providerMode still stages mock auth correctly

Validation

Current head validation after rebase:

CI=1 pnpm exec vitest run extensions/qa-lab/src/gateway-child.test.ts

Result: 26/26 passing.

Program-level verification update:

this branch is part of what made the offline structural parity rerun possible without real keys
on the integrated patched stack, with this branch plus the qa-lab mock/provider follow-ups, the full offline 10-scenario parity rerun passed end-to-end

Non-goals

no runtime strict-agentic behavior changes
no scenario YAML changes
no parity-report scoring changes
no live-frontier credential path changes

greptile-apps · 2026-04-11T16:40:10Z

Greptile Summary

This PR makes the GPT-5.4 / Opus 4.6 parity gate runnable without real API keys via two narrowly scoped fixes: staging placeholder api_key auth profiles per agent dir in mock-openai mode (so the auth resolver can route requests through providerBaseUrl), and replacing the bundled-dist-breaking dynamic import in legacy.registration.ts with a proper *.runtime.ts seam. Both changes are well-tested and the approach follows documented patterns in CLAUDE.md.

Confidence Score: 5/5

Safe to merge; only one minor P2 style note, both fixes are correct and well-tested.

All findings are P2 (redundant idempotent call inside a loop). No logic errors, security issues, or correctness problems. Tests cover the new staging function including the override path and confirm both config-side and per-agent store-side outputs.

No files require special attention.

Prompt To Fix All With AI

This is a comment left during a code review.
Path: extensions/qa-lab/src/gateway-child.ts
Line: 350-357

Comment:
**Redundant `applyAuthProfileConfig` calls inside the agent loop**

`applyAuthProfileConfig` mutates the global config and is keyed by `profileId`, which is agent-independent. As written, it is called once per `(agentId, provider)` pair, so with the defaults of 2 agents × 2 providers it runs 4 times when 2 would suffice. The operation is idempotent so the result is correct, but moving the config update outside the agentId loop and keeping only `upsertAuthProfile` (the per-agent store write) inside it better expresses the separation of concerns.

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (1): Last reviewed commit: "qa-lab: stage mock auth profiles so the ..." | Re-trigger Greptile}

Copilot

Pull request overview

Enables the qa-lab parity suite to run end-to-end in mock-openai mode without real provider credentials, and fixes legacy context-engine registration so it works in bundled dist builds.

Changes:

Stage placeholder api_key auth profiles for mock runs across the qa suite’s agent directories and patch runtime config to reference them.
Add unit tests covering default and overridden mock-auth staging behavior.
Introduce a legacy.runtime.ts seam and update legacy context-engine registration to avoid the prior bundled-dist dynamic import failure.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
src/context-engine/legacy.runtime.ts	Adds a runtime seam for legacy context engine loading.
src/context-engine/legacy.registration.ts	Switches legacy engine registration to use the runtime seam.
extensions/qa-lab/src/gateway-child.ts	Stages mock auth profiles in `mock-openai` mode and exposes helpers/constants for testing.
extensions/qa-lab/src/gateway-child.test.ts	Adds tests asserting mock auth profiles are staged correctly (defaults + overrides).

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5ebff21375

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

… credentials

100yenadmin · 2026-04-11T23:43:19Z

Current head: 2f3df64151

Maintainer update after the rebase/body cleanup:

Re-ran:
- CI=1 pnpm exec vitest run extensions/qa-lab/src/gateway-child.test.ts
- 26/26 passing
Current head is intentionally narrower than the original draft body was:
- stages placeholder mock auth profiles per provider / agent dir
- applies those auth-profile config changes once per provider after staging
- defaults provider mode through the shared helper so omitted providerMode still stages mock auth correctly
I updated the PR body to match that current rebased scope.
Program-level impact:
- this branch is part of what made the offline structural parity rerun possible without real keys
- on the integrated patched stack, the full offline 10-scenario parity run is now green end-to-end with this auth path in place

I do not see remaining branch-owned review work here. If GitHub is still red, it is not this PR’s auth-staging scope.

100yenadmin · 2026-04-11T23:47:41Z

One more maintainer note on top of the earlier status update: I updated the PR body after the rebase because the old draft still described the legacy runtime seam work. That part is no longer on this branch. The current public head is intentionally just the mock-auth staging / default-provider-mode fix that the offline structural parity rerun depended on.

100yenadmin · 2026-04-12T06:56:49Z

Superseded by #65216. The code, tests, and proof work from this slice were carried forward into the parity proof rollup so maintainers only need to review one proof/release-certification PR. Closing this preserves the discussion history while avoiding review sprawl.

Copilot AI review requested due to automatic review settings April 11, 2026 16:36

openclaw-barnacle Bot added extensions: qa-lab size: M labels Apr 11, 2026

Copilot started reviewing on behalf of 100yenadmin April 11, 2026 16:37 View session

greptile-apps Bot reviewed Apr 11, 2026

View reviewed changes

Comment thread extensions/qa-lab/src/gateway-child.ts Outdated

Copilot AI reviewed Apr 11, 2026

View reviewed changes

Comment thread src/context-engine/legacy.registration.ts Outdated

Comment thread src/context-engine/legacy.runtime.ts Outdated

chatgpt-codex-connector Bot reviewed Apr 11, 2026

View reviewed changes

Comment thread extensions/qa-lab/src/gateway-child.ts Outdated

100yenadmin mentioned this pull request Apr 11, 2026

GPT-5.4 / Codex agentic runtime parity in OpenClaw #64227

Closed

Eva added 2 commits April 12, 2026 06:26

qa-lab: stage mock auth profiles so the parity gate runs without real…

9da7feb

… credentials

fix(qa): clean up mock auth staging follow-ups

2f3df64

100yenadmin force-pushed the fix/qa-mock-auth-profiles branch from 5ebff21 to 2f3df64 Compare April 11, 2026 23:29

openclaw-barnacle Bot added size: S and removed size: M labels Apr 11, 2026

100yenadmin mentioned this pull request Apr 11, 2026

qa-lab: (GPT 5.4 Parity vs. Opus Agentic) add Anthropic /v1/messages mock route for parity baseline #64685

Closed

100yenadmin mentioned this pull request Apr 12, 2026

GPT-5.4 parity proof rollup #65216

Closed

100yenadmin closed this Apr 12, 2026

100yenadmin mentioned this pull request Apr 12, 2026

agents: GPT-5.4 parity proof rollup #65224

Closed

steipete mentioned this pull request Apr 25, 2026

qa-lab: run the parity gate end-to-end in CI and publish artifacts #64878

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

qa-lab: (GPT 5.4 Parity vs. Opus Agentic) stage mock auth profiles so the parity gate runs without real credentials#64909

qa-lab: (GPT 5.4 Parity vs. Opus Agentic) stage mock auth profiles so the parity gate runs without real credentials#64909
100yenadmin wants to merge 2 commits into
openclaw:mainfrom
electricsheephq:fix/qa-mock-auth-profiles

100yenadmin commented Apr 11, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Apr 11, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

100yenadmin commented Apr 11, 2026

Uh oh!

100yenadmin commented Apr 11, 2026

Uh oh!

100yenadmin commented Apr 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

100yenadmin commented Apr 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why this still matters

What changed

extensions/qa-lab/src/gateway-child.ts

extensions/qa-lab/src/gateway-child.test.ts

Validation

Non-goals

Uh oh!

greptile-apps Bot commented Apr 11, 2026

Greptile Summary

Confidence Score: 5/5

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

100yenadmin commented Apr 11, 2026

Uh oh!

100yenadmin commented Apr 11, 2026

Uh oh!

100yenadmin commented Apr 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

100yenadmin commented Apr 11, 2026 •

edited

Loading

`extensions/qa-lab/src/gateway-child.ts`

`extensions/qa-lab/src/gateway-child.test.ts`