test: promote OpenAI HTTP QA coverage#97369
Merged
Merged
Conversation
openclaw-barnacle
Bot
added
extensions: qa-lab
size: XS
maintainer
Contributor
|
ClawSweeper status: review started. I am starting a fresh review of this pull request: test: promote OpenAI HTTP QA coverage This is item 1/1 in the current shard. Shard 0/1. This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking. Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted. |
github-actions Bot
pushed a commit
to Desicool/openclaw
that referenced
this pull request
Jun 28, 2026
QiuYuang
pushed a commit
to QiuYuang/openclaw
that referenced
this pull request
Jul 1, 2026
chenyangjun-xy
pushed a commit
to chenyangjun-xy/openclaw
that referenced
this pull request
Jul 1, 2026
chenyangjun-xy
pushed a commit
to chenyangjun-xy/openclaw
that referenced
this pull request
Jul 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Problem This Solves
Batch 1 of the QA maturity linking work has existing native E2E wrappers for the OpenAI-compatible HTTP/API scripts, but two IDs were still recorded as secondary metadata even though the tests are direct primary proof for those behaviors. That leaves release-score primary coverage understated for the existing scenario inventory.
This PR does not change taxonomy and does not mint any new coverage IDs.
Why This Change Was Made
This promotes only the existing coverage IDs that the current tests directly prove:
openai-compatible-chat-toolstest/e2e/qa-lab/runtime/openai-compatible-chat-tools.e2e.test.tsgateway.openai-compatible-apis;runtime.hosted-tool-useopenwebui-openai-compatibletest/e2e/qa-lab/runtime/openwebui-probe.e2e.test.tsgateway.openai-compatible-apis;runtime.hosted-provider-turnsopenai-web-search-minimaltest/e2e/qa-lab/runtime/openai-web-search-minimal.e2e.test.tsruntime.reasoning-and-cache-controlsopenai-web-search-native-assertionstest/e2e/qa-lab/runtime/openai-web-search-minimal-assertions.e2e.test.tsplugins.web-search-and-fetch;web-search.openai-native-web-searchThe only newly promoted primary IDs in this PR are
runtime.hosted-tool-useandruntime.hosted-provider-turns.gateway.openai-compatible-apis,runtime.reasoning-and-cache-controls,plugins.web-search-and-fetch, andweb-search.openai-native-web-searchwere already primary in the batch-1 wrappers.I am intentionally not claiming
gateway.tool-invocation-apihere: the existing chat-tools test proves OpenAI-compatible/v1/chat/completionstool-call behavior, not the direct gateway/tools/invokeHTTP path.User Impact
The QA coverage inventory and evidence scorecard now count the existing OpenAI-compatible chat-tools and OpenWebUI probe tests as primary proof for the hosted-provider behaviors they already exercise. No runtime behavior, taxonomy, or scenario execution path changes.
Evidence
node scripts/run-vitest.mjs extensions/qa-lab/src/scenario-catalog.test.ts extensions/qa-lab/src/coverage-report.test.ts extensions/qa-lab/src/scorecard-evidence.test.ts test/e2e/qa-lab/runtime/openai-compatible-chat-tools.e2e.test.ts test/e2e/qa-lab/runtime/openwebui-probe.e2e.test.ts test/e2e/qa-lab/runtime/openai-web-search-minimal.e2e.test.ts test/e2e/qa-lab/runtime/openai-web-search-minimal-assertions.e2e.test.tspnpm openclaw qa suite --provider-mode mock-openai --scenario openai-compatible-chat-tools --scenario openwebui-openai-compatible --scenario openai-web-search-minimal --scenario openai-web-search-native-assertions --output-dir .artifacts/qa-e2e/openai-http-api-coverage.artifacts/qa-e2e/openai-http-api-coverage/qa-evidence.jsongit diff --check -- qa/scenarios/runtime/openai-compatible-chat-tools.yaml qa/scenarios/runtime/openwebui-openai-compatible.yaml extensions/qa-lab/src/scenario-catalog.test.ts.agents/skills/autoreview/scripts/autoreview --mode localgateway.tool-invocation-api; accepted and removed