fix(qa): keep non-assistant fixtures out of token usage gate#103650
Merged
Conversation
openclaw-barnacle
Bot
added
extensions: qa-lab
size: M
maintainer
Contributor
Author
|
Land-ready proof for exact head
Known proof gap: the full live runtime token/parity lane will be rerun on the landed main SHA. This PR intentionally does not relax the 15% token threshold or any runtime/parity failure gate. |
Contributor
Author
|
Merged via squash.
|
steipete
added a commit
that referenced
this pull request
Jul 10, 2026
github-actions Bot
pushed a commit
to Desicool/openclaw
that referenced
this pull request
Jul 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #103641
What Problem This Solves
Resolves a release-validation problem where runtime token-efficiency proof treated deterministic startup and local fixture scenarios as missing assistant-message usage, even though those scenarios intentionally run no assistant turn.
Why This Change Was Made
Adds explicit, per-scenario usage applicability metadata and carries it in the runtime-parity artifact. Only seven cataloged non-assistant fixtures are marked not applicable. Missing or malformed metadata remains fail-closed as assistant-message-required; runtime failures, drift, required zero usage, and live reports containing only not-applicable captures still fail.
User Impact
Release operators get token aggregates based only on scenarios that can produce assistant usage, while parity failures and insufficient live evidence remain blocking.
Evidence
4bc300843d5ec9f071257c2900701f91df44e1c3identified seven deterministic zero-usage fixture scenarios.tbx_01kx5qan3y2kpr49rd9h4c5ajb: 111 focused QA Lab tests passed on rebased head257c96b1e0c6a260f653855398d848a5ee6a59ae.