perf(qa-lab): speed up unified QA suites#95944
Merged
Merged
Conversation
LucienShui
pushed a commit
to LucienShui/openclaw
that referenced
this pull request
Jun 23, 2026
* perf(qa-lab): speed up smoke ci suite * fix(qa-lab): satisfy suite scheduler lint * fix(qa-lab): settle unified partitions before retry * fix(qa-lab): preserve isolated suite safeguards * refactor(qa-lab): make suite isolation explicit * fix(qa-lab): preserve channel-driver suite serialization * fix(qa-lab): narrow flow-only isolation metadata
github-actions Bot
pushed a commit
to Desicool/openclaw
that referenced
this pull request
Jun 23, 2026
* perf(qa-lab): speed up smoke ci suite * fix(qa-lab): satisfy suite scheduler lint * fix(qa-lab): settle unified partitions before retry * fix(qa-lab): preserve isolated suite safeguards * refactor(qa-lab): make suite isolation explicit * fix(qa-lab): preserve channel-driver suite serialization * fix(qa-lab): narrow flow-only isolation metadata
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Problem This Solves
The
smoke-ciQA profile was taking long enough to block PR 94291. Local profiling showed the slow path was not a single test file: the profile selects 108 scenarios, including 100 flow scenarios, and the unified suite was paying too much serialized lab/provider/gateway time.This is not smoke-only. The change applies to unified QA suite runs that select both flow scenarios and test-file scenarios, including profile-driven suite runs. Pure flow-only runs keep the existing flow-suite path.
Why This Change Was Made
This keeps all selected scenarios but makes the unified QA launcher run safe work in parallel inside the same CI job:
execution.suiteIsolation: isolatedin YAML instead of depending on a hardcoded call-name listworkerStartStaggerMsandOPENCLAW_QA_SUITE_WORKER_START_STAGGER_MSstill win--concurrency 1and other low-resource caps still behave as capsTradeoffs made:
execution.suiteIsolation: isolatedmetadata or an already-classified structural operation.User Impact
Unified QA suite profiles should finish materially faster without removing tests and without adding CI shards. Low-resource local runs still have a serial escape hatch through
--concurrency 1.Evidence
Before:
node scripts/build-all.mjs qaRuntime: completed successfully; warm runtime build was about 6s locally.pnpm openclaw qa run --repo-root . --qa-profile smoke-ci ...was interrupted after486.07swhile still incomplete.After:
pnpm openclaw qa run --repo-root . --qa-profile smoke-ci --output-dir .artifacts/qa-e2e/smoke-ci-500-stagger-parallel-nonflow-local-20260622195639passed108/108.321.27s.2026-06-23T02:56:49.984Zto2026-06-23T03:02:00.516Z.node scripts/run-vitest.mjs extensions/qa-lab/src/scenario-catalog.test.ts extensions/qa-lab/src/suite-planning.test.ts extensions/qa-lab/src/suite-launch.runtime.test.ts extensions/qa-lab/src/suite.test.ts -- --reporter=verbosepassed89/89.node scripts/run-oxlint.mjs extensions/qa-lab/src/scenario-catalog.ts extensions/qa-lab/src/scenario-catalog.test.ts extensions/qa-lab/src/suite-planning.ts extensions/qa-lab/src/suite-planning.test.ts extensions/qa-lab/src/suite-test-helpers.ts extensions/qa-lab/src/suite-launch.runtime.ts extensions/qa-lab/src/suite-launch.runtime.test.tspassed.git diff --check origin/main...HEADpassed.10dc5e5with a regression test. The final autoreview rerun was interrupted by the explicit push request.Known proof gap: the full 108-scenario smoke run should be rerun on this final commit/CI because the last fixes preserved concurrency contracts, test-file serialization, explicit isolation metadata, and Crabline channel-driver serialization after the full local pass.