test: fold remaining Docker lanes into QA#97708
Conversation
|
Codex review: needs maintainer review before merge. Reviewed June 29, 2026, 3:31 AM ET / 07:31 UTC. Summary PR surface: Tests +7, Other +167. Total +174 across 19 files. Reproducibility: not applicable. This PR is a QA coverage and Docker lane refactor, not a bug report with a current-main failure to reproduce. The relevant check is whether the updated scenario metadata and wrapper paths remain valid. Review metrics: 1 noteworthy metric.
Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Risk before merge
Maintainer options:
Next step before merge
Security Review detailsBest possible solution: Land the QA wrapper and metadata refactor after required CI finishes, with any extra full Docker lane proof captured as maintainer release validation rather than a code repair. Do we have a high-confidence way to reproduce the issue? Not applicable: this PR is a QA coverage and Docker lane refactor, not a bug report with a current-main failure to reproduce. The relevant check is whether the updated scenario metadata and wrapper paths remain valid. Is this the best way to solve the issue? Yes: the PR uses the existing QA script-scenario contract and existing Docker E2E shell entrypoints instead of inventing a new runner. The only remaining uncertainty is full Docker lane execution, not the code shape. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 0ce10d779302. Label changesLabel changes:
Label justifications:
Evidence reviewedPR surface: Tests +7, Other +167. Total +174 across 19 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
What Problem This Solves
The QA maturity inventory still treated several existing Docker E2E lanes as unaccounted-for coverage even though they already prove Docker network access, first-run onboarding, local image setup, first-run notes, and Docker-backed agent tooling behavior.
Why This Change Was Made
This folds the remaining Docker lane evidence into QA Lab scenario metadata without adding taxonomy IDs. The PR keeps critical Docker shell entrypoints in place for package/release workflows, moves reusable TypeScript Docker clients into
test/e2e/qa-lab/runtime, and adds QA lane wrappers throughscripts/qa/docker-e2e-lane.ts.Definite primary coverage added for existing IDs:
docker.compose-network-accessviaqa/scenarios/runtime/docker-gateway-network.yamldocker.first-run-onboardingviaqa/scenarios/runtime/docker-crestodian-first-run.yamldocker.local-image-setup-scriptviaqa/scenarios/runtime/docker-compose-setup.yamldocker.only-first-run-notesviaqa/scenarios/runtime/docker-compose-setup.yamldocker.backed-agent-sandbox-supportviaqa/scenarios/runtime/docker-agent-bundle-mcp-tools.yaml,qa/scenarios/runtime/docker-agents-delete-shared-workspace.yaml, andqa/scenarios/runtime/docker-npm-onboard-channel-agent.yamlSecondary coverage only:
raspberry-pi.first-run-verificationis linked secondarily fromqa/scenarios/runtime/docker-crestodian-first-run.yamlbecause the proof is generic Linux/container first-run behavior, not Pi-specific hardware proof.docker.first-run-onboardingis secondary onqa/scenarios/runtime/docker-npm-onboard-channel-agent.yamlbecause the lane exercises onboarding as setup for Docker-backed channel/agent behavior.User Impact
Operators and maintainers get more accurate QA maturity accounting for existing Docker proof without changing product behavior or creating new taxonomy coverage IDs. Future QA evidence runs can now associate these Docker lanes with scenario inventory instead of leaving them as loose scripts.
Evidence
node scripts/run-vitest.mjs extensions/qa-lab/src/scenario-catalog.test.ts test/scripts/docker-e2e-crestodian.test.ts test/scripts/openai-image-auth-docker-client.test.ts test/scripts/test-projects.test.ts src/docker-setup.e2e.test.ts --reporter=verbosepassed.pnpm openclaw qa coverage --json > /tmp/openclaw-qa-coverage-docker-lanes.jsonpassed and reportedscenarios=150 primary=131 any=197 unknown=0.docker.compose-network-access: primarydocker-gateway-networkdocker.first-run-onboarding: primarydocker-crestodian-first-run, secondarydocker-npm-onboard-channel-agentdocker.local-image-setup-script: primarydocker-compose-setupdocker.only-first-run-notes: primarydocker-compose-setupdocker.backed-agent-sandbox-support: primarydocker-agent-bundle-mcp-tools,docker-agents-delete-shared-workspace,docker-npm-onboard-channel-agentraspberry-pi.first-run-verification: secondarydocker-crestodian-first-runnode --import tsx scripts/qa/docker-e2e-lane.ts --lane agent-bundle-mcp-toolspassed locally. This ran the full Docker lane through the new QA adapter, built the package/image, and completed withOKplus{ "ok": true, "tool": "dockerProbe__docker_probe" }.bash -n scripts/e2e/agent-bundle-mcp-tools-docker.sh scripts/e2e/crestodian-first-run-docker.sh scripts/e2e/openai-image-auth-docker.shpassed.git diff --checkpassed..agents/skills/autoreview/scripts/autoreview --mode localpassed with no accepted/actionable findings.