Skip to content

test: fold remaining Docker lanes into QA#97708

Merged
RomneyDa merged 1 commit into
mainfrom
dallin/qa-docker-remaining-lanes
Jun 29, 2026
Merged

test: fold remaining Docker lanes into QA#97708
RomneyDa merged 1 commit into
mainfrom
dallin/qa-docker-remaining-lanes

Conversation

@RomneyDa

@RomneyDa RomneyDa commented Jun 29, 2026

Copy link
Copy Markdown
Member

What Problem This Solves

The QA maturity inventory still treated several existing Docker E2E lanes as unaccounted-for coverage even though they already prove Docker network access, first-run onboarding, local image setup, first-run notes, and Docker-backed agent tooling behavior.

Why This Change Was Made

This folds the remaining Docker lane evidence into QA Lab scenario metadata without adding taxonomy IDs. The PR keeps critical Docker shell entrypoints in place for package/release workflows, moves reusable TypeScript Docker clients into test/e2e/qa-lab/runtime, and adds QA lane wrappers through scripts/qa/docker-e2e-lane.ts.

Definite primary coverage added for existing IDs:

  • docker.compose-network-access via qa/scenarios/runtime/docker-gateway-network.yaml
  • docker.first-run-onboarding via qa/scenarios/runtime/docker-crestodian-first-run.yaml
  • docker.local-image-setup-script via qa/scenarios/runtime/docker-compose-setup.yaml
  • docker.only-first-run-notes via qa/scenarios/runtime/docker-compose-setup.yaml
  • docker.backed-agent-sandbox-support via qa/scenarios/runtime/docker-agent-bundle-mcp-tools.yaml, qa/scenarios/runtime/docker-agents-delete-shared-workspace.yaml, and qa/scenarios/runtime/docker-npm-onboard-channel-agent.yaml

Secondary coverage only:

  • raspberry-pi.first-run-verification is linked secondarily from qa/scenarios/runtime/docker-crestodian-first-run.yaml because the proof is generic Linux/container first-run behavior, not Pi-specific hardware proof.
  • docker.first-run-onboarding is secondary on qa/scenarios/runtime/docker-npm-onboard-channel-agent.yaml because the lane exercises onboarding as setup for Docker-backed channel/agent behavior.

User Impact

Operators and maintainers get more accurate QA maturity accounting for existing Docker proof without changing product behavior or creating new taxonomy coverage IDs. Future QA evidence runs can now associate these Docker lanes with scenario inventory instead of leaving them as loose scripts.

Evidence

  • node scripts/run-vitest.mjs extensions/qa-lab/src/scenario-catalog.test.ts test/scripts/docker-e2e-crestodian.test.ts test/scripts/openai-image-auth-docker-client.test.ts test/scripts/test-projects.test.ts src/docker-setup.e2e.test.ts --reporter=verbose passed.
  • pnpm openclaw qa coverage --json > /tmp/openclaw-qa-coverage-docker-lanes.json passed and reported scenarios=150 primary=131 any=197 unknown=0.
  • Coverage inventory confirmed:
    • docker.compose-network-access: primary docker-gateway-network
    • docker.first-run-onboarding: primary docker-crestodian-first-run, secondary docker-npm-onboard-channel-agent
    • docker.local-image-setup-script: primary docker-compose-setup
    • docker.only-first-run-notes: primary docker-compose-setup
    • docker.backed-agent-sandbox-support: primary docker-agent-bundle-mcp-tools, docker-agents-delete-shared-workspace, docker-npm-onboard-channel-agent
    • raspberry-pi.first-run-verification: secondary docker-crestodian-first-run
  • node --import tsx scripts/qa/docker-e2e-lane.ts --lane agent-bundle-mcp-tools passed locally. This ran the full Docker lane through the new QA adapter, built the package/image, and completed with OK plus { "ok": true, "tool": "dockerProbe__docker_probe" }.
  • bash -n scripts/e2e/agent-bundle-mcp-tools-docker.sh scripts/e2e/crestodian-first-run-docker.sh scripts/e2e/openai-image-auth-docker.sh passed.
  • git diff --check passed.
  • .agents/skills/autoreview/scripts/autoreview --mode local passed with no accepted/actionable findings.

@openclaw-barnacle openclaw-barnacle Bot added scripts Repository scripts docker Docker and sandbox tooling extensions: qa-lab size: M maintainer Maintainer-authored PR labels Jun 29, 2026
@clawsweeper

clawsweeper Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 29, 2026, 3:31 AM ET / 07:31 UTC.

Summary
This PR registers the remaining Docker E2E scripts as QA Lab scenarios, moves three Docker client harnesses under test/e2e/qa-lab/runtime, and updates changed-test routing plus Docker setup coverage assertions.

PR surface: Tests +7, Other +167. Total +174 across 19 files.

Reproducibility: not applicable. This PR is a QA coverage and Docker lane refactor, not a bug report with a current-main failure to reproduce. The relevant check is whether the updated scenario metadata and wrapper paths remain valid.

Review metrics: 1 noteworthy metric.

  • Docker QA lane registrations: 5 added. The PR expands the QA wrapper from release-upgrade lanes to five additional Docker E2E scripts, so maintainers should notice the automation coverage change.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🌊 off-meta tidepool
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Let the remaining broad CI checks finish before landing.

Risk before merge

  • [P1] The patch changes QA/Docker automation paths; this read-only review did not rerun full Docker containers, so maintainers may want one full lane or Testbox/release-gate proof before landing if required CI does not exercise the heavy Docker path.

Maintainer options:

  1. Accept Targeted QA Evidence (recommended)
    Land once required CI is green if maintainers are comfortable with the PR body's targeted tests and the existing Docker lanes covering the heavy runtime path.
  2. Ask For One Full Docker Lane
    Before merge, request a fresh run of one newly registered lane through scripts/qa/docker-e2e-lane.ts --lane crestodian-first-run or equivalent Testbox proof.

Next step before merge

  • No automated repair is needed; this should stay in normal maintainer review and landing because the remaining action is validation/merge judgment, not a line-level defect.

Security
Cleared: The diff touches QA metadata, test clients, and Docker E2E script paths but adds no dependency, secret, permission, release, or third-party code-execution surface that raises a concrete security concern.

Review details

Best possible solution:

Land the QA wrapper and metadata refactor after required CI finishes, with any extra full Docker lane proof captured as maintainer release validation rather than a code repair.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this PR is a QA coverage and Docker lane refactor, not a bug report with a current-main failure to reproduce. The relevant check is whether the updated scenario metadata and wrapper paths remain valid.

Is this the best way to solve the issue?

Yes: the PR uses the existing QA script-scenario contract and existing Docker E2E shell entrypoints instead of inventing a new runner. The only remaining uncertainty is full Docker lane execution, not the code shape.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 0ce10d779302.

Label changes

Label changes:

  • add P3: This is a low-risk maintainer-authored QA/test coverage cleanup with no product behavior change or confirmed user-facing regression.
  • add merge-risk: 🚨 automation: The diff changes QA Lab scenario metadata and Docker E2E script dispatch paths, so merge correctness depends on automation behavior beyond ordinary source compilation.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: This is a maintainer-authored PR, so the external-contributor real-behavior-proof gate does not apply; the PR body still lists targeted command evidence and CI includes passing proof checks.

Label justifications:

  • P3: This is a low-risk maintainer-authored QA/test coverage cleanup with no product behavior change or confirmed user-facing regression.
  • merge-risk: 🚨 automation: The diff changes QA Lab scenario metadata and Docker E2E script dispatch paths, so merge correctness depends on automation behavior beyond ordinary source compilation.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: This is a maintainer-authored PR, so the external-contributor real-behavior-proof gate does not apply; the PR body still lists targeted command evidence and CI includes passing proof checks.
Evidence reviewed

PR surface:

Tests +7, Other +167. Total +174 across 19 files.

View PR surface stats
Area Files Added Removed Net
Source 0 0 0 0
Tests 7 34 27 +7
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 12 177 10 +167
Total 19 211 37 +174

What I checked:

  • Live PR state: Live GitHub data shows this PR is open, authored by RomneyDa, labeled maintainer, mergeable but unstable while some broad checks were still running, with no human review comments yet. (5cb8c23edfac)
  • QA lane wrapper diff: The PR head adds five new Docker lane keys to the existing scripts/qa/docker-e2e-lane.ts wrapper while preserving the current spawn-through-bash contract. (scripts/qa/docker-e2e-lane.ts:10, 5cb8c23edfac)
  • Docker harness mount contract: Current main already mounts test/e2e/qa-lab into /app/test/e2e/qa-lab:ro, so moving Docker client harnesses under that subtree is consistent with the Docker E2E harness path contract. (scripts/lib/docker-e2e-package.sh:260, 0ce10d779302)
  • Scenario schema compatibility: The QA Lab scenario schema supports native script executions with repo-root-relative path and optional args, and requires dotted coverage IDs under coverage.primary/secondary. (extensions/qa-lab/src/scenario-catalog.ts:78, 0ce10d779302)
  • Taxonomy coverage IDs exist: The Docker and Raspberry Pi coverage IDs used by the new scenarios already exist in taxonomy.yaml, including docker.first-run-onboarding, docker.compose-network-access, and docker.backed-agent-sandbox-support. (taxonomy.yaml:6350, 0ce10d779302)
  • Current Docker setup test coverage: Current main already has Docker setup assertions around image build args, onboarding command shape, token redaction, and gateway config writes; the PR only extends the asserted first-run note coverage in that existing test. (src/docker-setup.e2e.test.ts:383, 0ce10d779302)

Likely related people:

  • RomneyDa: Authored and merged the adjacent QA Docker lane wrapper work in chore: fold release upgrade Docker lanes into QA #97689, and live PR search shows several recent merged QA coverage PRs by the same author. (role: recent QA/Docker lane contributor; confidence: high; commits: 5327340a09ed, c563c6f171d4; files: scripts/qa/docker-e2e-lane.ts, qa/scenarios/runtime/*docker*.yaml)
  • wyf027: The shallow checkout blames the existing Docker QA scenario and Docker E2E harness files to commit 63fe5c74021d00721fb62a309d838e6883e6b73c; the commit title is unrelated, so this is useful only as a low-confidence routing signal from available local history. (role: shallow-history current-line owner; confidence: low; commits: 63fe5c74021d; files: qa/scenarios/runtime/docker-compose-setup.yaml, scripts/lib/docker-e2e-package.sh, scripts/e2e/agent-bundle-mcp-tools-docker-client.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. labels Jun 29, 2026
@RomneyDa
RomneyDa merged commit 96c8338 into main Jun 29, 2026
167 of 178 checks passed
@RomneyDa
RomneyDa deleted the dallin/qa-docker-remaining-lanes branch June 29, 2026 07:39
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 30, 2026
QiuYuang pushed a commit to QiuYuang/openclaw that referenced this pull request Jul 1, 2026
chenyangjun-xy pushed a commit to chenyangjun-xy/openclaw that referenced this pull request Jul 1, 2026
chenyangjun-xy pushed a commit to chenyangjun-xy/openclaw that referenced this pull request Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docker Docker and sandbox tooling extensions: qa-lab maintainer Maintainer-authored PR merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. scripts Repository scripts size: M status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant