Skip to content

chore: fold release upgrade Docker lanes into QA#97689

Merged
RomneyDa merged 1 commit into
mainfrom
dallin/qa-release-upgrade-survivor
Jun 29, 2026
Merged

chore: fold release upgrade Docker lanes into QA#97689
RomneyDa merged 1 commit into
mainfrom
dallin/qa-release-upgrade-survivor

Conversation

@RomneyDa

Copy link
Copy Markdown
Member

What Problem This Solves

Release and update Docker E2E lanes already proved package install, release upgrade, update restart, update status, and plugin convergence behavior, but QA Lab could not select them as scenario evidence for the existing maturity coverage IDs.

Why This Change Was Made

This adds QA scenario YAML wrappers for the existing Docker E2E lanes and one tiny scripts/qa/docker-e2e-lane.ts dispatcher that invokes those existing shell entrypoints. The Docker shell scripts stay authoritative for setup and assertions; this PR only makes them selectable through pnpm openclaw qa suite --scenario ... and visible in QA coverage inventory.

Primary coverage added:

  • docker-release-user-journey: docker.release-path-install because it installs the packed tarball in Docker and verifies onboarding, agent turn, plugin install/uninstall, ClickClack messaging, Gateway restart, and doctor.
  • docker-release-upgrade-user-journey: docker.release-path-install, docker.release-workflow because it installs a published baseline, upgrades to the candidate tarball, then reruns the release journey through agent/plugin/channel/Gateway assertions.
  • docker-update-channel-switch: cli.install-kind-switching, cli.update-status-and-rpc because it switches package -> git -> package and asserts openclaw update status --json after each switch.
  • docker-update-restart-auth: cli.managed-gateway-restart because it runs the update-restart-auth upgrade-survivor lane where update owns the managed Gateway restart without caller-owned Gateway auth env.
  • docker-upgrade-survivor: cli.update-status-and-rpc because it runs update/doctor over dirty user state and asserts Gateway health, readiness, and gateway status --require-rpc --json after the update.
  • docker-update-migration: cli.plugin-convergence because it runs the plugin-deps-cleanup upgrade-survivor variant and requires candidate update plus doctor to clean legacy plugin dependency roots before final probes pass.

User Impact

Maintainers can now include these existing release/update Docker proofs in QA Lab evidence runs without duplicating the Docker tests or changing the taxonomy.

Evidence

  • node scripts/run-vitest.mjs extensions/qa-lab/src/scenario-catalog.test.ts --reporter=verbose passed: 30 tests.
  • node scripts/run-oxlint.mjs --tsconfig tsconfig.json scripts/qa/docker-e2e-lane.ts passed.
  • node scripts/run-tsgo.mjs -p tsconfig.json --pretty false passed.
  • git diff --check passed.
  • pnpm openclaw qa coverage --match docker-release --json showed docker-release-user-journey and docker-release-upgrade-user-journey with the expected coverage IDs.
  • pnpm openclaw qa coverage --match docker-update --json showed update-channel-switch, update-restart-auth, and update-migration with the expected coverage IDs.
  • pnpm openclaw qa coverage --match docker-upgrade --json showed upgrade-survivor with the expected coverage IDs.
  • .agents/skills/autoreview/scripts/autoreview --mode local passed with no accepted/actionable findings.

Not run locally: the Docker lanes themselves. This PR does not change their Docker/runtime logic; it wires existing heavy lanes into QA Lab scenario metadata.

@openclaw-barnacle openclaw-barnacle Bot added scripts Repository scripts docker Docker and sandbox tooling extensions: qa-lab size: M maintainer Maintainer-authored PR labels Jun 29, 2026
@clawsweeper

clawsweeper Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 29, 2026, 2:50 AM ET / 06:50 UTC.

Summary
The PR adds six QA scenario YAML wrappers and a scripts/qa dispatcher that maps QA Lab script scenarios to existing release/update Docker E2E shell lanes.

PR surface: Other +255. Total +255 across 7 files.

Reproducibility: not applicable. This PR is QA evidence wiring around existing Docker lanes rather than a reported runtime bug. Source inspection verifies the catalog and script-runner paths that would exercise the new scenarios.

Review metrics: 1 noteworthy metric.

  • QA Docker scenarios added: 6 scenario IDs added. These make existing heavy release/update Docker lanes selectable in QA coverage and suite commands, so the scenario-to-lane mapping is the main review surface.

Stored data model
Persistent data-model change detected: migration/backfill/repair: qa/scenarios/runtime/docker-release-upgrade-user-journey.yaml, migration/backfill/repair: qa/scenarios/runtime/docker-release-user-journey.yaml, migration/backfill/repair: qa/scenarios/runtime/docker-update-channel-switch.yaml, migration/backfill/repair: qa/scenarios/runtime/docker-update-migration.yaml, migration/backfill/repair: qa/scenarios/runtime/docker-update-restart-auth.yaml, migration/backfill/repair: qa/scenarios/runtime/docker-upgrade-survivor.yaml, and 1 more. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🌊 off-meta tidepool
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P2] Run one representative added QA scenario through the new wrapper if maintainers want stronger end-to-end automation proof before merge.

Risk before merge

  • [P1] The PR body explicitly says the Docker lanes themselves were not run locally through the new QA wrapper; if a lane-specific env mapping is wrong, that would appear only when maintainers select one of these new scenarios.

Maintainer options:

  1. Run one wrapper lane before merge (recommended)
    Run one representative added QA scenario through pnpm openclaw qa suite --scenario ... in Docker/Testbox to prove the QA wrapper invokes the shell lane and records evidence as expected.
  2. Accept wrapper-only validation
    Maintainers may accept the catalog, coverage, lint, type, and diff evidence because the authoritative Docker shell scripts are unchanged and already exist as package aliases.

Next step before merge

  • [P2] No automated code repair is identified; the remaining action is maintainer review or optional Docker/Testbox proof for the new wrapper path.

Security
Cleared: No concrete security or supply-chain concern found; the diff adds YAML metadata and a fixed-map dispatcher without dependency, workflow, lockfile, permission, or secret-handling changes.

Review details

Best possible solution:

Keep the Docker shell scripts authoritative and land the QA wrappers after either a representative wrapper lane is run in Docker/Testbox or maintainers intentionally accept wrapper-only validation for this cleanup.

Do we have a high-confidence way to reproduce the issue?

Not applicable; this PR is QA evidence wiring around existing Docker lanes rather than a reported runtime bug. Source inspection verifies the catalog and script-runner paths that would exercise the new scenarios.

Is this the best way to solve the issue?

Yes, this is an appropriate owner-boundary fix: QA Lab gets thin scenario wrappers while the existing Docker shell scripts remain authoritative. A representative wrapper run would reduce the only material proof gap before merge.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against b01e6127d96c.

Label changes

Label changes:

  • add P3: This is low-risk maintainer QA automation cleanup with no direct user runtime behavior change.
  • add merge-risk: 🚨 automation: The diff changes QA Lab proof automation by adding script-dispatched Docker scenarios, and the posted evidence did not run the heavy Docker lanes through that new wrapper.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The PR is authored by a repository member and has the maintainer label, so the external-contributor real behavior proof gate does not apply; the PR body still lists catalog/lint/type evidence and notes the Docker lanes were not run.

Label justifications:

  • P3: This is low-risk maintainer QA automation cleanup with no direct user runtime behavior change.
  • merge-risk: 🚨 automation: The diff changes QA Lab proof automation by adding script-dispatched Docker scenarios, and the posted evidence did not run the heavy Docker lanes through that new wrapper.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The PR is authored by a repository member and has the maintainer label, so the external-contributor real behavior proof gate does not apply; the PR body still lists catalog/lint/type evidence and notes the Docker lanes were not run.
Evidence reviewed

PR surface:

Other +255. Total +255 across 7 files.

View PR surface stats
Area Files Added Removed Net
Source 0 0 0 0
Tests 0 0 0 0
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 7 255 0 +255
Total 7 255 0 +255

What I checked:

  • Repository policy read: Root AGENTS.md and scripts/AGENTS.md were read fully; their review, QA scenario, YAML-only scenario, and script-wrapper guidance shaped this review. (AGENTS.md:1, b01e6127d96c)
  • PR diff surface: The PR adds six scenario YAML files plus scripts/qa/docker-e2e-lane.ts, with fixed lane names mapped to existing Docker shell entrypoints. (scripts/qa/docker-e2e-lane.ts:8, c563c6f171d4)
  • Current main does not already contain the wrappers: No matching docker-release/update/upgrade QA scenario files or docker-e2e-lane dispatcher exist on current main, so the PR is still adding unique work. (b01e6127d96c)
  • Catalog discoverability: Current QA Lab recursively discovers nested qa/scenarios/**/*.yaml except index.yaml, so new runtime scenario files do not need a separate index entry. (extensions/qa-lab/src/scenario-catalog.ts:421, b01e6127d96c)
  • Script scenario contract: Script scenarios run through node --import tsx with expanded args, and the runner can produce fallback script-test evidence when a script does not write producer evidence. (extensions/qa-lab/src/test-file-scenario-runner.ts:153, b01e6127d96c)
  • Existing Docker lane aliases: Current package scripts already expose the update-channel-switch, update-migration, update-restart-auth, and upgrade-survivor Docker lanes with the same core env contracts the dispatcher mirrors. (package.json:1848, b01e6127d96c)

Likely related people:

  • RomneyDa: GitHub history shows recent main-branch QA catalog, coverage ID, suite runtime, and taxonomy-profile work in the same area, and this PR author also appears in that merged history. (role: recent QA scenario and coverage contributor; confidence: high; commits: 438f208a7666, 63b13ea837fe, e12cf72b17b8; files: extensions/qa-lab/src/scenario-catalog.ts, docs/concepts/qa-e2e-automation.md)
  • vincentkoc: GitHub history shows repeated recent maintenance of upgrade-survivor Docker scripts and the script scenario runner robustness paths this PR depends on. (role: recent Docker E2E and upgrade-survivor contributor; confidence: high; commits: e7f2b125f628, 0f67474251f6, cc3d346c1564; files: scripts/e2e/upgrade-survivor-docker.sh, scripts/e2e/lib/upgrade-survivor/run.sh, extensions/qa-lab/src/test-file-scenario-runner.ts)
  • Solvely-Colin: Merged history credits Solvely-Colin with adding script-backed evidence scenario support, the QA Lab path used by the new dispatcher. (role: script-backed QA evidence introducer; confidence: medium; commits: 591313e80a5e; files: extensions/qa-lab/src/scenario-catalog.ts, extensions/qa-lab/src/test-file-scenario-runner.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. labels Jun 29, 2026
@RomneyDa
RomneyDa merged commit 5327340 into main Jun 29, 2026
137 of 145 checks passed
@RomneyDa
RomneyDa deleted the dallin/qa-release-upgrade-survivor branch June 29, 2026 06:59
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 30, 2026
QiuYuang pushed a commit to QiuYuang/openclaw that referenced this pull request Jul 1, 2026
chenyangjun-xy pushed a commit to chenyangjun-xy/openclaw that referenced this pull request Jul 1, 2026
chenyangjun-xy pushed a commit to chenyangjun-xy/openclaw that referenced this pull request Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docker Docker and sandbox tooling extensions: qa-lab maintainer Maintainer-authored PR merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. scripts Repository scripts size: M status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant