Skip to content

test: link ClawHub packaged plugin QA evidence#97712

Merged
RomneyDa merged 2 commits into
mainfrom
dallin/qa-clawhub-packaged-plugins
Jun 29, 2026
Merged

test: link ClawHub packaged plugin QA evidence#97712
RomneyDa merged 2 commits into
mainfrom
dallin/qa-clawhub-packaged-plugins

Conversation

@RomneyDa

Copy link
Copy Markdown
Member

What Problem This Solves

The QA maturity inventory still had existing ClawHub and packaged-plugin proof that was not represented as native QA scenario evidence. That left these existing coverage IDs under-counted even though the repo already had targeted release/package proof:

  • clawhub.marketplace-list
  • clawhub.npm-pack-local-release-candidate-installs
  • plugins.packaged-bundled-plugins

Why This Change Was Made

This adds three QA scenario YAML entries that link the existing proof into QA Lab metadata without changing taxonomy IDs. The release marketplace proof stays on its existing Docker release lane through scripts/qa/docker-e2e-lane.ts; the release-candidate checklist and packaged bundled plugin coverage use their focused Vitest proof instead of broad or unsharded Docker sweeps.

The underlying scripts stay in scripts/ because these are active release, package, Docker scheduler, or plugin-prerelease pathways rather than orphaned proof scripts.

User Impact

Maintainers get primary QA coverage accounting for the existing ClawHub marketplace, release-candidate package install checklist, and packaged bundled plugin proof. No runtime behavior changes.

Definite primary coverage added:

  • clawhub.marketplace-list: clawhub-marketplace-list runs the package-installed release plugin marketplace lane, which creates a fixture marketplace, lists it as JSON, installs by marketplace shortcut, updates by plugin id, and uninstalls.
  • clawhub.npm-pack-local-release-candidate-installs: clawhub-release-candidate-checklist points at the release checklist tests that carry the exact prepared candidate tarball into npm-update install proof and validate npm preflight tarball metadata/digest before publish.
  • plugins.packaged-bundled-plugins: packaged-bundled-plugin-install-uninstall points at the bundled plugin probe tests that select package-root bundled plugins, reject source-only roots, and assert packaged install/uninstall record paths.

Evidence

  • pnpm exec oxfmt --write scripts/qa/docker-e2e-lane.ts
  • node scripts/run-vitest.mjs extensions/qa-lab/src/scenario-catalog.test.ts test/scripts/release-candidate-checklist.test.ts test/scripts/bundled-plugin-install-uninstall-probe.test.ts --reporter=verbose
  • pnpm openclaw qa coverage --match 'clawhub marketplace' --json
  • pnpm openclaw qa coverage --match 'packaged bundled plugin' --json
  • pnpm openclaw qa coverage --match 'release candidate checklist' --json
  • pnpm openclaw qa coverage --json reports scenarios=153 coverageIds=204 primary=134 secondary=98 and primary rows for all three target IDs.
  • bash -n scripts/e2e/release-plugin-marketplace-docker.sh scripts/e2e/bundled-plugin-install-uninstall-docker.sh scripts/e2e/lib/bundled-plugin-install-uninstall/sweep.sh scripts/e2e/lib/release-plugin-marketplace/scenario.sh
  • .agents/skills/autoreview/scripts/autoreview --mode local clean after fixing the initial unsharded bundled-plugin lane concern.

Heavy Docker lanes were not run locally for this metadata PR; the PR links existing release/package proof and validates the native QA inventory plus focused tests.

@openclaw-barnacle openclaw-barnacle Bot added scripts Repository scripts docker Docker and sandbox tooling extensions: qa-lab size: S maintainer Maintainer-authored PR labels Jun 29, 2026
@clawsweeper

clawsweeper Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 29, 2026, 1:40 PM ET / 17:40 UTC.

Summary
This PR adds two QA scenario YAML wrappers for ClawHub marketplace and packaged bundled plugin proof, plus a release-plugin-marketplace entry in scripts/qa/docker-e2e-lane.ts.

PR surface: Other +64. Total +64 across 3 files.

Reproducibility: not applicable. this is QA metadata wiring rather than a bug report. The review path is source validation of scenario discovery, taxonomy IDs, referenced proof paths, and the latest PR diff.

Review metrics: 1 noteworthy metric.

  • Primary QA coverage IDs: 2 added in latest diff. The PR changes maturity accounting for ClawHub marketplace list and packaged bundled plugins only; the release-candidate primary coverage mentioned in the stale body is no longer in the patch.

Stored data model
Persistent data-model change detected: vector/embedding metadata: qa/scenarios/plugins/clawhub-marketplace-list.yaml. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🌊 off-meta tidepool
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Refresh the PR body so it describes two scenario files and no longer claims the deleted release-candidate scenario is added.
  • [P2] Wait for remaining exact-head CI jobs to finish before merge.

Risk before merge

  • [P1] The PR body and earlier ClawSweeper comment still say three scenario entries and include release-candidate coverage, but the latest head deleted that scenario; update the body before merge so maintainer evidence matches the diff.
  • [P1] Exact-head CI had pending jobs when checked, including QA Smoke CI and dependency/shrinkwrap lanes, so merge should wait for the final required check state.
  • [P1] The heavy release-plugin-marketplace Docker lane was not rerun locally for this metadata PR; maintainers should rely on exact-head CI plus the existing release/package proof before landing.

Maintainer options:

  1. Decide the mitigation before merge
    Land the narrow QA metadata after the PR body reflects the latest two-scenario diff and exact-head checks finish, keeping the existing proof scripts as the execution source instead of duplicating assertions in new scenario logic.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P2] This protected maintainer PR has no narrow repair defect; the remaining action is maintainer review, PR-body refresh, and exact-head check completion.

Security
Cleared: No concrete security or supply-chain concern was found; the diff adds QA metadata and a fixed mapping to an existing repository Docker script without new dependencies, workflows, permissions, lockfiles, or secret handling.

Review details

Best possible solution:

Land the narrow QA metadata after the PR body reflects the latest two-scenario diff and exact-head checks finish, keeping the existing proof scripts as the execution source instead of duplicating assertions in new scenario logic.

Do we have a high-confidence way to reproduce the issue?

Not applicable; this is QA metadata wiring rather than a bug report. The review path is source validation of scenario discovery, taxonomy IDs, referenced proof paths, and the latest PR diff.

Is this the best way to solve the issue?

Yes for the patch shape: current QA Lab contracts already support nested YAML, vitest execution, and script execution with args, and the maturity investigation asks for small wrappers around existing proof. The PR body should be refreshed before merge because it no longer matches the latest two-scenario diff.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 597a0ba43ca3.

Label changes

Label justifications:

  • P3: This is low-risk QA metadata and script-lane wiring for maintainer coverage accounting, with no runtime behavior change.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The contributor real-behavior-proof gate is not applicable because this is a maintainer-authored PR with the maintainer label; the body still lists focused catalog, coverage, syntax, and autoreview validation.
Evidence reviewed

PR surface:

Other +64. Total +64 across 3 files.

View PR surface stats
Area Files Added Removed Net
Source 0 0 0 0
Tests 0 0 0 0
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 3 64 0 +64
Total 3 64 0 +64

What I checked:

Likely related people:

  • RomneyDa: Recent current-main commits introduced and extended scripts/qa/docker-e2e-lane.ts, the same wrapper this PR extends. (role: recent area contributor; confidence: high; commits: 5327340a09ed, 96c8338d5bfa; files: scripts/qa/docker-e2e-lane.ts)
  • Alix-007: Available grafted history attributes the QA catalog, maturity investigation, and proof scripts being linked by this PR to this commit, though the broad graft makes ownership lower-confidence. (role: adjacent existing-proof contributor; confidence: low; commits: 89b5a879090a; files: qa/maturity-coverage-investigation.md, extensions/qa-lab/src/scenario-catalog.ts, scripts/e2e/lib/release-plugin-marketplace/scenario.sh)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. labels Jun 29, 2026
@RomneyDa
RomneyDa merged commit beab0ec into main Jun 29, 2026
74 checks passed
@RomneyDa
RomneyDa deleted the dallin/qa-clawhub-packaged-plugins branch June 29, 2026 17:46
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 30, 2026
* test: link ClawHub packaged plugin QA evidence

* test: narrow ClawHub QA primary links
QiuYuang pushed a commit to QiuYuang/openclaw that referenced this pull request Jul 1, 2026
* test: link ClawHub packaged plugin QA evidence

* test: narrow ClawHub QA primary links
chenyangjun-xy pushed a commit to chenyangjun-xy/openclaw that referenced this pull request Jul 1, 2026
* test: link ClawHub packaged plugin QA evidence

* test: narrow ClawHub QA primary links
chenyangjun-xy pushed a commit to chenyangjun-xy/openclaw that referenced this pull request Jul 3, 2026
* test: link ClawHub packaged plugin QA evidence

* test: narrow ClawHub QA primary links
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docker Docker and sandbox tooling extensions: qa-lab maintainer Maintainer-authored PR P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. scripts Repository scripts size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant