Skip to content

ci: add maturity scorecard renderer#94272

Merged
RomneyDa merged 26 commits into
ci/release-qa-profile-evidencefrom
migrate-maturity-scorecard-renderer
Jun 22, 2026
Merged

ci: add maturity scorecard renderer#94272
RomneyDa merged 26 commits into
ci/release-qa-profile-evidencefrom
migrate-maturity-scorecard-renderer

Conversation

@RomneyDa

Copy link
Copy Markdown
Member

Summary

  • add scripts/render-maturity-docs.mjs for deterministic maturity scorecard docs rendering from root taxonomy.yaml and docs/maturity-scores.yaml
  • add pnpm maturity:render and pnpm maturity:check
  • add a Maturity scorecard workflow that checks committed generated docs when present and can upload evidence-enriched docs from downloaded qa-evidence.json artifacts

Verification

  • pnpm maturity:check (skips because generated docs are not initialized on this branch)
  • pnpm maturity:render -- --output-dir <tmp>/out
  • package/workflow parse check
  • git diff --check

Notes

This is split from #94263. It is safe to merge before the generated-doc PR: maturity:check skips only when none of the generated maturity docs exist yet; once the docs PR lands, the same command enforces drift.

@openclaw-barnacle openclaw-barnacle Bot added scripts Repository scripts size: L maintainer Maintainer-authored PR labels Jun 17, 2026
@github-actions github-actions Bot added the dependencies-changed PR changes dependency-related files label Jun 17, 2026
@github-actions

github-actions Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Dependency graph guard cleared

This PR no longer has blocked dependency graph changes. A future dependency graph change requires a fresh /allow-dependencies-change comment after the guard blocks that new head SHA.

  • Current SHA: a6faf066b0a4ab85068cddd703302e2ff5efa296

@clawsweeper

clawsweeper Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs changes before merge. Reviewed June 21, 2026, 5:37 PM ET / 21:37 UTC.

Summary
The PR adds a TypeScript maturity scorecard docs renderer, root maturity:render/maturity:check scripts, and a GitHub Actions workflow for checking and artifact-rendering maturity docs.

PR surface: Config +126, Other +1347. Total +1473 across 3 files.

Reproducibility: yes. for the PR defect: source inspection shows generated links use surface IDs while target headings use surface names, with cli-install-update-onboard-doctor/CLI as a concrete mismatch. I did not run the renderer because this review is read-only, but the failure path is clear from the renderer and taxonomy source.

Review metrics: 3 noteworthy metrics.

  • Root Package Scripts: 2 added. The new maturity:* commands become stable maintainer and CI entry points for generated scorecard docs.
  • Workflow Dispatch Inputs: 3 added. Manual scorecard rendering gains operator-facing artifact source, artifact pattern, and strict-input controls.
  • Generated Markdown Targets: 3 outputs. The renderer/check contract depends on scorecard, taxonomy, and taxonomy-outline Markdown staying aligned with YAML sources.

Root-cause cluster
Relationship: canonical
Canonical: #94272
Summary: This PR is the canonical renderer/workflow slice for the maturity scorecard stack; related items own source data, local skill setup, generated docs, or release QA evidence.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🌊 off-meta tidepool
Patch quality: 🦐 gold shrimp
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P2] Fix the generated surface-anchor contract and rerun focused maturity render/link validation.

Risk before merge

  • [P1] Generated scorecard and taxonomy navigation can ship broken links until the renderer emits stable surface-ID anchors or derives links and headings from one shared slug helper.
  • [P1] This PR is stacked on ci: add release QA profile evidence #95094, so maintainers should keep the landing order or rebase target intentional before merging to main.
  • [P1] The new workflow and root package scripts become maintainer automation entry points; green CI does not prove the generated-doc link contract.

Maintainer options:

  1. Repair Generated Anchors Before Merge (recommended)
    Emit explicit stable anchors for every taxonomy surface ID, or use one shared slug helper for both links and headings, so generated scorecard and taxonomy navigation resolves.
  2. Pause If Link Ownership Moves
    If maintainers want the generated-docs PR to own the final navigation contract, keep this PR paused until that dependency is explicit and the generated docs can prove the links.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Emit explicit stable anchors for every taxonomy surface ID and update both scorecard and taxonomy index links to use the same helper; keep workflow/package-script scope unchanged and run the focused maturity render/link validation.

Next step before merge

  • [P2] The remaining blocker is a narrow mechanical renderer repair that an automated worker can attempt without choosing product direction.

Security
Cleared: No concrete security or supply-chain concern found: the workflow uses read-only permissions, pinned action SHAs, and parses or copies same-repo artifacts rather than executing them.

Review findings

  • [P2] Emit matching surface-id anchors — scripts/qa/render-maturity-docs.ts:1130
Review details

Best possible solution:

Fix the surface-anchor contract in this renderer, keep the workflow/script scope unchanged, and land this scorecard slice before the generated-docs PR consumes the new commands.

Do we have a high-confidence way to reproduce the issue?

Yes for the PR defect: source inspection shows generated links use surface IDs while target headings use surface names, with cli-install-update-onboard-doctor/CLI as a concrete mismatch. I did not run the renderer because this review is read-only, but the failure path is clear from the renderer and taxonomy source.

Is this the best way to solve the issue?

No as-is. The repo-owned renderer/workflow is the right layer and current main does not duplicate it, but the branch should fix generated anchors before merge.

Full review comments:

  • [P2] Emit matching surface-id anchors — scripts/qa/render-maturity-docs.ts:1130
    The scorecard table links each surface to /taxonomy#${surface.id}, but the taxonomy page renders the target as #### ${surface.name} without an explicit matching ID anchor. Surfaces like cli-install-update-onboard-doctor render under the heading CLI, so generated navigation points to anchors that will not exist; emit stable surface-ID anchors or derive both sides from the same slug helper.
    Confidence: 0.93

Overall correctness: patch is incorrect
Overall confidence: 0.88

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 108d6d7eca00.

Label changes

Label justifications:

  • P2: This is a normal-priority maintainer CI/docs-generation improvement with limited runtime blast radius and one concrete pre-merge renderer defect.
  • merge-risk: 🚨 automation: The diff adds a GitHub Actions workflow and root scripts that affect generated-doc checks, artifact rendering, and maintainer CI behavior.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Not applicable: This MEMBER-authored, maintainer-labeled PR is not subject to the external-contributor real behavior proof gate; PR-body commands and CI remain validation context.
Evidence reviewed

PR surface:

Config +126, Other +1347. Total +1473 across 3 files.

View PR surface stats
Area Files Added Removed Net
Source 0 0 0 0
Tests 0 0 0 0
Docs 0 0 0 0
Config 2 126 0 +126
Generated 0 0 0 0
Other 1 1347 0 +1347
Total 3 1473 0 +1473

Acceptance criteria:

  • [P1] pnpm maturity:render -- --output-dir /tmp/openclaw-maturity-docs.
  • [P1] rg 'cli-install-update-onboard-doctor' /tmp/openclaw-maturity-docs/taxonomy.md /tmp/openclaw-maturity-docs/maturity-scorecard.md.
  • [P1] node scripts/run-oxlint-shards.mjs --threads=8.
  • [P1] git diff --check.

What I checked:

  • Repository policy read: Root AGENTS.md and the scoped scripts/AGENTS.md and docs/AGENTS.md were read fully; their generated-artifact and docs-link guidance applies to this workflow/script/docs-generation PR. (AGENTS.md:18, 108d6d7eca00)
  • Current main lacks the renderer surface: A current-main search found no maturity:* package scripts, maturity renderer, maturity scorecard workflow, or generated maturity docs paths, so this PR is not already implemented on main. (package.json:1577, 108d6d7eca00)
  • Generated link defect remains: The scorecard emits /taxonomy#${surface.id} links while the taxonomy renderer emits target sections as #### ${surface.name}, leaving surface IDs such as cli-install-update-onboard-doctor without matching target anchors. (scripts/qa/render-maturity-docs.ts:1130, 367690c63048)
  • Taxonomy ID/name mismatch proof: The taxonomy source has id: cli-install-update-onboard-doctor with display name CLI, which demonstrates why linking to the ID will not match a generated #### CLI heading without an explicit anchor. (taxonomy.yaml:729, 108d6d7eca00)
  • Latest CI state: The earlier lint blocker is no longer current: live PR checks show check-lint and the new Render maturity docs workflow completed successfully on the latest head. (367690c63048)
  • Workflow security posture: The new workflow uses read-only actions/contents permissions, pinned checkout/upload-artifact action SHAs, and parses or copies same-repo artifacts rather than executing downloaded artifact contents. (.github/workflows/maturity-scorecard.yml:42, 367690c63048)

Likely related people:

  • RomneyDa: Authored the merged claw-score skill migration and the current renderer/workflow branch, and also owns the related release-QA evidence branch. (role: recent scorecard workflow contributor; confidence: high; commits: 846591b55427, 367690c63048, a740724b8750; files: .agents/skills/claw-score/SKILL.md, scripts/qa/render-maturity-docs.ts, .github/workflows/maturity-scorecard.yml)
  • kevinslin: Authored the merged taxonomy and maturity score source import that this renderer consumes, with adjacent scorecard skill/history context. (role: taxonomy and score source introducer; confidence: high; commits: a54f50a41ba0; files: taxonomy.yaml, docs/maturity-scores.yaml)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 security-boundary 🚨 May affect sandboxing, authorization, credentials, or sensitive data. merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. labels Jun 17, 2026
@RomneyDa
RomneyDa force-pushed the migrate-maturity-scorecard-renderer branch from b238e94 to c3c6f3b Compare June 19, 2026 19:32
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. merge-risk: 🚨 security-boundary 🚨 May affect sandboxing, authorization, credentials, or sensitive data. labels Jun 19, 2026
@github-actions github-actions Bot removed the dependencies-changed PR changes dependency-related files label Jun 19, 2026
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jun 19, 2026
@RomneyDa
RomneyDa force-pushed the migrate-maturity-scorecard-renderer branch from 4f8d9fe to f76f58f Compare June 19, 2026 20:27
@RomneyDa
RomneyDa changed the base branch from main to ci/release-qa-profile-evidence June 19, 2026 20:27
@clawsweeper clawsweeper Bot added status: 🛠️ actively grinding The PR author has acted after the latest ClawSweeper review and work remains. and removed status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jun 19, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the docs Improvements or additions to documentation label Jun 22, 2026
@RomneyDa
RomneyDa force-pushed the migrate-maturity-scorecard-renderer branch from 739f1af to dde7e35 Compare June 22, 2026 21:14
@RomneyDa

Copy link
Copy Markdown
Member Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@RomneyDa
RomneyDa force-pushed the migrate-maturity-scorecard-renderer branch 6 times, most recently from 9bae017 to 969851a Compare June 22, 2026 21:46
@RomneyDa
RomneyDa force-pushed the migrate-maturity-scorecard-renderer branch from 969851a to a6faf06 Compare June 22, 2026 21:48
@RomneyDa
RomneyDa merged commit 6e725ec into ci/release-qa-profile-evidence Jun 22, 2026
171 checks passed
@RomneyDa
RomneyDa deleted the migrate-maturity-scorecard-renderer branch June 22, 2026 21:56
RomneyDa added a commit that referenced this pull request Jun 22, 2026
* ci: add maturity scorecard renderer

* ci: render qa scorecard evidence

* ci: type maturity docs renderer

* ci: tighten maturity artifact inputs

* ci: move maturity renderer under qa scripts

* ci: share maturity score schema

* ci: centralize maturity taxonomy validation

* ci: move maturity scores under qa

* ci: remove docs maturity score source

* docs: simplify maturity scorecard output

* docs: commit generated maturity scorecard

* docs: group maturity pages

* docs: simplify maturity scorecard dates

* docs: promote maturity nav tab

* docs: clean up maturity pages

* docs: remove maturity outline page

* docs: filter maturity taxonomy doc links

* docs: simplify maturity taxonomy tables

* docs: keep artifact taxonomy links

* docs: simplify lts scorecard display

* docs: clarify maturity score definitions

* docs: derive maturity coverage from evidence

* docs: hide maturity scorecard until evidence

* docs: remove placeholder maturity pages

* docs: keep maturity scores out of pr

* ci: open maturity scorecard docs pr
RomneyDa added a commit that referenced this pull request Jun 23, 2026
* ci: add maturity scorecard renderer

* ci: render qa scorecard evidence

* ci: type maturity docs renderer

* ci: tighten maturity artifact inputs

* ci: move maturity renderer under qa scripts

* ci: share maturity score schema

* ci: centralize maturity taxonomy validation

* ci: move maturity scores under qa

* ci: remove docs maturity score source

* docs: simplify maturity scorecard output

* docs: commit generated maturity scorecard

* docs: group maturity pages

* docs: simplify maturity scorecard dates

* docs: promote maturity nav tab

* docs: clean up maturity pages

* docs: remove maturity outline page

* docs: filter maturity taxonomy doc links

* docs: simplify maturity taxonomy tables

* docs: keep artifact taxonomy links

* docs: simplify lts scorecard display

* docs: clarify maturity score definitions

* docs: derive maturity coverage from evidence

* docs: hide maturity scorecard until evidence

* docs: remove placeholder maturity pages

* docs: keep maturity scores out of pr

* ci: open maturity scorecard docs pr
RomneyDa added a commit that referenced this pull request Jun 23, 2026
* ci: add maturity scorecard renderer

* ci: render qa scorecard evidence

* ci: type maturity docs renderer

* ci: tighten maturity artifact inputs

* ci: move maturity renderer under qa scripts

* ci: share maturity score schema

* ci: centralize maturity taxonomy validation

* ci: move maturity scores under qa

* ci: remove docs maturity score source

* docs: simplify maturity scorecard output

* docs: commit generated maturity scorecard

* docs: group maturity pages

* docs: simplify maturity scorecard dates

* docs: promote maturity nav tab

* docs: clean up maturity pages

* docs: remove maturity outline page

* docs: filter maturity taxonomy doc links

* docs: simplify maturity taxonomy tables

* docs: keep artifact taxonomy links

* docs: simplify lts scorecard display

* docs: clarify maturity score definitions

* docs: derive maturity coverage from evidence

* docs: hide maturity scorecard until evidence

* docs: remove placeholder maturity pages

* docs: keep maturity scores out of pr

* ci: open maturity scorecard docs pr
RomneyDa added a commit that referenced this pull request Jun 23, 2026
* ci: add maturity scorecard renderer

* ci: render qa scorecard evidence

* ci: type maturity docs renderer

* ci: tighten maturity artifact inputs

* ci: move maturity renderer under qa scripts

* ci: share maturity score schema

* ci: centralize maturity taxonomy validation

* ci: move maturity scores under qa

* ci: remove docs maturity score source

* docs: simplify maturity scorecard output

* docs: commit generated maturity scorecard

* docs: group maturity pages

* docs: simplify maturity scorecard dates

* docs: promote maturity nav tab

* docs: clean up maturity pages

* docs: remove maturity outline page

* docs: filter maturity taxonomy doc links

* docs: simplify maturity taxonomy tables

* docs: keep artifact taxonomy links

* docs: simplify lts scorecard display

* docs: clarify maturity score definitions

* docs: derive maturity coverage from evidence

* docs: hide maturity scorecard until evidence

* docs: remove placeholder maturity pages

* docs: keep maturity scores out of pr

* ci: open maturity scorecard docs pr
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jun 23, 2026
* ci: add maturity scorecard renderer

* ci: render qa scorecard evidence

* ci: type maturity docs renderer

* ci: tighten maturity artifact inputs

* ci: move maturity renderer under qa scripts

* ci: share maturity score schema

* ci: centralize maturity taxonomy validation

* ci: move maturity scores under qa

* ci: remove docs maturity score source

* docs: simplify maturity scorecard output

* docs: commit generated maturity scorecard

* docs: group maturity pages

* docs: simplify maturity scorecard dates

* docs: promote maturity nav tab

* docs: clean up maturity pages

* docs: remove maturity outline page

* docs: filter maturity taxonomy doc links

* docs: simplify maturity taxonomy tables

* docs: keep artifact taxonomy links

* docs: simplify lts scorecard display

* docs: clarify maturity score definitions

* docs: derive maturity coverage from evidence

* docs: hide maturity scorecard until evidence

* docs: remove placeholder maturity pages

* docs: keep maturity scores out of pr

* ci: open maturity scorecard docs pr
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

extensions: qa-lab maintainer Maintainer-authored PR merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. P2 Normal backlog priority with limited blast radius. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. scripts Repository scripts size: XL status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant