Skip to content

Add claw-score agent skill#91452

Closed
kevinslin wants to merge 4 commits into
openclaw:mainfrom
kevinslin:dev/kevinlin/claw-score-skill-copy
Closed

Add claw-score agent skill#91452
kevinslin wants to merge 4 commits into
openclaw:mainfrom
kevinslin:dev/kevinlin/claw-score-skill-copy

Conversation

@kevinslin

@kevinslin kevinslin commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Summary

  • copy the claw-score agent skill from claw/maintainers branch dev/kevinlin/claw-score-discrawl-scope into .agents/skills/claw-score
  • allowlist .agents/skills/claw-score/** in .gitignore so the repo-owned skill can be tracked
  • remove the copied references/completeness/*.md note set for now and make completeness_instructions optional in the skill docs and render/validation scripts
  • remove the stale mem dependency from the copied skill frontmatter

Verification

  • git diff --check origin/main...HEAD -- .gitignore .agents/skills/claw-score
  • python3 .agents/skills/claw-score/scripts/render_scorecard_from_taxonomy.py --validate-only
  • python3 .agents/skills/claw-score/scripts/render_taxonomy_from_taxonomy.py --taxonomy-doc /tmp/claw-score-taxonomy.md --taxonomy-outline-doc /tmp/claw-score-taxonomy-outline.md

@openclaw-barnacle openclaw-barnacle Bot added size: XL maintainer Maintainer-authored PR labels Jun 8, 2026
@clawsweeper

clawsweeper Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed June 21, 2026, 12:26 PM ET / 16:26 UTC.

Summary
The PR adds a repo-local claw-score skill bundle, skill-local taxonomy source, renderer scripts, reference templates, and .gitignore allowlist entries.

PR surface: Docs +1212, Other +12149. Total +13361 across 17 files.

Reproducibility: yes. for the review blockers: source inspection shows the branch uses skill-local taxonomy and generated maturity-scorecard artifacts while current main uses root taxonomy.yaml and docs/maturity-scores.yaml. No runtime bug reproduction applies to this skill-promotion PR.

Review metrics: 2 noteworthy metrics.

  • Repo-Owned Skill Promotion: 1 SKILL.md added, 1 .gitignore allowlist changed. The PR promotes a local agent skill into tracked repo automation, which needs explicit ownership and source-of-truth agreement before merge.
  • Unchecked Python Automation: 6 scripts added under .agents/skills; 0 matched by current skills-python changed-scope regex. The executable scripts are outside the current Python skill CI path, so routine CI does not prove the new automation.

Stored data model
Persistent data-model change detected: migration/backfill/repair: .agents/skills/claw-score/scripts/migrate_maturity_scorecard_layout.py, migration/backfill/repair: .agents/skills/claw-score/scripts/render_scorecard_from_taxonomy.py, vector/embedding metadata: .agents/skills/claw-score/references/surface-report-template.md, vector/embedding metadata: .agents/skills/claw-score/scripts/sync_taxonomy_categories.py. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🧂 unranked krab
Result: blocked until real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Refresh the branch against current main and reconcile with the root taxonomy/docs maturity-scorecard model.
  • [P1] Add redacted terminal output or logs showing the documented skill workflow succeeding after the refresh.
  • After adding proof, update the PR body so ClawSweeper can re-review automatically; if it does not, ask a maintainer to comment @clawsweeper re-review.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body lists commands but no redacted terminal output, logs, screenshots, recording, or linked artifact showing the latest skill workflow succeeding after the patch. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P1] The protected maintainer label means this PR needs explicit maintainer handling rather than cleanup closure.
  • [P1] Merging the branch as-is would revive a skill-local taxonomy and docs/maturity-scorecard inventory workflow that conflicts with current main's root taxonomy.yaml plus docs/maturity-scores.yaml model.
  • [P1] The new .agents Python scripts import PyYAML and sit outside the current skills-python changed-scope path, so skipped or green CI would not prove the added automation.
  • [P1] The PR body lists commands but no redacted terminal output, logs, recording, screenshot, or linked artifact showing the latest workflow succeeding.

Maintainer options:

  1. Refresh Onto Current Scorecard Model (recommended)
    Update the branch to use current main's root taxonomy.yaml and docs/maturity-scores.yaml model, remove the copied skill-local taxonomy/rendered-inventory assumptions, and add proof for any remaining scripts.
  2. Accept The Copied Renderer Stack
    Maintainers can intentionally keep the skill-local taxonomy and Python renderer design, but should explicitly own dependency setup, CI coverage, source synchronization, and proof before merge.
  3. Pause Or Supersede The Branch
    If the current-main skill plus related root-renderer/docs PRs are the chosen direction, keep this branch paused until a maintainer confirms whether it should be closed or replaced.

Next step before merge

  • [P1] Manual review is needed because the protected maintainer label and competing scorecard source models require a maintainer ownership decision before any repair lane is safe.

Security
Cleared: No concrete secret, permission, download, package-publishing, or supply-chain execution defect was found; the blockers are automation correctness and ownership risks.

Review findings

  • [P1] Use the current scorecard source — .agents/skills/claw-score/SKILL.md:61-69
  • [P2] Declare or remove the Python YAML dependency — .agents/skills/claw-score/scripts/render_scorecard_from_taxonomy.py:12-15
  • [P2] Cover .agents skill scripts in CI — .agents/skills/claw-score/scripts/render_score_matrix.py:1
Review details

Best possible solution:

Refresh or replace this branch on top of the current-main root scorecard model, route any renderer work through the active root-taxonomy renderer path if maintainers want it, and add redacted real-output proof before merge.

Do we have a high-confidence way to reproduce the issue?

Yes for the review blockers: source inspection shows the branch uses skill-local taxonomy and generated maturity-scorecard artifacts while current main uses root taxonomy.yaml and docs/maturity-scores.yaml. No runtime bug reproduction applies to this skill-promotion PR.

Is this the best way to solve the issue?

No; this is not the best merge shape while current main has a narrower root-source claw-score model. The safer path is to refresh onto that model or have maintainers explicitly accept the older copied renderer stack.

Full review comments:

  • [P1] Use the current scorecard source — .agents/skills/claw-score/SKILL.md:61-69
    Current main's claw-score skill makes root taxonomy.yaml and docs/maturity-scores.yaml canonical, but this branch points the skill at .agents/skills/claw-score/taxonomy.yaml and docs/maturity-scorecard/** outputs. Merging that shape would revive a competing scorecard source unless maintainers explicitly choose to replace the current model.
    Confidence: 0.91
  • [P2] Declare or remove the Python YAML dependency — .agents/skills/claw-score/scripts/render_scorecard_from_taxonomy.py:12-15
    The documented commands run these scripts directly, but the renderer imports Python yaml and the PR does not add a supported bootstrap or dependency contract for contributors running the .agents skill scripts. Remove the dependency, vendor it through an owned path, or document and validate the required setup.
    Confidence: 0.87
  • [P2] Cover .agents skill scripts in CI — .agents/skills/claw-score/scripts/render_score_matrix.py:1
    The branch adds executable Python scripts under .agents/skills/claw-score/scripts, while current changed-scope routing and the skills-python job only cover skills/**. Add a CI route for this path or move the scripts under an already checked skill automation surface before relying on them.
    Confidence: 0.86

Overall correctness: patch is incorrect
Overall confidence: 0.9

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against c2ee9b0be8ae.

Label changes

Label justifications:

  • P3: This is low-priority repo-local skill and maturity-scorecard maintenance with no current runtime user regression.
  • merge-risk: 🚨 automation: The diff adds .agents Python automation and changes scorecard source-of-truth assumptions that current CI/proof paths do not settle.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🧂 unranked krab.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body lists commands but no redacted terminal output, logs, screenshots, recording, or linked artifact showing the latest skill workflow succeeding after the patch. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Docs +1212, Other +12149. Total +13361 across 17 files.

View PR surface stats
Area Files Added Removed Net
Source 0 0 0 0
Tests 0 0 0 0
Docs 7 1212 0 +1212
Config 0 0 0 0
Generated 0 0 0 0
Other 10 12149 0 +12149
Total 17 13361 0 +13361

What I checked:

Likely related people:

  • RomneyDa: Authored the merged current-main claw-score skill and later taxonomy coverage cleanup that define the active root-source model this PR must reconcile with. (role: recent current-main scorecard skill contributor; confidence: high; commits: 846591b55427, c4ae2be947c7; files: .agents/skills/claw-score/SKILL.md, .agents/skills/claw-score/references/completeness, taxonomy.yaml)
  • kevinslin: Authored prior merged taxonomy/scorecard work and this branch's copied skill direction, so they have direct context on the older model and the split taxonomy PR. (role: original scorecard and taxonomy contributor; confidence: high; commits: a54f50a41ba0, 4c5d8afa38c0, a5f6615d3db1; files: taxonomy.yaml, docs/maturity-scores.yaml, .agents/skills/claw-score/taxonomy.yaml)
  • vincentkoc: Recent history includes the current changed-scope and skills-python gating area that determines whether .agents Python scripts are validated. (role: adjacent CI and skill automation contributor; confidence: medium; commits: 76dabd5214b4, 6a9deb21b8a0, faa8e272912f; files: scripts/ci-changed-scope.mjs, .github/workflows/ci.yml, .gitignore)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e4487735ac

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .agents/skills/claw-score/SKILL.md Outdated
Comment on lines +27 to +34
Treat `docs/kevinslin/maturity-scorecard/` as an artifact root maintained by
this skill. That includes:

- `docs/kevinslin/maturity-scorecard/README.md`
- `docs/kevinslin/maturity-scorecard/taxonomy.md`
- `docs/kevinslin/maturity-scorecard/taxonomy-outline.md`
- `docs/kevinslin/maturity-scorecard/maturity-scorecard.md`
- `docs/kevinslin/maturity-scorecard/inventory/**`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add the declared scorecard artifacts

This skill declares docs/kevinslin/maturity-scorecard/ as the maintained artifact root, and later workflows require README.md, taxonomy.md, maturity-scorecard.md, inventory reports, and LTS.md; however the reviewed tree contains no docs/kevinslin/maturity-scorecard/** files at all. As a result, a user invoking the new skill’s default verification/scoring commands immediately hits missing-file failures instead of a usable scorecard workflow. Either commit the initial artifact root or change the skill/scripts so the documented defaults are generated before they are required.

Useful? React with 👍 / 👎.

Comment thread .agents/skills/claw-score/SKILL.md Outdated
Comment on lines +4 to +5
dependencies:
- mem

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Declare or vendor the Python YAML dependency

The new workflow commands run these Python scripts directly, and they import yaml/PyYAML, but the skill metadata only declares mem and the repo dependency I found is the Node yaml package, not Python PyYAML. In a fresh checkout without global PyYAML, the scoring/render/validation commands exit before doing any work, so agents cannot use the skill as documented unless the dependency is declared, bootstrapped, or removed.

Useful? React with 👍 / 👎.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. labels Jun 8, 2026
@steipete

Copy link
Copy Markdown
Contributor

Closing as superseded by #94263, merged in 846591b5542787fda15be56e97d63fbff94cff75, plus the follow-up taxonomy and renderer commits now on main. The canonical skill uses root taxonomy.yaml, qa/maturity-scores.yaml, QA scenario metadata, and redacted QA evidence; this branch’s 13k-line skill-local taxonomy and script stack conflicts with that owner model and is no longer mergeable. Thanks for the original work that informed the canonical version.

@steipete steipete closed this Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintainer Maintainer-authored PR merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: XL status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants