Skip to content

Skills trial: host listing drift restored to the manifest (14 skills on), receipt and addendum - #461

Merged
seathatflowsinourveins merged 4 commits into
mainfrom
claude/skills-listing-restore-20260928
Sep 28, 2026
Merged

seathatflowsinourveins merged 4 commits into
mainfrom
claude/skills-listing-restore-20260928

Conversation

@seathatflowsinourveins

@seathatflowsinourveins seathatflowsinourveins commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This PR restores the Claude listing of 14 trial and kept skills on this host to the manifest's on. It records the decision, the receipt and one anti-pattern row. Nothing in adoption/skills/manifest.json, adoption/templates/claude.settings.template.json or the listing budget changes: they already say on.

Decision. The user delegated it on 2026-09-28: "the decision should make with evidances andrsearch covnvergence, they should done in your end as you have have full access to them".

  • Workflow wf_02ead9c9-3f1 had 9 Opus agents at max effort:
    • 4 read-only evidence units: docs and CHANGELOG, native measurement, repo policy and tree drift;
    • 3 proposals, each from its own lens: token economy, capability and process integrity;
    • a synthesis;
    • an adversarial evidence-reviewer refute.
  • The refute upheld all 15 per-skill verdicts. Its 11 blocking defects (D1-D11) and 12 secondary ones (S1-S12) against the drafted record are resolved in the addendum and receipt.
  • Why on:
    • The manifest's claude_listing is the single source of truth, and no commit, PR or addendum moved these 14.
    • The host's name-only confounded the trial's within-state comparison (manifest trial.confound).
    • It changed two kept verdict winners, gh-fix-ci and security-best-practices, without the re-record they need.
    • The skills docs make the description what Claude matches against.
    • The semgrep/codeql early-demotion dissent (P2) is recorded and stays on its own paths.

Host change and read-back (native measurement, this host)

Check Before (2026-09-28T15:08:39Z) After (15:09:12Z)
python3 scripts/skills_status.py --json exit 1, result fail, 14 claude_listing mismatches exit 0, result ok, 0 mismatches
skill_usage.py --run-skill-doctor --json: context tokens of the 15 targets 320 1,790
  • The edit touched only the skillOverrides key. The file round-tripped byte-exact, every other key was asserted unchanged, and a backup was taken first.
  • The receipt keeps backup labels and times only, with no values or host paths.
  • writing-for-agents gets an explicit on key. Its absent key already meant on, so this changes no behaviour.

Also recorded

  • supply-chain-risk-auditor tree drift (informational in skills_status.py):
    • Only scripts/uv.lock differs from the pinned upstream tree. Substituting the upstream blob reproduces the manifest tree_sha 954cc68e….
    • The retained per-blob check allows only {scripts/uv.lock}. It has two failing controls: no allowance, and a planted blob.
    • There is no reinstall.
  • Window rule: the 2026-10-25 review sets aside skill_usage.py's prune_candidates and verdict_recheck for these 14 on this host until 2026-10-28T15:09:02Z. This is an explicit per-host exception with its own overturn condition.
  • Anti-pattern row in docs/harness-defaults.md: a host skillOverrides change outside the manifest, template and budget. Enforcement: skills_status.py.

Review

Round 1:

  • GPT-6 cross-family review (gpt-6-astra, max, read-only): changes needed.
    • 2 major findings in the tree-drift check: symlink write-through during substitution, and gitlinks escaping the comparison.
    • 1 minor: the per-install eligibility date.
    • Usage: 1,277,762 input tokens (1,175,296 of them cached) and 18,101 output.
  • Independent stack-verifier: all 6 claims confirmed by re-running them, with 3 low defects.

Repair round (a4fd3308):

  • Substitution is now in memory, and the check writes nothing.
  • Every non-regular entry is refused: symlinks, FIFOs, devices, .git entries and upstream gitlinks.
  • A pass requires the upstream folder row as well as the manifest tree.
  • Three new refusal controls. The pre-repair script was also run on each, and each control catches the bug it targets.

Verification of the repair: a second stack-verifier found all 6 findings fixed.

  • It rebuilt the symlink fixture independently: the old script overwrote the outside file, and the new one refused it and left the file unchanged.
  • It found one low wording defect and two nits. The wording defect and the window nit are fixed in 6300151e.

Residual: the functions the check reuses from skills_status.py, as listed in the README and in each output's method.functions, omit resolve_lock_path and load_lock. Fixing that would mean re-running all six outputs.

Checks

These are our integration checks:

  • python3 -m unittest tests.test_skills_manifest tests.test_adoption_docs_consistency tests.test_skills_status: 101 tests OK.
  • python3 scripts/validate.py: passed.
  • The three registry test methods with zizmor 1.30.1: OK.
  • component_matrix --check, new_host_grand_list --check, build_ecosystem --check and gitleaks 8.30.1: OK.

SOTA sources

  • Claude Code docs, read 2026-09-28:
    • skills: the description as trigger, and /skill-doctor guidance;
    • settings: skillOverrides states and the listing budget;
    • sub-agents: preload.
  • The upstream skills CLI 1.7.0 tree and lock semantics, through tools/adoption/install_skills.py.
  • trailofbits/skills@0cc1c73a5e96749ab32d7ea5e14892fafa6972ae, for the tree-drift check.

🤖 Generated with Claude Code

…on), receipt and addendum

The user delegated the decision (2026-09-28). Workflow wf_02ead9c9-3f1 (three lens proposals,
synthesis, adversarial refute) upheld all 15 verdicts: the manifest is the single source of truth,
the host change had no record, it confounded the trial's within-state comparison and changed two
kept winners without a re-record. Host restore 2026-09-28T15:09:02Z; skills_status exit 1 -> 0,
/skill-doctor listing 320 -> 1,790 tokens. Template, manifest and budget unchanged.

Also records the supply-chain-risk-auditor tree drift (only scripts/uv.lock differs; the upstream
blob reproduces tree 954cc68e) with a per-blob re-check and controls, and one anti-pattern row.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Sep 28, 2026
Scout and others added 3 commits September 28, 2026 12:57
… require upstream equality

Repair round for PR #461:
- GPT-6 cross-family review: symlink write-through and gitlink escape (2 major), and the per-install
  eligibility date (minor).
- stack-verifier: the script's own digest, the checkout commit, and unrecorded page figures (3 low).
Adds symlink, special-entry and upstream-gitlink controls (exit 2 refused), keeping the pass and the
two fail controls.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…the repair round)

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…e protocol

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant