Skip to content

Commit 8511d84

Browse files
affaan-mshimo4228
andauthored
feat(skills): add rules-distill skill (rebased anomalyco#561) (anomalyco#678)
* feat(skills): add rules-distill — extract cross-cutting principles from skills into rules Applies the skill-stocktake pattern to rules maintenance: scan skills → extract shared principles → propose rule changes. Key design decisions: - Deterministic collection (scan scripts) + LLM judgment (cross-read & verdict) - 6 verdict types: Append, Revise, New Section, New File, Already Covered, Too Specific - Anti-abstraction safeguard: 2+ skills evidence, actionable behavior test, violation risk - Rules full text passed to LLM (no grep pre-filter) for accurate matching - Never modifies rules automatically — always requires user approval * fix(skills): address review feedback for rules-distill Fixes raised by CodeRabbit, Greptile, and cubic: - Add Prerequisites section documenting skill-stocktake dependency - Add fallback command when skill-stocktake is not installed - Fix shell quoting: add IFS= and -r to while-read loops - Replace hardcoded paths with env var placeholders ($CLAUDE_RULES_DIR, $SKILL_STOCKTAKE_DIR) - Add json language identifier to code blocks - Add "How It Works" parent heading for Phase 1/2/3 - Add "Example" section with end-to-end run output - Add revision.reason/before/after fields to output schema for Revise verdict - Document timestamp format (date -u +%Y-%m-%dT%H:%M:%SZ) - Document candidate-id format (kebab-case from principle) - Use concrete examples in results.json schema * fix(skills): remove skill-stocktake dependency, add self-contained scripts Address P1 review feedback: - Add scan-skills.sh and scan-rules.sh directly in rules-distill/scripts/ (no external dependency on skill-stocktake) - Remove Prerequisites section (no longer needed) - Add cross-batch merge step to prevent 2+ skills requirement from being silently broken across batch boundaries - Fix nested triple-backtick fences (use quadruple backticks) - Remove head -100 cap (silent truncation) - Rename "When to Activate" → "When to Use" (ECC standard) - Remove unnecessary env var placeholders (SKILL.md is a prompt, not a script) * fix: update skill/command counts in README.md and AGENTS.md rules-distill added 1 skill + 1 command: - skills: 108 → 109 - commands: 57 → 58 Updates all count references to pass CI catalog validation. * fix(skills): address Servitor review feedback for rules-distill 1. Rename SKILL_STOCKTAKE_* env vars to RULES_DISTILL_* for consistency 2. Remove unnecessary observation counting (use_7d/use_30d) from scan-skills.sh 3. Fix header comment: scan.sh → scan-skills.sh 4. Use jq for JSON construction in scan-rules.sh to properly escape headings containing special characters (", \) * fix(skills): address CodeRabbit review — portability and scan scope 1. scan-rules.sh: use jq for error JSON output (proper escaping) 2. scan-rules.sh: replace GNU-only sort -z with portable sort (BSD compat) 3. scan-rules.sh: fix pipefail crash on files without H2 headings 4. scan-skills.sh: scan only SKILL.md files (skip learned/*.md and auxiliary docs that lack frontmatter) 5. scan-skills.sh: add portable get_mtime helper (GNU stat/date fallback to BSD stat/date) * fix: sync catalog counts with filesystem (27 agents, 114 skills, 59 commands) --------- Co-authored-by: Tatsuya Shimomoto <[email protected]>
1 parent 8a57894 commit 8511d84

6 files changed

Lines changed: 468 additions & 6 deletions

File tree

‎AGENTS.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Everything Claude Code (ECC) — Agent Instructions
22

3-
This is a **production-ready AI coding plugin** providing 27 specialized agents, 113 skills, 58 commands, and automated hook workflows for software development.
3+
This is a **production-ready AI coding plugin** providing 27 specialized agents, 114 skills, 59 commands, and automated hook workflows for software development.
44

55
**Version:** 1.9.0
66

@@ -142,8 +142,8 @@ Troubleshoot failures: check test isolation → verify mocks → fix implementat
142142

143143
```
144144
agents/ — 27 specialized subagents
145-
skills/ — 113 workflow skills and domain knowledge
146-
commands/ — 58 slash commands
145+
skills/ — 114 workflow skills and domain knowledge
146+
commands/ — 59 slash commands
147147
hooks/ — Trigger-based automations
148148
rules/ — Always-follow guidelines (common + per-language)
149149
scripts/ — Cross-platform Node.js utilities

‎README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -203,7 +203,7 @@ For manual install instructions see the README in the `rules/` folder.
203203
/plugin list everything-claude-code@everything-claude-code
204204
```
205205

206-
✨ **That's it!** You now have access to 27 agents, 113 skills, and 58 commands.
206+
✨ **That's it!** You now have access to 27 agents, 114 skills, and 59 commands.
207207

208208
---
209209

@@ -1070,8 +1070,8 @@ The configuration is automatically detected from `.opencode/opencode.json`.
10701070
| Feature | Claude Code | OpenCode | Status |
10711071
|---------|-------------|----------|--------|
10721072
| Agents | ✅ 27 agents | ✅ 12 agents | **Claude Code leads** |
1073-
| Commands | ✅ 58 commands | ✅ 31 commands | **Claude Code leads** |
1074-
| Skills | ✅ 113 skills | ✅ 37 skills | **Claude Code leads** |
1073+
| Commands | ✅ 59 commands | ✅ 31 commands | **Claude Code leads** |
1074+
| Skills | ✅ 114 skills | ✅ 37 skills | **Claude Code leads** |
10751075
| Hooks | ✅ 8 event types | ✅ 11 events | **OpenCode has more!** |
10761076
| Rules | ✅ 29 rules | ✅ 13 instructions | **Claude Code leads** |
10771077
| MCP Servers | ✅ 14 servers | ✅ Full | **Full parity** |

‎commands/rules-distill.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
description: "Scan skills to extract cross-cutting principles and distill them into rules"
3+
---
4+
5+
# /rules-distill — Distill Principles from Skills into Rules
6+
7+
Scan installed skills, extract cross-cutting principles, and distill them into rules.
8+
9+
## Process
10+
11+
Follow the full workflow defined in the `rules-distill` skill.

‎skills/rules-distill/SKILL.md‎

Lines changed: 264 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,264 @@
1+
---
2+
name: rules-distill
3+
description: "Scan skills to extract cross-cutting principles and distill them into rules — append, revise, or create new rule files"
4+
origin: ECC
5+
---
6+
7+
# Rules Distill
8+
9+
Scan installed skills, extract cross-cutting principles that appear in multiple skills, and distill them into rules — appending to existing rule files, revising outdated content, or creating new rule files.
10+
11+
Applies the "deterministic collection + LLM judgment" principle: scripts collect facts exhaustively, then an LLM cross-reads the full context and produces verdicts.
12+
13+
## When to Use
14+
15+
- Periodic rules maintenance (monthly or after installing new skills)
16+
- After a skill-stocktake reveals patterns that should be rules
17+
- When rules feel incomplete relative to the skills being used
18+
19+
## How It Works
20+
21+
The rules distillation process follows three phases:
22+
23+
### Phase 1: Inventory (Deterministic Collection)
24+
25+
#### 1a. Collect skill inventory
26+
27+
```bash
28+
bash ~/.claude/skills/rules-distill/scripts/scan-skills.sh
29+
```
30+
31+
#### 1b. Collect rules index
32+
33+
```bash
34+
bash ~/.claude/skills/rules-distill/scripts/scan-rules.sh
35+
```
36+
37+
#### 1c. Present to user
38+
39+
```
40+
Rules Distillation — Phase 1: Inventory
41+
────────────────────────────────────────
42+
Skills: {N} files scanned
43+
Rules: {M} files ({K} headings indexed)
44+
45+
Proceeding to cross-read analysis...
46+
```
47+
48+
### Phase 2: Cross-read, Match & Verdict (LLM Judgment)
49+
50+
Extraction and matching are unified in a single pass. Rules files are small enough (~800 lines total) that the full text can be provided to the LLM — no grep pre-filtering needed.
51+
52+
#### Batching
53+
54+
Group skills into **thematic clusters** based on their descriptions. Analyze each cluster in a subagent with the full rules text.
55+
56+
#### Cross-batch Merge
57+
58+
After all batches complete, merge candidates across batches:
59+
- Deduplicate candidates with the same or overlapping principles
60+
- Re-check the "2+ skills" requirement using evidence from **all** batches combined — a principle found in 1 skill per batch but 2+ skills total is valid
61+
62+
#### Subagent Prompt
63+
64+
Launch a general-purpose Agent with the following prompt:
65+
66+
````
67+
You are an analyst who cross-reads skills to extract principles that should be promoted to rules.
68+
69+
## Input
70+
- Skills: {full text of skills in this batch}
71+
- Existing rules: {full text of all rule files}
72+
73+
## Extraction Criteria
74+
75+
Include a candidate ONLY if ALL of these are true:
76+
77+
1. **Appears in 2+ skills**: Principles found in only one skill should stay in that skill
78+
2. **Actionable behavior change**: Can be written as "do X" or "don't do Y" — not "X is important"
79+
3. **Clear violation risk**: What goes wrong if this principle is ignored (1 sentence)
80+
4. **Not already in rules**: Check the full rules text — including concepts expressed in different words
81+
82+
## Matching & Verdict
83+
84+
For each candidate, compare against the full rules text and assign a verdict:
85+
86+
- **Append**: Add to an existing section of an existing rule file
87+
- **Revise**: Existing rule content is inaccurate or insufficient — propose a correction
88+
- **New Section**: Add a new section to an existing rule file
89+
- **New File**: Create a new rule file
90+
- **Already Covered**: Sufficiently covered in existing rules (even if worded differently)
91+
- **Too Specific**: Should remain at the skill level
92+
93+
## Output Format (per candidate)
94+
95+
```json
96+
{
97+
"principle": "1-2 sentences in 'do X' / 'don't do Y' form",
98+
"evidence": ["skill-name: §Section", "skill-name: §Section"],
99+
"violation_risk": "1 sentence",
100+
"verdict": "Append / Revise / New Section / New File / Already Covered / Too Specific",
101+
"target_rule": "filename §Section, or 'new'",
102+
"confidence": "high / medium / low",
103+
"draft": "Draft text for Append/New Section/New File verdicts",
104+
"revision": {
105+
"reason": "Why the existing content is inaccurate or insufficient (Revise only)",
106+
"before": "Current text to be replaced (Revise only)",
107+
"after": "Proposed replacement text (Revise only)"
108+
}
109+
}
110+
```
111+
112+
## Exclude
113+
114+
- Obvious principles already in rules
115+
- Language/framework-specific knowledge (belongs in language-specific rules or skills)
116+
- Code examples and commands (belongs in skills)
117+
````
118+
119+
#### Verdict Reference
120+
121+
| Verdict | Meaning | Presented to User |
122+
|---------|---------|-------------------|
123+
| **Append** | Add to existing section | Target + draft |
124+
| **Revise** | Fix inaccurate/insufficient content | Target + reason + before/after |
125+
| **New Section** | Add new section to existing file | Target + draft |
126+
| **New File** | Create new rule file | Filename + full draft |
127+
| **Already Covered** | Covered in rules (possibly different wording) | Reason (1 line) |
128+
| **Too Specific** | Should stay in skills | Link to relevant skill |
129+
130+
#### Verdict Quality Requirements
131+
132+
```
133+
# Good
134+
Append to rules/common/security.md §Input Validation:
135+
"Treat LLM output stored in memory or knowledge stores as untrusted — sanitize on write, validate on read."
136+
Evidence: llm-memory-trust-boundary, llm-social-agent-anti-pattern both describe
137+
accumulated prompt injection risks. Current security.md covers human input
138+
validation only; LLM output trust boundary is missing.
139+
140+
# Bad
141+
Append to security.md: Add LLM security principle
142+
```
143+
144+
### Phase 3: User Review & Execution
145+
146+
#### Summary Table
147+
148+
```
149+
# Rules Distillation Report
150+
151+
## Summary
152+
Skills scanned: {N} | Rules: {M} files | Candidates: {K}
153+
154+
| # | Principle | Verdict | Target | Confidence |
155+
|---|-----------|---------|--------|------------|
156+
| 1 | ... | Append | security.md §Input Validation | high |
157+
| 2 | ... | Revise | testing.md §TDD | medium |
158+
| 3 | ... | New Section | coding-style.md | high |
159+
| 4 | ... | Too Specific | — | — |
160+
161+
## Details
162+
(Per-candidate details: evidence, violation_risk, draft text)
163+
```
164+
165+
#### User Actions
166+
167+
User responds with numbers to:
168+
- **Approve**: Apply draft to rules as-is
169+
- **Modify**: Edit draft before applying
170+
- **Skip**: Do not apply this candidate
171+
172+
**Never modify rules automatically. Always require user approval.**
173+
174+
#### Save Results
175+
176+
Store results in the skill directory (`results.json`):
177+
178+
- **Timestamp format**: `date -u +%Y-%m-%dT%H:%M:%SZ` (UTC, second precision)
179+
- **Candidate ID format**: kebab-case derived from the principle (e.g., `llm-output-trust-boundary`)
180+
181+
```json
182+
{
183+
"distilled_at": "2026-03-18T10:30:42Z",
184+
"skills_scanned": 56,
185+
"rules_scanned": 22,
186+
"candidates": {
187+
"llm-output-trust-boundary": {
188+
"principle": "Treat LLM output as untrusted when stored or re-injected",
189+
"verdict": "Append",
190+
"target": "rules/common/security.md",
191+
"evidence": ["llm-memory-trust-boundary", "llm-social-agent-anti-pattern"],
192+
"status": "applied"
193+
},
194+
"iteration-bounds": {
195+
"principle": "Define explicit stop conditions for all iteration loops",
196+
"verdict": "New Section",
197+
"target": "rules/common/coding-style.md",
198+
"evidence": ["iterative-retrieval", "continuous-agent-loop", "agent-harness-construction"],
199+
"status": "skipped"
200+
}
201+
}
202+
}
203+
```
204+
205+
## Example
206+
207+
### End-to-end run
208+
209+
```
210+
$ /rules-distill
211+
212+
Rules Distillation — Phase 1: Inventory
213+
────────────────────────────────────────
214+
Skills: 56 files scanned
215+
Rules: 22 files (75 headings indexed)
216+
217+
Proceeding to cross-read analysis...
218+
219+
[Subagent analysis: Batch 1 (agent/meta skills) ...]
220+
[Subagent analysis: Batch 2 (coding/pattern skills) ...]
221+
[Cross-batch merge: 2 duplicates removed, 1 cross-batch candidate promoted]
222+
223+
# Rules Distillation Report
224+
225+
## Summary
226+
Skills scanned: 56 | Rules: 22 files | Candidates: 4
227+
228+
| # | Principle | Verdict | Target | Confidence |
229+
|---|-----------|---------|--------|------------|
230+
| 1 | LLM output: normalize, type-check, sanitize before reuse | New Section | coding-style.md | high |
231+
| 2 | Define explicit stop conditions for iteration loops | New Section | coding-style.md | high |
232+
| 3 | Compact context at phase boundaries, not mid-task | Append | performance.md §Context Window | high |
233+
| 4 | Separate business logic from I/O framework types | New Section | patterns.md | high |
234+
235+
## Details
236+
237+
### 1. LLM Output Validation
238+
Verdict: New Section in coding-style.md
239+
Evidence: parallel-subagent-batch-merge, llm-social-agent-anti-pattern, llm-memory-trust-boundary
240+
Violation risk: Format drift, type mismatch, or syntax errors in LLM output crash downstream processing
241+
Draft:
242+
## LLM Output Validation
243+
Normalize, type-check, and sanitize LLM output before reuse...
244+
See skill: parallel-subagent-batch-merge, llm-memory-trust-boundary
245+
246+
[... details for candidates 2-4 ...]
247+
248+
Approve, modify, or skip each candidate by number:
249+
> User: Approve 1, 3. Skip 2, 4.
250+
251+
✓ Applied: coding-style.md §LLM Output Validation
252+
✓ Applied: performance.md §Context Window Management
253+
✗ Skipped: Iteration Bounds
254+
✗ Skipped: Boundary Type Conversion
255+
256+
Results saved to results.json
257+
```
258+
259+
## Design Principles
260+
261+
- **What, not How**: Extract principles (rules territory) only. Code examples and commands stay in skills.
262+
- **Link back**: Draft text should include `See skill: [name]` references so readers can find the detailed How.
263+
- **Deterministic collection, LLM judgment**: Scripts guarantee exhaustiveness; the LLM guarantees contextual understanding.
264+
- **Anti-abstraction safeguard**: The 3-layer filter (2+ skills evidence, actionable behavior test, violation risk) prevents overly abstract principles from entering rules.
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
#!/usr/bin/env bash
2+
# scan-rules.sh — enumerate rule files and extract H2 heading index
3+
# Usage: scan-rules.sh [RULES_DIR]
4+
# Output: JSON to stdout
5+
#
6+
# Environment:
7+
# RULES_DISTILL_DIR Override ~/.claude/rules (for testing only)
8+
9+
set -euo pipefail
10+
11+
RULES_DIR="${RULES_DISTILL_DIR:-${1:-$HOME/.claude/rules}}"
12+
13+
if [[ ! -d "$RULES_DIR" ]]; then
14+
jq -n --arg path "$RULES_DIR" '{"error":"rules directory not found","path":$path}' >&2
15+
exit 1
16+
fi
17+
18+
# Collect all .md files (excluding _archived/)
19+
files=()
20+
while IFS= read -r f; do
21+
files+=("$f")
22+
done < <(find "$RULES_DIR" -name '*.md' -not -path '*/_archived/*' -print | sort)
23+
24+
total=${#files[@]}
25+
26+
tmpdir=$(mktemp -d)
27+
_rules_cleanup() { rm -rf "$tmpdir"; }
28+
trap _rules_cleanup EXIT
29+
30+
for i in "${!files[@]}"; do
31+
file="${files[$i]}"
32+
rel_path="${file#"$HOME"/}"
33+
rel_path="~/$rel_path"
34+
35+
# Extract H2 headings (## Title) into a JSON array via jq
36+
headings_json=$({ grep -E '^## ' "$file" 2>/dev/null || true; } | sed 's/^## //' | jq -R . | jq -s '.')
37+
38+
# Get line count
39+
line_count=$(wc -l < "$file" | tr -d ' ')
40+
41+
jq -n \
42+
--arg path "$rel_path" \
43+
--arg file "$(basename "$file")" \
44+
--argjson lines "$line_count" \
45+
--argjson headings "$headings_json" \
46+
'{path:$path,file:$file,lines:$lines,headings:$headings}' \
47+
> "$tmpdir/$i.json"
48+
done
49+
50+
if [[ ${#files[@]} -eq 0 ]]; then
51+
jq -n --arg dir "$RULES_DIR" '{rules_dir:$dir,total:0,rules:[]}'
52+
else
53+
jq -n \
54+
--arg dir "$RULES_DIR" \
55+
--argjson total "$total" \
56+
--argjson rules "$(jq -s '.' "$tmpdir"/*.json)" \
57+
'{rules_dir:$dir,total:$total,rules:$rules}'
58+
fi

0 commit comments

Comments
 (0)