|
| 1 | +--- |
| 2 | +name: rules-distill |
| 3 | +description: "Scan skills to extract cross-cutting principles and distill them into rules — append, revise, or create new rule files" |
| 4 | +origin: ECC |
| 5 | +--- |
| 6 | + |
| 7 | +# Rules Distill |
| 8 | + |
| 9 | +Scan installed skills, extract cross-cutting principles that appear in multiple skills, and distill them into rules — appending to existing rule files, revising outdated content, or creating new rule files. |
| 10 | + |
| 11 | +Applies the "deterministic collection + LLM judgment" principle: scripts collect facts exhaustively, then an LLM cross-reads the full context and produces verdicts. |
| 12 | + |
| 13 | +## When to Use |
| 14 | + |
| 15 | +- Periodic rules maintenance (monthly or after installing new skills) |
| 16 | +- After a skill-stocktake reveals patterns that should be rules |
| 17 | +- When rules feel incomplete relative to the skills being used |
| 18 | + |
| 19 | +## How It Works |
| 20 | + |
| 21 | +The rules distillation process follows three phases: |
| 22 | + |
| 23 | +### Phase 1: Inventory (Deterministic Collection) |
| 24 | + |
| 25 | +#### 1a. Collect skill inventory |
| 26 | + |
| 27 | +```bash |
| 28 | +bash ~/.claude/skills/rules-distill/scripts/scan-skills.sh |
| 29 | +``` |
| 30 | + |
| 31 | +#### 1b. Collect rules index |
| 32 | + |
| 33 | +```bash |
| 34 | +bash ~/.claude/skills/rules-distill/scripts/scan-rules.sh |
| 35 | +``` |
| 36 | + |
| 37 | +#### 1c. Present to user |
| 38 | + |
| 39 | +``` |
| 40 | +Rules Distillation — Phase 1: Inventory |
| 41 | +──────────────────────────────────────── |
| 42 | +Skills: {N} files scanned |
| 43 | +Rules: {M} files ({K} headings indexed) |
| 44 | +
|
| 45 | +Proceeding to cross-read analysis... |
| 46 | +``` |
| 47 | + |
| 48 | +### Phase 2: Cross-read, Match & Verdict (LLM Judgment) |
| 49 | + |
| 50 | +Extraction and matching are unified in a single pass. Rules files are small enough (~800 lines total) that the full text can be provided to the LLM — no grep pre-filtering needed. |
| 51 | + |
| 52 | +#### Batching |
| 53 | + |
| 54 | +Group skills into **thematic clusters** based on their descriptions. Analyze each cluster in a subagent with the full rules text. |
| 55 | + |
| 56 | +#### Cross-batch Merge |
| 57 | + |
| 58 | +After all batches complete, merge candidates across batches: |
| 59 | +- Deduplicate candidates with the same or overlapping principles |
| 60 | +- Re-check the "2+ skills" requirement using evidence from **all** batches combined — a principle found in 1 skill per batch but 2+ skills total is valid |
| 61 | + |
| 62 | +#### Subagent Prompt |
| 63 | + |
| 64 | +Launch a general-purpose Agent with the following prompt: |
| 65 | + |
| 66 | +```` |
| 67 | +You are an analyst who cross-reads skills to extract principles that should be promoted to rules. |
| 68 | +
|
| 69 | +## Input |
| 70 | +- Skills: {full text of skills in this batch} |
| 71 | +- Existing rules: {full text of all rule files} |
| 72 | +
|
| 73 | +## Extraction Criteria |
| 74 | +
|
| 75 | +Include a candidate ONLY if ALL of these are true: |
| 76 | +
|
| 77 | +1. **Appears in 2+ skills**: Principles found in only one skill should stay in that skill |
| 78 | +2. **Actionable behavior change**: Can be written as "do X" or "don't do Y" — not "X is important" |
| 79 | +3. **Clear violation risk**: What goes wrong if this principle is ignored (1 sentence) |
| 80 | +4. **Not already in rules**: Check the full rules text — including concepts expressed in different words |
| 81 | +
|
| 82 | +## Matching & Verdict |
| 83 | +
|
| 84 | +For each candidate, compare against the full rules text and assign a verdict: |
| 85 | +
|
| 86 | +- **Append**: Add to an existing section of an existing rule file |
| 87 | +- **Revise**: Existing rule content is inaccurate or insufficient — propose a correction |
| 88 | +- **New Section**: Add a new section to an existing rule file |
| 89 | +- **New File**: Create a new rule file |
| 90 | +- **Already Covered**: Sufficiently covered in existing rules (even if worded differently) |
| 91 | +- **Too Specific**: Should remain at the skill level |
| 92 | +
|
| 93 | +## Output Format (per candidate) |
| 94 | +
|
| 95 | +```json |
| 96 | +{ |
| 97 | + "principle": "1-2 sentences in 'do X' / 'don't do Y' form", |
| 98 | + "evidence": ["skill-name: §Section", "skill-name: §Section"], |
| 99 | + "violation_risk": "1 sentence", |
| 100 | + "verdict": "Append / Revise / New Section / New File / Already Covered / Too Specific", |
| 101 | + "target_rule": "filename §Section, or 'new'", |
| 102 | + "confidence": "high / medium / low", |
| 103 | + "draft": "Draft text for Append/New Section/New File verdicts", |
| 104 | + "revision": { |
| 105 | + "reason": "Why the existing content is inaccurate or insufficient (Revise only)", |
| 106 | + "before": "Current text to be replaced (Revise only)", |
| 107 | + "after": "Proposed replacement text (Revise only)" |
| 108 | + } |
| 109 | +} |
| 110 | +``` |
| 111 | +
|
| 112 | +## Exclude |
| 113 | +
|
| 114 | +- Obvious principles already in rules |
| 115 | +- Language/framework-specific knowledge (belongs in language-specific rules or skills) |
| 116 | +- Code examples and commands (belongs in skills) |
| 117 | +```` |
| 118 | + |
| 119 | +#### Verdict Reference |
| 120 | + |
| 121 | +| Verdict | Meaning | Presented to User | |
| 122 | +|---------|---------|-------------------| |
| 123 | +| **Append** | Add to existing section | Target + draft | |
| 124 | +| **Revise** | Fix inaccurate/insufficient content | Target + reason + before/after | |
| 125 | +| **New Section** | Add new section to existing file | Target + draft | |
| 126 | +| **New File** | Create new rule file | Filename + full draft | |
| 127 | +| **Already Covered** | Covered in rules (possibly different wording) | Reason (1 line) | |
| 128 | +| **Too Specific** | Should stay in skills | Link to relevant skill | |
| 129 | + |
| 130 | +#### Verdict Quality Requirements |
| 131 | + |
| 132 | +``` |
| 133 | +# Good |
| 134 | +Append to rules/common/security.md §Input Validation: |
| 135 | +"Treat LLM output stored in memory or knowledge stores as untrusted — sanitize on write, validate on read." |
| 136 | +Evidence: llm-memory-trust-boundary, llm-social-agent-anti-pattern both describe |
| 137 | +accumulated prompt injection risks. Current security.md covers human input |
| 138 | +validation only; LLM output trust boundary is missing. |
| 139 | +
|
| 140 | +# Bad |
| 141 | +Append to security.md: Add LLM security principle |
| 142 | +``` |
| 143 | + |
| 144 | +### Phase 3: User Review & Execution |
| 145 | + |
| 146 | +#### Summary Table |
| 147 | + |
| 148 | +``` |
| 149 | +# Rules Distillation Report |
| 150 | +
|
| 151 | +## Summary |
| 152 | +Skills scanned: {N} | Rules: {M} files | Candidates: {K} |
| 153 | +
|
| 154 | +| # | Principle | Verdict | Target | Confidence | |
| 155 | +|---|-----------|---------|--------|------------| |
| 156 | +| 1 | ... | Append | security.md §Input Validation | high | |
| 157 | +| 2 | ... | Revise | testing.md §TDD | medium | |
| 158 | +| 3 | ... | New Section | coding-style.md | high | |
| 159 | +| 4 | ... | Too Specific | — | — | |
| 160 | +
|
| 161 | +## Details |
| 162 | +(Per-candidate details: evidence, violation_risk, draft text) |
| 163 | +``` |
| 164 | + |
| 165 | +#### User Actions |
| 166 | + |
| 167 | +User responds with numbers to: |
| 168 | +- **Approve**: Apply draft to rules as-is |
| 169 | +- **Modify**: Edit draft before applying |
| 170 | +- **Skip**: Do not apply this candidate |
| 171 | + |
| 172 | +**Never modify rules automatically. Always require user approval.** |
| 173 | + |
| 174 | +#### Save Results |
| 175 | + |
| 176 | +Store results in the skill directory (`results.json`): |
| 177 | + |
| 178 | +- **Timestamp format**: `date -u +%Y-%m-%dT%H:%M:%SZ` (UTC, second precision) |
| 179 | +- **Candidate ID format**: kebab-case derived from the principle (e.g., `llm-output-trust-boundary`) |
| 180 | + |
| 181 | +```json |
| 182 | +{ |
| 183 | + "distilled_at": "2026-03-18T10:30:42Z", |
| 184 | + "skills_scanned": 56, |
| 185 | + "rules_scanned": 22, |
| 186 | + "candidates": { |
| 187 | + "llm-output-trust-boundary": { |
| 188 | + "principle": "Treat LLM output as untrusted when stored or re-injected", |
| 189 | + "verdict": "Append", |
| 190 | + "target": "rules/common/security.md", |
| 191 | + "evidence": ["llm-memory-trust-boundary", "llm-social-agent-anti-pattern"], |
| 192 | + "status": "applied" |
| 193 | + }, |
| 194 | + "iteration-bounds": { |
| 195 | + "principle": "Define explicit stop conditions for all iteration loops", |
| 196 | + "verdict": "New Section", |
| 197 | + "target": "rules/common/coding-style.md", |
| 198 | + "evidence": ["iterative-retrieval", "continuous-agent-loop", "agent-harness-construction"], |
| 199 | + "status": "skipped" |
| 200 | + } |
| 201 | + } |
| 202 | +} |
| 203 | +``` |
| 204 | + |
| 205 | +## Example |
| 206 | + |
| 207 | +### End-to-end run |
| 208 | + |
| 209 | +``` |
| 210 | +$ /rules-distill |
| 211 | +
|
| 212 | +Rules Distillation — Phase 1: Inventory |
| 213 | +──────────────────────────────────────── |
| 214 | +Skills: 56 files scanned |
| 215 | +Rules: 22 files (75 headings indexed) |
| 216 | +
|
| 217 | +Proceeding to cross-read analysis... |
| 218 | +
|
| 219 | +[Subagent analysis: Batch 1 (agent/meta skills) ...] |
| 220 | +[Subagent analysis: Batch 2 (coding/pattern skills) ...] |
| 221 | +[Cross-batch merge: 2 duplicates removed, 1 cross-batch candidate promoted] |
| 222 | +
|
| 223 | +# Rules Distillation Report |
| 224 | +
|
| 225 | +## Summary |
| 226 | +Skills scanned: 56 | Rules: 22 files | Candidates: 4 |
| 227 | +
|
| 228 | +| # | Principle | Verdict | Target | Confidence | |
| 229 | +|---|-----------|---------|--------|------------| |
| 230 | +| 1 | LLM output: normalize, type-check, sanitize before reuse | New Section | coding-style.md | high | |
| 231 | +| 2 | Define explicit stop conditions for iteration loops | New Section | coding-style.md | high | |
| 232 | +| 3 | Compact context at phase boundaries, not mid-task | Append | performance.md §Context Window | high | |
| 233 | +| 4 | Separate business logic from I/O framework types | New Section | patterns.md | high | |
| 234 | +
|
| 235 | +## Details |
| 236 | +
|
| 237 | +### 1. LLM Output Validation |
| 238 | +Verdict: New Section in coding-style.md |
| 239 | +Evidence: parallel-subagent-batch-merge, llm-social-agent-anti-pattern, llm-memory-trust-boundary |
| 240 | +Violation risk: Format drift, type mismatch, or syntax errors in LLM output crash downstream processing |
| 241 | +Draft: |
| 242 | + ## LLM Output Validation |
| 243 | + Normalize, type-check, and sanitize LLM output before reuse... |
| 244 | + See skill: parallel-subagent-batch-merge, llm-memory-trust-boundary |
| 245 | +
|
| 246 | +[... details for candidates 2-4 ...] |
| 247 | +
|
| 248 | +Approve, modify, or skip each candidate by number: |
| 249 | +> User: Approve 1, 3. Skip 2, 4. |
| 250 | +
|
| 251 | +✓ Applied: coding-style.md §LLM Output Validation |
| 252 | +✓ Applied: performance.md §Context Window Management |
| 253 | +✗ Skipped: Iteration Bounds |
| 254 | +✗ Skipped: Boundary Type Conversion |
| 255 | +
|
| 256 | +Results saved to results.json |
| 257 | +``` |
| 258 | + |
| 259 | +## Design Principles |
| 260 | + |
| 261 | +- **What, not How**: Extract principles (rules territory) only. Code examples and commands stay in skills. |
| 262 | +- **Link back**: Draft text should include `See skill: [name]` references so readers can find the detailed How. |
| 263 | +- **Deterministic collection, LLM judgment**: Scripts guarantee exhaustiveness; the LLM guarantees contextual understanding. |
| 264 | +- **Anti-abstraction safeguard**: The 3-layer filter (2+ skills evidence, actionable behavior test, violation risk) prevents overly abstract principles from entering rules. |
0 commit comments