docs: ADR-009 — recipe health tracking and communication#1132
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds ADR-009 proposing V1 "Recipe Health Tracking and Communication." The ADR defines hermetic structural health computation across a criteria matrix, grading semantics and roll-up rules, a CLI enumeration seam ( Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/design/009-recipe-health-tracking.md`:
- Around line 1-15: The document uses several acronyms without expansion; update
the ADR title/intro to expand each acronym on first use (e.g., "Architecture
Decision Record (ADR)" for ADR, expand AICR where introduced, "Continuous
Integration/Site Reliability Engineering (CI/SRE)" for CI/SRE, and "You Aren't
Gonna Need It (YAGNI)" for YAGNI), then continue using the short forms
thereafter; ensure the expansions appear the first time each acronym is
mentioned in the ADR-009: Recipe Health Tracking and Communication document and
apply the same change in later sections where those acronyms first occur.
- Around line 436-438: Replace the inconsistent deferred-signal name
chart_pin_drift with the canonical name chart_drift everywhere in this doc so
references match the deferred-features table and earlier sections; update the
occurrence in the sentence containing kwok_scheduling to use chart_drift and
scan the document for any other uses of chart_pin_drift to rename them for
consistency.
- Around line 410-413: Replace the absolute phrasing "the strongest possible
determinism guarantee" in the sentence that begins "V1 signal is hermetic and
offline, so the generated doc is reproducible..." with a neutral, factual
statement describing current behavior (for example: "deterministic under the
listed V1 inputs and constraints" or "reproducible from a checkout with no
network and no wall-clock given the V1 hermetic/offline inputs"), and ensure the
surrounding text keeps the original note that V1 is hermetic and offline and
that the two-axis principle is fixed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: eb2c907a-f9c5-4170-9a06-57096c2149cc
📒 Files selected for processing (1)
docs/design/009-recipe-health-tracking.md
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/design/009-recipe-health-tracking.md (1)
244-245:⚠️ Potential issue | 🟠 Major | ⚡ Quick winAvoid version-bound roadmap commitment wording (
v1.1).“introduced in v1.1” reads like a release commitment. Rephrase to a condition-based statement (e.g., “introduced when the validation axis lands”) to keep the ADR policy-neutral and non-committal.
As per coding guidelines: "Do not invent features, guarantees, timelines, or roadmap commitments in documentation".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/design/009-recipe-health-tracking.md` around lines 244 - 245, Replace the version-bound phrase "introduced in v1.1" with a condition-based formulation (for example "introduced when the validation axis lands" or "introduced once the validation axis is available") so the sentence in the paragraph referencing Supported/Preview/Experimental no longer commits to a release version; update the same sentence that currently contains "introduced in v1.1" to use the new wording and ensure surrounding context still reads smoothly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@docs/design/009-recipe-health-tracking.md`:
- Around line 244-245: Replace the version-bound phrase "introduced in v1.1"
with a condition-based formulation (for example "introduced when the validation
axis lands" or "introduced once the validation axis is available") so the
sentence in the paragraph referencing Supported/Preview/Experimental no longer
commits to a release version; update the same sentence that currently contains
"introduced in v1.1" to use the new wording and ensure surrounding context still
reads smoothly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 95507a7d-b0fc-4c00-af2a-59cca6dafd58
📒 Files selected for processing (1)
docs/design/009-recipe-health-tracking.md
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/design/009-recipe-health-tracking.md (1)
224-224:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winUse tri-state vocabulary consistently in failure-mode examples.
Line 224 states "renders that combo as
F(broken)," but §3 (lines 198-206) deliberately defers letter grades (A–F) in favor of the tri-statepass | warn | failvocabulary. TheFnotation here appears to meanfail, but using the single-letter form is inconsistent and could confuse readers skimming the doc who see "A–F" deferred in one section but "F" used in another.✏️ Proposed fix
- (broken), with the error captured; the report still emits for all other + as `fail` (broken), with the error captured; the report still emits for all other🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/design/009-recipe-health-tracking.md` at line 224, Replace the single-letter grade "F" with the tri-state term "fail" in the sentence that currently reads "A non-transient resolution error for one combo renders that combo as `F`" so it consistently uses the `pass | warn | fail` vocabulary (also update the parenthetical "(broken)" if present to "(fail)" or remove it for clarity); search for that exact phrase in docs/design/009-recipe-health-tracking.md and change occurrences to use "fail" instead of "F" to maintain consistent terminology with §3's tri-state language.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@docs/design/009-recipe-health-tracking.md`:
- Line 224: Replace the single-letter grade "F" with the tri-state term "fail"
in the sentence that currently reads "A non-transient resolution error for one
combo renders that combo as `F`" so it consistently uses the `pass | warn |
fail` vocabulary (also update the parenthetical "(broken)" if present to
"(fail)" or remove it for clarity); search for that exact phrase in
docs/design/009-recipe-health-tracking.md and change occurrences to use "fail"
instead of "F" to maintain consistent terminology with §3's tri-state language.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 37789c38-46b8-465a-b932-3b55f9374136
📒 Files selected for processing (1)
docs/design/009-recipe-health-tracking.md
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
docs/design/009-recipe-health-tracking.md (2)
252-266: 🧹 Nitpick | 🔵 Trivial | 💤 Low valuePromote substantial bold label to heading.
The "V1 public rendering." paragraph spans ~14 lines describing the public matrix structure — a substantial topic. Per coding guidelines, bold labels should be promoted to real headings when naming substantial topics (≥8 content lines).
Consider promoting this to a
####heading or numbered subsection (e.g.,#### V1 public renderingor as part of a restructured §3).As per coding guidelines: "Promote bold labels to real headings only when naming substantial topics (≥8 content lines); keep as bold paragraphs for scaffolding, generic labels, thin sections, and FAQ entries".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/design/009-recipe-health-tracking.md` around lines 252 - 266, The bold label "V1 public rendering." should be promoted to a real subsection heading (e.g., "#### V1 public rendering" or an appropriately numbered subsection) because the paragraph is a substantial topic; update the markdown heading for that block and ensure related references (Coverage, declared_coverage, Evidence, $GITHUB_STEP_SUMMARY and the Supported / Preview / Experimental vocabulary) remain unchanged in the body so links and terms still resolve correctly.
269-374: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick winPromote substantial tooling subsections to headings for better organization.
Section 4 "Tooling surface" contains three major bold-labeled subsections, each describing substantial architectural components:
- "Enumeration is leaf-driven, not cartesian." (~21 lines, lines 269-290)
- "CLI —
aicr recipe listonly in V1." (~20 lines, lines 291-311)- "Business logic — new
pkg/healthpackage." (~62 lines, lines 312-374)Each subsection far exceeds the 8-line threshold for promotion to headings. Consider restructuring §4 with
####subheadings or numbered subsections (4.1, 4.2, 4.3) to improve navigation and document structure.📋 Suggested restructuring
### 4. Tooling surface -**Enumeration is leaf-driven, not cartesian.** The product of all criteria +#### Enumeration strategy + +Enumeration is leaf-driven, not cartesian. The product of all criteria ... -**CLI — `aicr recipe list` only in V1.** Added as a subcommand of the +#### CLI surface + +`aicr recipe list` only ships in V1. Added as a subcommand of the ... -**Business logic — new `pkg/health` package.** V1 has no dependency on +#### Business logic package + +Health computation lives in a new `pkg/health` package. V1 has no dependency on ...As per coding guidelines: "Promote bold labels to real headings only when naming substantial topics (≥8 content lines)".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/design/009-recipe-health-tracking.md` around lines 269 - 374, Section "Tooling surface" uses bold inline labels for three substantial subsections; replace each bold label ("Enumeration is leaf-driven, not cartesian.", "CLI — `aicr recipe list` only in V1.", "Business logic — new `pkg/health` package.") with proper markdown subsection headings (e.g., "#### Enumeration is leaf-driven, not cartesian.", "#### CLI — `aicr recipe list` only in V1.", "#### Business logic — new `pkg/health` package." or numbered 4.1/4.2/4.3) so they appear in navigation and are promoted per the ≥8-line rule, and ensure any internal anchors/links or the table of contents (if present) are updated to match the new heading texts.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/design/009-recipe-health-tracking.md`:
- Around line 224-226: Split the long sentence that begins "A non-transient
resolution error for one combo renders that combo as `fail`, with the error
captured; the report still emits for all other combos — one broken recipe must
not blank the matrix." into two shorter sentences at the semicolon so it reads
like: first sentence ending after "captured." and a second sentence starting
with "The report still emits for all other combos — one broken recipe must not
blank the matrix."; update the line accordingly in the document
`009-recipe-health-tracking.md`.
---
Outside diff comments:
In `@docs/design/009-recipe-health-tracking.md`:
- Around line 252-266: The bold label "V1 public rendering." should be promoted
to a real subsection heading (e.g., "#### V1 public rendering" or an
appropriately numbered subsection) because the paragraph is a substantial topic;
update the markdown heading for that block and ensure related references
(Coverage, declared_coverage, Evidence, $GITHUB_STEP_SUMMARY and the Supported /
Preview / Experimental vocabulary) remain unchanged in the body so links and
terms still resolve correctly.
- Around line 269-374: Section "Tooling surface" uses bold inline labels for
three substantial subsections; replace each bold label ("Enumeration is
leaf-driven, not cartesian.", "CLI — `aicr recipe list` only in V1.", "Business
logic — new `pkg/health` package.") with proper markdown subsection headings
(e.g., "#### Enumeration is leaf-driven, not cartesian.", "#### CLI — `aicr
recipe list` only in V1.", "#### Business logic — new `pkg/health` package." or
numbered 4.1/4.2/4.3) so they appear in navigation and are promoted per the
≥8-line rule, and ensure any internal anchors/links or the table of contents (if
present) are updated to match the new heading texts.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 9947593d-a41a-4897-848e-66d36b3eeb73
📒 Files selected for processing (1)
docs/design/009-recipe-health-tracking.md
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
docs/design/009-recipe-health-tracking.md (3)
77-77:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winDefine BOM acronym on first use.
BOM is introduced here without expansion. Please define it on first mention (e.g., "Bill of Materials (BOM)") before using the acronym throughout the document.
As per coding guidelines: "Define acronyms on first use in documentation".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/design/009-recipe-health-tracking.md` at line 77, Replace the first occurrence of the acronym "BOM" in the sentence containing "current with the same machinery the BOM already uses." with its expanded form "Bill of Materials (BOM)" so the document defines the acronym on first use; thereafter leave subsequent uses as "BOM".
21-21:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winDefine KWOK acronym on first use.
KWOK is introduced here without expansion. Please define it on first mention (e.g., "Kubernetes WithOut Kubelet (KWOK)") before using the acronym in subsequent sections.
As per coding guidelines: "Define acronyms on first use in documentation".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/design/009-recipe-health-tracking.md` at line 21, The term "KWOK" is used without being defined; update the first occurrence of "KWOK" in this document (line containing "while `recipes/evidence/` is empty and KWOK results are ephemeral") to expand the acronym on first use—e.g., change to "Kubernetes WithOut Kubelet (KWOK)"—and thereafter use just "KWOK" throughout the doc.
254-267: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick winConsider converting dense paragraph to a list for clarity.
This paragraph covers multiple distinct aspects of the V1 public rendering (rolled-up status, Coverage column format, Evidence column, prose note, tri-state vocabulary). Consider restructuring as a bulleted or numbered list to improve scannability and make each element more distinct.
As per coding guidelines: "Convert dense paragraphs into lists or tables in documentation" and "Use short, declarative sentences in documentation".
📋 Suggested restructure
#### V1 public rendering The matrix includes: - **Rolled-up structural status**: `pass | warn | fail`, varies across recipes. - **Coverage column**: Derived from `declared_coverage`, shows a compact per-phase summary (e.g., `R:2 D:4 P:1 C:10` — counts of declared checks per phase). The detailed view (`$GITHUB_STEP_SUMMARY`) lists the named checks per phase in full. - **Evidence column**: Shows literal `pending` for every recipe in V1 (honest: no evidence exists yet), with a one-line prose note in the hand-written header explaining why. - **Conservative tri-state vocabulary**: (Supported / Preview / Experimental) is introduced in v1.1 with the validation axis, when it can actually differentiate recipes. Folding everything into a uniform "Preview" today would convey no information and undersell structurally sound recipes.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/design/009-recipe-health-tracking.md` around lines 254 - 267, Convert the dense paragraph under the "V1 public rendering" discussion into a short bulleted list for clarity: break out the concepts into separate list items for "Rolled-up structural status" (`pass | warn | fail`), "Coverage column" (derived from declared_coverage, example format like `R:2 D:4 P:1 C:10` and note that `$GITHUB_STEP_SUMMARY` lists named checks), "Evidence column" (literal `pending` in V1 with the one-line prose header note), and "Conservative tri-state vocabulary" (Supported / Preview / Experimental introduced in v1.1); keep each item as a concise declarative sentence following the suggested restructure.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@docs/design/009-recipe-health-tracking.md`:
- Line 77: Replace the first occurrence of the acronym "BOM" in the sentence
containing "current with the same machinery the BOM already uses." with its
expanded form "Bill of Materials (BOM)" so the document defines the acronym on
first use; thereafter leave subsequent uses as "BOM".
- Line 21: The term "KWOK" is used without being defined; update the first
occurrence of "KWOK" in this document (line containing "while
`recipes/evidence/` is empty and KWOK results are ephemeral") to expand the
acronym on first use—e.g., change to "Kubernetes WithOut Kubelet (KWOK)"—and
thereafter use just "KWOK" throughout the doc.
- Around line 254-267: Convert the dense paragraph under the "V1 public
rendering" discussion into a short bulleted list for clarity: break out the
concepts into separate list items for "Rolled-up structural status" (`pass |
warn | fail`), "Coverage column" (derived from declared_coverage, example format
like `R:2 D:4 P:1 C:10` and note that `$GITHUB_STEP_SUMMARY` lists named
checks), "Evidence column" (literal `pending` in V1 with the one-line prose
header note), and "Conservative tri-state vocabulary" (Supported / Preview /
Experimental introduced in v1.1); keep each item as a concise declarative
sentence following the suggested restructure.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: d1149f9b-81b9-4cab-bc7e-06bf6c04a53a
📒 Files selected for processing (1)
docs/design/009-recipe-health-tracking.md
2 participants
Summary
Adds ADR-009, a design-only proposal for tracking and communicating recipe
health across every supported criteria combination: an enumeration command
(
aicr recipe list) plus a per-recipe structural-health matrix, generatedand published with the same advisory bot-PR machinery the BOM already uses.
Motivation / Context
AICR ships ~50 leaf recipes across criteria combinations, but there is no
durable record of their health and no way to communicate it — not even a
command to enumerate the resolvable catalog. ADR-007 answers "did this run
pass, verifiably?" for one recipe; this ADR is its catalog-wide complement:
"across the whole matrix, what is each recipe's current structural state?"
It also lays the frame for incorporating ADR-007 evidence as it lands.
Fixes: N/A
Related: ADR-007 (Verifiable Recipe Test Evidence), ADR-006 (Image Pinning)
Type of Change
Component(s) Affected
docs/)Implementation Notes
Design-only — no code ships in this PR; implementation will be tracked under
a follow-on epic if the ADR is accepted.
Key decisions:
A–F) and validation posture (evidence-derived). Structural signals must
never read as "validated"; this is the anti-false-confidence backbone.
Every V1 signal is hermetic and offline (no GPU, no network), so the
generated doc is reproducible from a checkout.
declared_coveragedescriptor records which named checks eachvalidation phase declares (not just whether the phase exists) — answering
"which validations are defined for this recipe?" It is informational and
does not move the grade.
(architecture, supply-chain security, DX, CI/SRE, YAGNI). The
evidence-driven validation axis, a committed JSON artifact, a daily cron,
the PR-comment apparatus, and the KWOK / live-chart-drift signals are
deferred — each with a concrete pull-trigger in What V1 does not ship —
because they could not satisfy the ADR's own hermeticity/trust constraints
while
recipes/evidence/is empty and KWOK results are ephemeral.Testing
Documentation-only change; no Go sources touched. The relevant CI is
fern-docs-ci.yaml(lychee link check + MDX safety ondocs/**); the ADR'sinternal cross-links to
006-image-pinning-policy.mdand007-recipe-evidence.mdresolve.Risk Assessment
Rollout notes: N/A — the ADR proposes future work; nothing is implemented
or wired into any gate by this PR.
Checklist
make testwith-race) — N/A (docs-only)make lint) — N/A (docs-only)git commit -S)