Skip to content

Epic: RQ — Recipe-quality and coverage surfaces (recipe-health ⇄ TestGrid + coverage matrix) #1265

Description

@njhensley

Summary

Make AICR's public quality story coherent across the offline structural matrix (#1224 / ADR-009) and the live AICR TestGrid (the TestGrid epic), and publish a versioned CUJ/CLI coverage matrix. Three children:

  1. RQ1 replaces the literal pending Evidence column in docs/user/recipe-health.md with a deterministic, stable deep-link to each recipe's TestGrid coordinate, using the TestGrid epic's shared mapping function — coordinated into tools/health generator + make targets + public recipe-health matrix #1229's tools/health generator as the Epic: Recipe health tracking (ADR-009 V1) — enumeration, structural soundness, public matrix #1224 v1.1 Evidence rendering. Link only; never duplicate TestGrid data into markdown.
  2. RQ2 adds a warning-only link-integrity bot (clone of the shipped bom-refresh / recipe-evidence pattern) that verifies Evidence links resolve once a recipe has a TestGrid presence — advisory, never blocking.
  3. RQ3 promotes CUJs (training, inference) and CLI-verb coverage from demo+chainsaw-only into a versioned, generated public coverage matrix that runs on main at a cadence, cross-linked from the other two surfaces. RQ3 has no hard dependency and is the lead deliverable — it ships value even if Epic: Recipe health tracking (ADR-009 V1) — enumeration, structural soundness, public matrix #1224 / the TestGrid epic slip.

Children

Epic acceptance criteria

  1. The Evidence column in docs/user/recipe-health.md renders, per recipe with a TestGrid presence, a stable deep-link to its (group, dashboard, tab) coordinate; recipes without a presence retain an honest pending/. The link is byte-deterministic from criteria and survives k8s-version rolls.
  2. No TestGrid pass/fail, duration, or status data is embedded in any markdown matrix — links only. ADR-009's "reproducible from a checkout, no network" property is preserved for the generator's hermetic portion (link construction is offline; link resolution is the bot's job, not the generator's).
  3. A weekly warning-only bot reports any Evidence link that does not resolve for a recipe with a TestGrid presence; it never blocks merges and never edits the doc.
  4. docs/user/coverage-matrix.md is generated, lists which CUJs and which CLI verbs are exercised, on what hardware class, at what cadence, and is registered in docs/index.yml and cross-linked from docs/user/recipe-health.md. It is distinct from — and cross-references — ADR-009 V1's per-recipe declared_coverage Coverage column (009-recipe-health-tracking.md:259-262).
  5. All three artifacts honor the determinism contract (no time.Now()/UUID on the emit path; stable order; MarshalYAMLDeterministic-style output where serialized), pass make qualify (including the make lint MDX gate tools/check-docs-mdx — see RQ3's required splice-marker/exclusion change), and any new Go package clears the 75% coverage floor (.settings.yaml:107) measured both standalone and against the origin/main baseline-delta gate (no per-package regression > 0.5%; no new exported func at 0%).
  6. The coordinate-mapping contract (§2) is the single function the TestGrid epic ships — used by the TestGrid epic's UI, RQ1, and Epic: Recipe health tracking (ADR-009 V1) — enumeration, structural soundness, public matrix #1224's join key — verified by one shared test in the owning package. No epic re-implements the function.

Out of scope (deferred — see Deferred)

Alerting on TestGrid regressions; embedding live TestGrid data in markdown; a blocking merge gate; per-recipe historical trend rendering in markdown; the verify-gated in-cell evidence-freshness state and letter grades / tri-state vocabulary (owned by #1224's v1.1); a REST endpoint for coverage.


Staged from docs/design/012-recipe-quality-surfaces.md.

Metadata

Metadata

Assignees

Type

Fields

No fields configured for Epic.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions