Skip to content

fix(corroborate): PARTIAL rollup for incomplete phase coverage#1503

Merged
njhensley merged 1 commit into
NVIDIA:mainfrom
njhensley:corroborate/partial-rollup
Jun 26, 2026
Merged

fix(corroborate): PARTIAL rollup for incomplete phase coverage#1503
njhensley merged 1 commit into
NVIDIA:mainfrom
njhensley:corroborate/partial-rollup

Conversation

@njhensley

Copy link
Copy Markdown
Member

Summary

On the dashboard overview, a recipe corroborated on some phases but missing others (e.g. a contributor without the hardware to run the performance phase) was dragged to UNTESTED, hiding the coverage it did have. This adds a PARTIAL rollup state so incomplete-but-clean coverage reads honestly.

Motivation / Context

The recipe-wide overview rollup (rollupRecipe) picks the worst row state, and UNTESTED ranks above SINGLE/CONFIRMED. So a single not-run row flipped the entire recipe badge to UNTESTED — making a well-corroborated recipe look untested purely because one phase (commonly performance) was never run on the available hardware.

Fixes: N/A
Related: #1404 (GP4 corroborate generator), #1400 (interim evidence dashboard epic)

Type of Change

  • Bug fix (non-breaking change that fixes an issue)

Component(s) Affected

  • Other: pkg/corroborate dashboard renderer (display-only)

Implementation Notes

  • New PARTIAL state used only by rollupRecipe (the overview badge, landing tallies, and left-tree dot) — never a per-row or per-phase state.
  • Rule: positive coverage on some phases plus any not-run rows → PARTIAL. Real problems still dominate (CONTESTED/FAILING win); fully-covered recipes stay CONFIRMED/SINGLE; recipes with no positive coverage anywhere stay UNTESTED.
  • Distinct violet chip (--partial:#a78bfa); amber was already CONTESTED.
  • Renderer-only and display-only. PARTIAL is derived client-side from the per-row consensus already present in index.json. The grid, per-phase cells, dashboards, and per-run views keep UNTESTED/not-run unchanged. No change to the evidence/ingest format, meta.json, the baked JSON schema, or the Go consensus model — a re-publish of the same evidence just renders the overview badge correctly. Byte-determinism of the emit is preserved (static asset).

Testing

go test -race ./pkg/corroborate/...          # embed + byte-determinism: PASS
node --check <renderer script>               # valid JS
# functional test against the extracted rollupRecipe (8 cases): all PASS
#   [SINGLE,UNTESTED]->PARTIAL  [CONFIRMED,UNTESTED]->PARTIAL
#   [CONFIRMED,CONFIRMED]->CONFIRMED  [SINGLE,CONFIRMED]->SINGLE
#   [UNTESTED,UNTESTED]->UNTESTED  []->UNTESTED
#   [FAILING,SINGLE,UNTESTED]->FAILING  [CONTESTED,SINGLE,UNTESTED]->CONTESTED

No Go source files changed (HTML renderer only), so the Go lint gate is N/A.

Risk Assessment

  • Low — Isolated, display-only renderer change, easy to revert.

Rollout notes: Takes effect on the next dashboard publish; no data migration. Backwards-compatible (reads existing index.json).

Checklist

  • Tests pass locally (make test with -race) — go test -race ./pkg/corroborate/...
  • Linter passes (make lint) — no Go files changed; renderer JS syntax checked via node --check
  • I did not skip/disable tests to make CI green
  • I added/updated tests for new functionality — covered by the determinism test + the rollupRecipe functional check above
  • I updated docs if user-facing behavior changed — N/A (no doc surface enumerates rollup states)
  • Changes follow existing patterns in the codebase
  • Commits are cryptographically signed (git commit -S)

…rview

The recipe-wide overview rollup picked the worst row state with UNTESTED
ranked above SINGLE/CONFIRMED, so a single not-run row (e.g. a phase a
contributor lacked the hardware to run, such as performance) dragged the
whole recipe badge to UNTESTED — hiding the coverage it did have.

Add a PARTIAL rollup state used only by rollupRecipe (the overview badge,
landing tallies, and tree dot): a recipe with positive coverage on some
phases plus any not-run rows now reads PARTIAL instead of UNTESTED. Real
problems still dominate (CONTESTED/FAILING win); fully-covered recipes
stay CONFIRMED/SINGLE; recipes with no positive coverage stay UNTESTED.

Renderer-only and display-only: PARTIAL is derived client-side from the
per-row consensus already in index.json. The grid, per-phase cells,
dashboards, and per-run views keep UNTESTED/not-run unchanged, and the
evidence/ingest format, meta.json, and baked JSON schema are untouched.

Signed-off-by: Nathan Hensley <[email protected]>
@njhensley njhensley requested a review from a team as a code owner June 26, 2026 23:19
@njhensley njhensley added the theme/supply-chain SLSA, SBOM, Sigstore, and provenance verification label Jun 26, 2026
@njhensley njhensley enabled auto-merge (squash) June 26, 2026 23:20
@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 897e072b-6b47-4392-beb2-7549e61adcb8

📥 Commits

Reviewing files that changed from the base of the PR and between ec48bbd and 85bcdc4.

📒 Files selected for processing (1)
  • pkg/corroborate/renderer/index.html

📝 Walkthrough

Walkthrough

The renderer adds a new PARTIAL consensus state. It introduces PARTIAL styling and state metadata, updates rollup logic so some UNTESTED results become PARTIAL when confirmed or single counts exist, and extends the landing overview to count and display PARTIAL in the bar order.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: adding PARTIAL rollup behavior for incomplete phase coverage.
Description check ✅ Passed The description matches the implemented dashboard rollup change and accurately explains its purpose and scope.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@njhensley njhensley merged commit fc5e814 into NVIDIA:main Jun 26, 2026
32 of 33 checks passed
@njhensley njhensley deleted the corroborate/partial-rollup branch June 26, 2026 23:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/S theme/supply-chain SLSA, SBOM, Sigstore, and provenance verification

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants