Skip to content

feat(tools): add s3c supply-chain presence checker#791

Merged
mchmarny merged 3 commits into
mainfrom
feat/s3c-supply-chain-check
May 7, 2026
Merged

feat(tools): add s3c supply-chain presence checker#791
mchmarny merged 3 commits into
mainfrom
feat/s3c-supply-chain-check

Conversation

@mchmarny

@mchmarny mchmarny commented May 7, 2026

Copy link
Copy Markdown
Member

Summary

Adds tools/s3c, a presence-only cosign wrapper that probes a component's container images for signatures, SBOMs, and SLSA-style build provenance, and documents the underlying cosign commands in docs/user/container-images.md.

Motivation / Context

Customers performing security review need to know which AICR-deployed images carry cosign signatures, SBOMs, and build provenance. Rather than maintain a static matrix that drifts (or wire flaky CI probes that conflate transient registry errors with absence), this gives users a one-shot check they can run themselves on demand.

Fixes: #745
Related: #739

Type of Change

  • New feature (non-breaking change that adds functionality)
  • Documentation update

Component(s) Affected

  • Docs/examples (docs/, examples/)
  • Other: tools/s3c

Implementation Notes

  • Presence, not trust. Verifying who signed an image needs per-publisher trust roots (keys / Sigstore identities) which differ per upstream and would need hand-curation. v1 reports presence only and flags this in the report header, the script --help, and the doc.
  • Source of truth stays the BOM inventory. The script parses the per-component image list from docs/user/container-images.md, so weekly bom-refresh keeps it current. No new data file.
  • Three checks, two cosign calls per image. cosign tree returns sig + SBOM presence in one round-trip; cosign download attestation | jq is needed to filter the predicateType for SLSA-style provenance specifically (attestation presence alone is broader).
  • Unreachable ≠ absent. Regional ECR (no AWS creds), private mirrors, and transient network errors are reported as ? with a reason in the Notes column rather than conflated with -.

Testing

shellcheck -x tools/s3c        # clean
make lint                      # 0 issues (golangci-lint, yamllint, license headers, AGENTS.md sync, sidebar, BOM pinning)
make test                      # passes; coverage 76.4% (>= 75% threshold)

Smoke runs:

Component Result Verified path
aws-efa (regional ECR) ? ? ? auth required unreachable detection
kueue (registry.k8s.io) Y - - signed-only path
kgateway-crds (no images) "0 images, nothing to verify" exit 0 empty-component path
bogus-component exit 1 with clear error unknown-component path

Risk Assessment

  • Low — Standalone tool plus doc additions; no changes to runtime, CLI, or APIs. Easily reverted.

Rollout notes: N/A — no migration; users opt in by running tools/s3c <component>.

Checklist

  • Tests pass locally (make test with -race)
  • Linter passes (make lint)
  • I did not skip/disable tests to make CI green
  • I added/updated tests for new functionality (N/A — bash script smoke-tested across all four codepaths; no Go test surface)
  • I updated docs if user-facing behavior changed
  • Changes follow existing patterns in the codebase (script style mirrors tools/check-tools / tools/describe; sources tools/common)
  • Commits are cryptographically signed (git commit -S)

Adds tools/s3c, an on-demand cosign wrapper that probes every container
image of a given component for cosign signatures, SBOMs, and SLSA-style
build provenance. The script reads the per-component image list from
docs/user/container-images.md (the existing BOM inventory) so the
source of truth stays unchanged.

Documents the three underlying cosign commands in a new "Verifying
supply-chain provenance" section of the inventory page so users can run
the same checks by hand.

Scope is presence, not trust: verifying *who* signed an image requires
per-publisher trust roots (keys or OIDC identities) which differ per
upstream and would need hand-curation. Called out explicitly in the
doc and the script's report header.

Unreachable registries (regional ECR, private mirrors) are reported
with `?` plus a reason in the Notes column rather than conflated with
`-` (absent).

Closes: #745
@mchmarny

mchmarny commented May 7, 2026

Copy link
Copy Markdown
Member Author

Example:

$ tools/s3c gpu-operator

Component: gpu-operator (14 images)

Presence-only check: does NOT verify publisher trust/identity.
Y = artifact attached, - = artifact absent, ? = could not probe.

  Image                                                            Sig  SBOM  Prov  Notes
  ---------------------------------------------------------------  ---  ----  ----  -----
  nvcr.io/nvidia/cloud-native/dcgm:4.5.2-1-ubuntu22.04             Y    -     -
  nvcr.io/nvidia/cloud-native/gdrdrv:v2.5.2                        ?    ?     ?     image not found
  nvcr.io/nvidia/cloud-native/k8s-cc-manager:v0.4.0                Y    -     -
  nvcr.io/nvidia/cloud-native/k8s-driver-manager:v0.10.0           Y    -     -
  nvcr.io/nvidia/cloud-native/k8s-mig-manager:v0.14.0              Y    -     -
  nvcr.io/nvidia/cloud-native/nvidia-fs:2.27.3                     ?    ?     ?     image not found
  nvcr.io/nvidia/cloud-native/nvidia-sandbox-device-plugin:v0.0.3  Y    -     -
  nvcr.io/nvidia/cloud-native/vgpu-device-manager:v0.4.2           Y    -     -
  nvcr.io/nvidia/driver:580.105.08                                 ?    ?     ?     image not found
  nvcr.io/nvidia/gpu-operator:v26.3.1                              Y    -     -
  nvcr.io/nvidia/k8s-device-plugin:v0.19.0                         Y    -     -
  nvcr.io/nvidia/k8s/container-toolkit:v1.19.0                     Y    -     -
  nvcr.io/nvidia/k8s/dcgm-exporter:4.5.1-4.8.0-distroless          Y    -     -
  nvcr.io/nvidia/kubevirt-gpu-device-plugin:v1.5.0                 Y    -     -

Summary: 11/14 signed · 0/14 SBOM · 0/14 provenance · 3 unreachable

@github-actions

github-actions Bot commented May 7, 2026

Copy link
Copy Markdown
Contributor

@mchmarny mchmarny self-assigned this May 7, 2026
@coderabbitai

This comment was marked as resolved.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/user/container-images.md`:
- Around line 278-279: The markdown link target "../../tools/s3c" is a dead link
causing the VitePress build to fail; update the two occurrences (the link around
`tools/s3c` at the snippet and the occurrence around line 360) to a docs-valid
target by either replacing the relative link with a stable GitHub blob URL
pointing to the tools/s3c path in the repo or convert the anchor to plain inline
code formatting (`tools/s3c`) so it is not a navigable link; ensure both
occurrences are changed consistently.
- Line 287: Change the fenced code block that begins with "Component:
nvidia-dra-driver-gpu (1 images)" to include a language tag by replacing the
opening ``` with ```text (or ```console) so the example output satisfies
markdownlint MD040; ensure only the opening fence is modified and the closing
``` remains unchanged.

In `@tools/s3c`:
- Around line 163-175: The attestation probe currently treats any non-OK result
from `cosign download attestation` as absent; update the logic around
`att=$(cosign download attestation "$image" 2>/dev/null) || att_rc=$?` to mirror
the `cosign tree` handling: if `att_rc` is non-zero (probe failure), set
`prov="?"` (unreachable) and add the same descriptive note used for tree probes
rather than leaving `prov` as "-" (absent); keep the existing path that inspects
`$att` and sets `prov="Y"` only when the attestation is fetched and contains
slsa/provenance. Ensure you reference and update the `att`, `att_rc`, and `prov`
variables and the `cosign download attestation` probe branch.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 178dab88-14e1-49bc-9e22-fc7b7c680377

📥 Commits

Reviewing files that changed from the base of the PR and between c711a31 and 95067cd.

📒 Files selected for processing (2)
  • docs/user/container-images.md
  • tools/s3c

Comment thread docs/user/container-images.md Outdated
Comment thread docs/user/container-images.md Outdated
Comment thread tools/s3c Outdated
Relative links like ../../tools/s3c resolve outside the VitePress site
tree and fail the dead-link check during the Fern docs build. The
existing convention in docs/README.md (links to CONTRIBUTING.md,
SECURITY.md) uses absolute github.com/NVIDIA/aicr/blob/main/... URLs;
match it.
lockwobr
lockwobr previously approved these changes May 7, 2026

@lockwobr lockwobr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

github-actions Bot commented May 7, 2026

Copy link
Copy Markdown
Contributor

Coverage Report ✅

Metric Value
Coverage 76.4%
Threshold 75%
Status Pass
Coverage Badge
![Coverage](https://img.shields.io/badge/coverage-76.4%25-green)

No Go source files changed in this PR.

coderabbitai[bot]

This comment was marked as resolved.

Address review feedback on #791:

- docs/user/container-images.md: add `text` language tag to the example
  output fenced block so markdownlint MD040 passes.
- tools/s3c: capture cosign download attestation stderr separately and
  classify its non-zero exit. Network/auth errors flip prov to "?" with
  a "prov probe unreachable" note; unrecognized stderr (typically
  "no attestations" — the common case for unsigned-but-reachable
  images) stays "-" so we don't mis-mark every unattested image as
  unreachable.
@mchmarny mchmarny enabled auto-merge (squash) May 7, 2026 16:23
coderabbitai[bot]

This comment was marked as resolved.

@mchmarny mchmarny merged commit 2de4742 into main May 7, 2026
36 checks passed
@mchmarny mchmarny deleted the feat/s3c-supply-chain-check branch May 7, 2026 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Supply-chain provenance audit per component

2 participants