Summary
The UAT pipeline publishes signed evidence bundles to an OCI ref that bakes the CI run_id into the repository name and duplicates recipe-coordinate dimensions across both the repo and the tag. This fragments evidence across thousands of GHCR packages and makes the published evidence hard to discover, list, and consume. Switch to the stable, digest-addressed layout the codebase already uses elsewhere.
Current behavior
A run produces, e.g.:
ghcr.io/nvidia/aicr-uat-eks-h100-training-28148560505:h100-eks-ubuntu-training-kubeflow-32ee00e3b6e9
└────────────── repository ──────────────────────────┘ └──────────────── tag ───────────────────────┘
Built from:
tests/uat/aws/tests/h100-training-config.yaml (and the gcp sibling): push: ghcr.io/nvidia/aicr-uat-eks-h100-training — the stable repo prefix.
tests/uat/aws/run (inject_push_target, ~L75-84): appends -${RUN_ID} to the repo.
aicr evidence then derives the tag :<recipe-slug>-<fingerprint>.
Problems
run_id in the repo name → a new GHCR package per CI run. oras repo tags … is useless, per-package visibility/RBAC must be set per run, and there is no stable address for "all evidence for this recipe."
- Coordinate dims duplicated —
eks, h100, training appear in both the repo path and the tag (the tag already carries the full coordinate slug).
- Digest prefix in the tag (
-32ee00e3b6e9) duplicates what OCI provides natively via @sha256:; a 12-hex truncation is non-verifiable and collision-prone as a tag.
- Dims split inconsistently —
ubuntu/kubeflow live only in the tag; no single field is the canonical coordinate.
Proposed behavior
Repo = stable recipe identity; tag = run pointer; consumers pin by digest. This matches the existing conventions: the aicr evidence push default (ghcr.io/<owner>/aicr-evidence:<recipe-slug>-<fingerprint>) and the corroborate evidenceRef testdata (ghcr.io/nvidia/aicr-evidence/<recipe>@sha256:<digest>).
Preferred — repo-per-recipe:
ghcr.io/nvidia/aicr-evidence/h100-eks-ubuntu-training-kubeflow:run-28148560505
- One package per recipe; all runs accumulate.
oras repo tags … lists run history; visibility set once.
- Tag = run identity (
run-<gha-run-id> or a date); CI provenance preserved, just out of the repo name.
- Consume immutably via
@sha256:….
Alternative — flat single repo (smallest GHCR footprint, mirrors the aicr tool default):
ghcr.io/nvidia/aicr-evidence:h100-eks-ubuntu-training-kubeflow-28148560505
Either eliminates all four redundancies. Maps 1:1 onto the corroborate GCS tree (results/<service>/<accel>-<os>/<intent>[-platform]/<signer>/<run>/).
The run_id is not lost by removing it from the repo name — the bundle predicate already carries rekorLogIndex + attestedAt, and corroborate meta.runId carries the run.
Scope / affected files
tests/uat/aws/tests/h100-training-config.yaml, tests/uat/gcp/tests/h100-training-config.yaml — push: prefix.
tests/uat/aws/run (and gcp sibling) — inject_push_target appends -${RUN_ID} to the repo; move run identity to the tag.
.github/workflows/uat-aws.yaml, .github/workflows/uat-gcp.yaml — the **Evidence (OCI):** summary echo (and surrounding comments) reference the old ref.
- Docs referencing the UAT evidence ref, if any.
Out of scope
The aicr evidence push tool default already produces a clean ref; this issue is specifically about the UAT pipeline push target.
Summary
The UAT pipeline publishes signed evidence bundles to an OCI ref that bakes the CI
run_idinto the repository name and duplicates recipe-coordinate dimensions across both the repo and the tag. This fragments evidence across thousands of GHCR packages and makes the published evidence hard to discover, list, and consume. Switch to the stable, digest-addressed layout the codebase already uses elsewhere.Current behavior
A run produces, e.g.:
Built from:
tests/uat/aws/tests/h100-training-config.yaml(and thegcpsibling):push: ghcr.io/nvidia/aicr-uat-eks-h100-training— the stable repo prefix.tests/uat/aws/run(inject_push_target, ~L75-84): appends-${RUN_ID}to the repo.aicr evidencethen derives the tag:<recipe-slug>-<fingerprint>.Problems
run_idin the repo name → a new GHCR package per CI run.oras repo tags …is useless, per-package visibility/RBAC must be set per run, and there is no stable address for "all evidence for this recipe."eks,h100,trainingappear in both the repo path and the tag (the tag already carries the full coordinate slug).-32ee00e3b6e9) duplicates what OCI provides natively via@sha256:; a 12-hex truncation is non-verifiable and collision-prone as a tag.ubuntu/kubeflowlive only in the tag; no single field is the canonical coordinate.Proposed behavior
Repo = stable recipe identity; tag = run pointer; consumers pin by digest. This matches the existing conventions: the
aicr evidence pushdefault (ghcr.io/<owner>/aicr-evidence:<recipe-slug>-<fingerprint>) and the corroborateevidenceReftestdata (ghcr.io/nvidia/aicr-evidence/<recipe>@sha256:<digest>).Preferred — repo-per-recipe:
oras repo tags …lists run history; visibility set once.run-<gha-run-id>or a date); CI provenance preserved, just out of the repo name.@sha256:….Alternative — flat single repo (smallest GHCR footprint, mirrors the
aicrtool default):Either eliminates all four redundancies. Maps 1:1 onto the corroborate GCS tree (
results/<service>/<accel>-<os>/<intent>[-platform]/<signer>/<run>/).The
run_idis not lost by removing it from the repo name — the bundle predicate already carriesrekorLogIndex+attestedAt, and corroboratemeta.runIdcarries the run.Scope / affected files
tests/uat/aws/tests/h100-training-config.yaml,tests/uat/gcp/tests/h100-training-config.yaml—push:prefix.tests/uat/aws/run(and gcp sibling) —inject_push_targetappends-${RUN_ID}to the repo; move run identity to the tag..github/workflows/uat-aws.yaml,.github/workflows/uat-gcp.yaml— the**Evidence (OCI):**summary echo (and surrounding comments) reference the old ref.Out of scope
The
aicr evidence pushtool default already produces a clean ref; this issue is specifically about the UAT pipeline push target.