[Backport release-1.7] feat(release): sign the release assets and attach a source SBOM by themoriarti · Pull Request #5172 · cozystack/cozystack

Marian Koreniuk (themoriarti) · 2026-10-31T02:53:55Z

What this PR does

Backport of #4861 to release-1.7.

OpenSSF Scorecard reports Signed-Releases 0/10: none of the last five releases (v1.7.0, v1.7.0-rc.3, v1.7.0-rc.2, v1.6.5, v1.6.5-rc.3) has a signature or provenance asset. #4861 adds a signature, but release-1.7 forked from main at 38d6ac7ef on 2026-10-12, #4861 merged on 2026-10-20, and every v1.7 tag sits on that branch, so no tag carries it yet. With this backport the next v1.7.x ships cozystack-source-sbom.spdx.json, cozystack-checksums.txt and cozystack-checksums.txt.sigstore.json. Scorecard takes a .sigstore.json asset as a signature. It averages over the last five releases and a signature alone counts 8 out of 10, so the score rises one release at a time and stops at 8 without provenance.

This is a feat on a patch line, and #4861 left open whether it should go there, so that decision is the maintainers'. It changes no shipped code: only release tooling, its docs and its tests.

bc24b588b, the only commit of #4861, applies cleanly. The eight files match main byte for byte, except for one line in docs/release.md that this PR does not touch (a Talos path in the cherry-pick triage snippet, changed on main only). The -m1 pick of the merge commit gives the same tree.

Which tree each release job runs

Job	Workflow file from	`hack/upload-assets.sh` from
`tags.yaml::prepare-release` (rc)	the tag	the tag
`promote-rc.yaml::promote` (stable)	the dispatch ref, `main`	the rc staging branch `release-X.Y.Z-rc.N`, so the rc tag's tree

promote-rc.yaml has two checkouts. make upload_assets runs in the primary one, the rc staging branch. The .release-tooling checkout of the dispatch ref serves promote-rewrite-tags.sh, changelog-preserve.sh, promote-packages-artifact.sh and the changelog validator, and is not where the upload runs from.

The v1.7.0 promotion shows this on a real release. Run 40853526862 was dispatched from main, which has #4861, and Set up syft and Set up cosign were green, yet v1.7.0 has no signature assets, because v1.7.0-rc.3's tree has no hack/sign-release-assets.sh.

So a signed v1.7.1 needs an rc cut after this merges. An rc tagged earlier stays unsigned, and so does the stable promoted from it, even though main's promote-rc.yaml is ready. No v1.7.1 rc exists today, so merging before v1.7.1-rc.1 is cut needs nothing extra.

Certificate identity on this line

The identity regexp in hack/sign-release-assets.sh needs no change. The certificate names the ref that triggered the run, not the branch a tag was cut from, so releases from this line sign as tags.yaml@refs/tags/v1.7.1-rc.N and promote-rc.yaml@refs/heads/main. Run through Go's regexp, the pattern as it stands in the script accepts both and rejects promote-rc.yaml@refs/heads/release-1.7, tags.yaml@refs/heads/release-1.7 and promote-rc.yaml@refs/heads/main-evil.

One thing to watch: v1.6.5 was first dispatched from release-1.6 (run 38882938797) and then again from main (run 38884889709). The same slip on a signed rc now fails in Create draft stable release and upload assets, before the first upload. Dispatching again from main recovers it, since the draft and the staging branch are overwritable. docs/release.md says main only.

promote-rc.yaml on this branch runs only on such a wrong-ref dispatch. It is backported anyway so the file stays equal to main, and so the hack/ci-tool-installs-contract.bats updated here holds.

Not exercised yet

No tag on any line contains #4861, so signing has not run end to end on a real release. Set up syft and Set up cosign have run green on oracle-vm-4cpu-16gb-x86-64 in the v1.7.0 promotion above. They have not run on oracle-vm-24cpu-96gb-x86-64, which tags.yaml::prepare-release uses, and neither runner has been shown to reach Fulcio and Rekor from a job.

A signing failure fails the job before the first upload, by design, so the first rc carrying this goes red rather than shipping unsigned, and that rc would be a patch release. A v1.8.0-alpha from main would exercise the same path first, if you would rather see it there.

Nothing writes to a release's assets after hack/upload-assets.sh, so the checksums stay correct through finalize. Promotion downloads the rc's three files along with its other assets, and they are overwritten: hack/sign-release-assets.sh refuses them as inputs, and cosign v3.1.3 writes the bundle with a plain os.WriteFile.

Testing

hack/sign-release-assets.bats and hack/upload-assets.bats: 12 of 12.
The contract and meta suites that read these files: 385 of 386 across 19 files. The one failure, a path owned by several sources counts as covered if any one is in hack/select-e2e_test.bats, is not from this PR: it fails the same way on plain release-1.7 and on main in my environment (yq 4.47.2 there, the repo pins 4.53.3), and this diff does not touch select-e2e.
shellcheck on both scripts: clean.
zizmor --offline .github/workflows/ (1.26.1, as in the gate): no findings.
hack/check-commit-trailers.sh upstream/release-1.7..HEAD: OK.

Screenshots

Not a UI change.

Downstream repositories

I walked the trigger map against this diff, which is the same change as #4861. Three assets are added and none is renamed, so the website install pages are not affected. ccp depends on release-prep behaviour in tags.yaml and on what make targets do, but the steps a maintainer runs do not change: the signing happens inside make upload_assets.

Release note

feat(release): releases carry cozystack-source-sbom.spdx.json, an SPDX SBOM of the source tree and the cozypkg binaries (not of the container or Talos images), cozystack-checksums.txt with the SHA-256 of each asset, and cozystack-checksums.txt.sigstore.json, a keyless Sigstore signature over the checksums made by the release workflow; docs/release.md shows how to verify a download with cosign v3.1.3 or later.

coderabbitai · 2026-10-31T02:54:03Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

Source mode: Auto
Repository: cozystack/cozystack
Branch: release-1.7
File: .coderabbit.yaml
Commit: ec30722
Layers: Repository YAML
Review profile: CHILL
Plan: Advanced
Run ID: c249f8a3-e5ba-46b8-afa6-34dc79a78195

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

Trigger review

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Marian Koreniuk (themoriarti) · 2026-11-01T19:21:57Z

Ready for review. One note on the red Unit & controller tests check (job): both failures are in release-1.7 itself, not in this backport, which touches only the two release workflows, hack/upload-assets.sh, hack/sign-release-assets.sh, their bats files and docs/release.md.

helm unittest monitoring/tests/vtraces_collector_test.yaml expects opentelemetry-collector-contrib:.*@sha256:, but the branch ships ghcr.io/cozystack/cozystack/otelcol-tracing:v0.136.0-cozystack.0.
bats hack/cni-plugins-staging-contract.bats:364 keeps the kube-ovn marker at v1.15.26 while packages/system/kubeovn/values.yaml vendors v1.7.0.

Both need their own fix on release-1.7; I can send one if nobody has it in flight.

Why this matters for timing: promote-rc.yaml takes syft and cosign from the dispatch ref, but runs hack/upload-assets.sh from the rc tree, so v1.7.0-rc.2, rc.3 and v1.7.0 all went out with cozypkg-checksums.txt only. The first rc tagged after this merges is the first signed one.

Aleksei Sviridkin (lexfrei)

Marian Koreniuk (@themoriarti) LGTM. This is the same change as #4861, and everything it calls already exists on release-1.7.

What I checked at f196e4b73:

The patch of this commit and of bc24b588b differ only in blob indexes and one hunk offset in docs/release.md. Against origin/main, the eight files differ only in the Talos-path line the description names. The original message, sign-off and Assisted-by: LLM are kept, plus the cherry-pick line.
Before the pick, tags.yaml, promote-rc.yaml, hack/upload-assets.sh and hack/ci-tool-installs-contract.bats on release-1.7 are identical to bc24b588b^. So the pick lands on the same context as on main. The Makefile and every hack/ script the two workflows call also match main. The only differences in that set are changelog-generate.yaml and hack/validate-changelog.sh, and signing does not use them.
The three bats files pass locally, 20 of 20, and shellcheck is clean on both scripts.
The identity regexp accepts tags.yaml@refs/tags/v1.7.1-rc.1 and promote-rc.yaml@refs/heads/main. It rejects both workflows at refs/heads/release-1.7 and promote-rc.yaml@refs/heads/main-evil.
No tag contains bc24b588b yet. v1.7.0 has no signature assets even though its promote run from main had green syft and cosign steps, so the note about cutting an rc after merge holds.

The red checks come from the branch, not from this diff. Both Unit & controller tests failures reproduce on plain release-1.7 at a32334aea, the v1.7.0 promote merge: the monitoring collector-image unittest and bats 567 in hack/cni-plugins-staging-contract.bats. E2E Tests is red only because that workflow failed before e2e started. Those two need a separate fix on the branch.

Whether a feat goes into a patch line is for the maintainers to decide. For v1.7.x users the change is three more assets per release, and nothing changes in what they install. The release process changes more. The next v1.7.x rc would be the first release on any line that signs, and if the tags.yaml runner cannot reach Fulcio or Rekor, that rc fails before upload instead of shipping unsigned.

…#5179) ## What this PR does Every PR against `release-1.7` has had a red `Unit & controller tests` check since v1.7.0-rc.3 (seen on #5172, [job](https://github.com/cozystack/cozystack/actions/runs/41162009892/job/124421449910)). The cause is the release build itself: the `Prepare release` commit (0402103) rewrites `packages/*/values.yaml`, and two unit tests read exactly the fields it rewrites. | test | reads | before the release build | after | |---|---|---|---| | `hack/cni-plugins-staging-contract.bats` "the kube-ovn release the CNI plugin pin was checked against is the one vendored" | `.global.images.kubeovn.tag` in `packages/system/kubeovn/values.yaml` | `v1.15.26@sha256:…` | `v1.7.0@sha256:…` | | `packages/system/monitoring/tests/vtraces_collector_test.yaml` "pins the collector image through the cozy-lib.image helper" | `tracingCollector.image` | `docker.io/otel/opentelemetry-collector-contrib:0.136.0@sha256:…` | `ghcr.io/cozystack/cozystack/otelcol-tracing:v0.136.0-cozystack.0@sha256:…` | Changes: - The bats test now takes the vendored kube-ovn release from `appVersion` in `packages/system/kubeovn/charts/kube-ovn/Chart.yaml`. That is the value `packages/system/kubeovn/Makefile` builds the image from, and a release build leaves it alone. The hint printed on a mismatch names the new file. - The helm unittest case no longer names the image at all. It sets `_cluster.images-registry` and asserts that the registry lands in front of the rendered image and that the image stays digest-pinned (the `grafana_upstream_image_plugins_test.yaml` pattern). Removing the `cozy-lib.image` call from the template makes it fail; the old regex did not. Verified locally on this branch: `bats hack/cni-plugins-staging-contract.bats` 10/10, `helm unittest packages/system/monitoring -f tests/vtraces_collector_test.yaml` 28/28. Not a backport: `main` still carries the pre-release pins, so both tests pass there today. The same patch is open against `main` as #5180, so `release-1.8` does not meet the same surprise. Fixes #5178. Refs #5172 (blocked on this check). ### Downstream repositories - [x] No downstream repository is affected by this change ### Release note ```release-note NONE ```

…#5180) ## What this PR does Same change as #5179, on `main`. Both tests pass here today, because `main` still carries the pre-release image pins; they fail on every release line as soon as the first rc is cut, since the `Prepare release` commit rewrites `packages/*/values.yaml` and two unit tests read exactly the fields it rewrites. On `release-1.7` this has kept `Unit & controller tests` red on every PR since v1.7.0-rc.3 ([job](https://github.com/cozystack/cozystack/actions/runs/41162009892/job/124421449910) on #5172), and `release-1.8` would repeat it. | test | reads | before the release build | after (release-1.7 today) | |---|---|---|---| | `hack/cni-plugins-staging-contract.bats` "the kube-ovn release the CNI plugin pin was checked against is the one vendored" | `.global.images.kubeovn.tag` in `packages/system/kubeovn/values.yaml` | `v1.15.26@sha256:…` | `v1.7.0@sha256:…` | | `packages/system/monitoring/tests/vtraces_collector_test.yaml` "pins the collector image through the cozy-lib.image helper" | `tracingCollector.image` | `docker.io/otel/opentelemetry-collector-contrib:0.136.0@sha256:…` | `ghcr.io/cozystack/cozystack/otelcol-tracing:v0.136.0-cozystack.0@sha256:…` | Changes: - The bats test takes the vendored kube-ovn release from `appVersion` in `packages/system/kubeovn/charts/kube-ovn/Chart.yaml`. That is the value `packages/system/kubeovn/Makefile` builds the image from, and a release build leaves it alone. The hint printed on a mismatch names the new file. - The helm unittest case no longer names the image at all. It sets `_cluster.images-registry` and asserts that the registry lands in front of the rendered image and that the image stays digest-pinned (the `grafana_upstream_image_plugins_test.yaml` pattern). Removing the `cozy-lib.image` call from the template makes it fail; the old regex did not. Same single commit as #5179; the only difference is the unrelated `DOC=` line next to the edit, kept as `main` has it. Verified locally on this branch: `bats hack/cni-plugins-staging-contract.bats` 10/10, `helm unittest packages/system/monitoring -f tests/vtraces_collector_test.yaml` 28/28. Refs #5178 (the release-1.7 report), #5179, #5172. ### Downstream repositories - [x] No downstream repository is affected by this change ### Release note ```release-note NONE ```   ## Summary by CodeRabbit * **Tests** * Version validation now compares the checked version with the chart’s application version and reports missing or mismatched versions with the chart path. * Monitoring image checks now verify use of the configured mirror registry and require a 64-character lowercase SHA-256 digest, without requiring a specific image name.

No release carried a signature or an SBOM, so users could verify none of the ISO, disk images, kernel, initramfs, manifests and cozypkg binaries they download, and the OpenSSF Scorecard Signed-Releases check and CLOMonitor's signed_releases and sbom checks failed. hack/upload-assets.sh now hands its asset list to the new hack/sign-release-assets.sh before the first upload. That script writes cozystack-source-sbom.spdx.json with syft over the committed tree and the cozypkg binaries, cozystack-checksums.txt with the SHA-256 of every asset and the SBOM, and cozystack-checksums.txt.sigstore.json with a keyless cosign signature over the checksums. The SBOM does not describe the container images or the Talos boot images, which is what its name says; image SBOMs are cozystack#2937. The script verifies the bundle against the identities of tags.yaml on a v* tag and promote-rc.yaml on main before returning, so a bad signature fails the job with nothing uploaded, and hack/upload-assets.bats pins the upload list and that order. The two jobs that call make upload_assets get id-token: write, cosign v3.1.3 from the SHA-pinned cosign-installer, and syft from its release tarball checked against a committed SHA-256, because anchore/sbom-action/download-syft runs an install script fetched at run time; hack/ci-tool-installs-contract.bats now covers both. docs/release.md describes the files, what the SBOM covers, the minimum cosign version to verify a download with, what the certificate identity proves, and that promote-rc.yaml is dispatched from main only. Refs cozystack#4759 Assisted-by: LLM Signed-off-by: Marian Koreniuk <[email protected]> (cherry picked from commit bc24b58)

github-actions Bot added area/release Issues or PRs related to release tooling (changelog, backport, release pipeline) kind/feature Categorizes issue or PR as related to a new feature size/XL This PR changes 500-999 lines, ignoring generated files labels Oct 31, 2026

Marian Koreniuk (themoriarti) marked this pull request as ready for review November 1, 2026 19:19

Marian Koreniuk (themoriarti) requested review from Aleksei Sviridkin (lexfrei) and myasnikovdaniil November 1, 2026 19:19

Marian Koreniuk (themoriarti) requested review from IvanHunters, Andrei Kvapil (kvaps), Timofei Larkin (lllamnyp) and Timur Tukaev (tym83) as code owners November 1, 2026 19:19

Aleksei Sviridkin (lexfrei) approved these changes Nov 1, 2026

View reviewed changes

This was referenced Nov 1, 2026

fix(tests): keep two unit tests green after the release image restamp #5179

Merged

fix(tests): keep two unit tests green after the release image restamp #5180

Merged

[Tracking] Remaining CI / build / release / E2E automation work #3278

Open

Marian Koreniuk (themoriarti) force-pushed the backport/release-1.7-release-signing branch from f196e4b to bdcf8e1 Compare November 9, 2026 09:27

Marian Koreniuk (themoriarti) requested a review from Aleksei Sviridkin (lexfrei) November 9, 2026 09:27

Navigation Menu

Repository navigation

[Backport release-1.7] feat(release): sign the release assets and attach a source SBOM - #5172

[Backport release-1.7] feat(release): sign the release assets and attach a source SBOM#5172
Marian Koreniuk (themoriarti) wants to merge 1 commit into
cozystack:release-1.7from
themoriarti:backport/release-1.7-release-signing

Marian Koreniuk (themoriarti) commented Oct 31, 2026

Uh oh!

coderabbitai Bot commented Oct 31, 2026 •

edited

Loading

Review skipped

Uh oh!

Marian Koreniuk (themoriarti) commented Nov 1, 2026

Uh oh!

Aleksei Sviridkin (lexfrei) left a comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Marian Koreniuk (themoriarti) commented Oct 31, 2026

What this PR does

Which tree each release job runs

Certificate identity on this line

Not exercised yet

Testing

Screenshots

Downstream repositories

Release note

Uh oh!

coderabbitai Bot commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

Marian Koreniuk (themoriarti) commented Nov 1, 2026

Uh oh!

Aleksei Sviridkin (lexfrei) left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

coderabbitai Bot commented Oct 31, 2026 •

edited

Loading