feat(release): sign the release assets and attach a source SBOM by themoriarti · Pull Request #4861 · cozystack/cozystack

Marian Koreniuk (themoriarti) · 2026-10-11T04:45:29Z

What this PR does

Implements the release-asset part of #4759: a release cut from a tree that has this change gets an SBOM of its source and one keyless signature that covers all of its assets.

These are the two CLOMonitor checks still failing for this repository, sbom and signed_releases, after #4757 and #4899 brought the project score to 96.69.

hack/upload-assets.sh now collects its asset list into an array and hands it to the new hack/sign-release-assets.sh before the first upload. The script writes three files into _out/assets, and they are uploaded with the rest:

File	What it is
`cozystack-source-sbom.spdx.json`	SPDX 2.3 SBOM made with syft, over `git archive HEAD` (the committed release tree, without `_out/`, `.release-tooling` or runner leftovers) and the unpacked `cozypkg-.tar.gz` binaries. It does not* describe the container images that `packages/` pins by digest (their SBOMs are #2937, A2 in #3278) or the Talos ISO, disk, kernel and initramfs images. It is named by what it covers, so `cozystack-sbom` stays free for an SBOM of the release as a whole. CLOMonitor's `sbom` check matches `sbom` in an asset name.
`cozystack-checksums.txt`	SHA-256 of every uploaded asset and of the SBOM.
`cozystack-checksums.txt.sigstore.json`	Sigstore bundle of `cosign sign-blob` over the checksums, keyless, signed with the job's GitHub OIDC identity.

Before it returns, the script runs the same cosign verify-blob that docs/release.md gives users, with --certificate-identity-regexp '^https://github\.com/cozystack/cozystack/\.github/workflows/(tags\.yaml@refs/tags/v|promote-rc\.yaml@refs/heads/main$)': tags.yaml on a v* tag push, promote-rc.yaml dispatched from main. A signature that check would reject therefore fails the job before anything is uploaded. It also refuses its own three outputs as inputs: promote-rc.yaml downloads every rc asset, and a stable release has to get freshly computed checksums and a fresh signature, since promotion restamps several assets.

Workflows: tags.yaml::prepare-release and promote-rc.yaml::promote, the two jobs that call make upload_assets, get id-token: write and two setup steps:

syft v1.54.1 from its release tarball, checked against a committed SHA-256 (c069905b…, the release's checksums entry; the checksums file verifies against anchore/syft's keyless release signature). Not anchore/sbom-action/download-syft: pinned by commit, it still runs syft's install.sh fetched from a tag at run time, in a job that holds the app token, the GHCR login and id-token: write. v1.54.1 is past GHSA-mw2c-m758-9v5q (fixed in v1.52.0). hack/ci-tool-installs-contract.bats now fails on anchore/sbom-action, on a syft install script, on a syft download without sha256sum -c and on version or checksum drift between the two workflows.
sigstore/cosign-installer v4.1.2, pinned by SHA, with cosign-release: v3.1.3: the version docs/release.md asks users for, so the job checks its own signature the way they will. The action verifies that binary against cosign's keyless and KMS bundles with its digest-pinned bootstrap cosign. hack/ci-tool-installs-contract.bats holds the action commit and the release equal in both workflows and fails on a step without cosign-release.

docs/release.md gets a "Release signatures and SBOM" section with the user-side verification: cosign v3.1.3 or later (v2.6.5 on the v2 line), the two commands joined with &&, what the SBOM covers and what the certificate identity proves. It now also says, at each place promote-rc.yaml is run, that it is dispatched from main only.

Decisions for the release owners

Fail closed. If Fulcio or Rekor is down, the release job fails and nothing is uploaded. I chose that so that no release goes out unsigned. If you would rather publish unsigned than block, the change is a || echo ::warning:: around the call in upload-assets.sh.
Promotion runs the rc tree's script. promote-rc.yaml calls make upload_assets from the rc checkout, not from .release-tooling. The first signed stable release is therefore the promotion of the first rc tagged after this merges. release-1.6 gets signatures only if this is backported. CLOMonitor reads the latest release, so the score moves only when that release carries the files.
Certificate identity pins the refs (decided in review): tags.yaml@refs/tags/v… and promote-rc.yaml@refs/heads/main. A promote-rc.yaml dispatched from another branch fails the job's own check at the upload step, after the staging branch, the promotion-* candidate and the empty draft release exist; nothing in parse refuses a non-main dispatch up front. Until the v* tag rulesets exist, a hand-pushed v* tag whose tree has an edited tags.yaml still signs, which docs/release.md says.
Follow-ups after the first signed release (Sign release assets and attach an SBOM to each release #4759 stays open for them):
- project.documentation.signature-verification in security-insights.yml (docs(security): add an OpenSSF Security Insights manifest #4757);
- the SBOM row in docs/security/self-assessment.md, which says "Not published at this time"; the image caveat above stays in it.
I left both alone so that they do not describe releases that don't exist yet.

How it was checked

Bats 1.14.0 (the version CI installs), with gh, syft and cosign stubbed:
- hack/sign-release-assets.bats, 8 tests: what the checksums cover; that syft sees the committed tree and the cozypkg binaries but not untracked files; that the verify identity accepts tags.yaml on a v* tag and promote-rc.yaml on main, and rejects a fork, another workflow, tags.yaml.evil, tags.yaml on a branch, and promote-rc.yaml on release-1.6, on main-evil and on a tag; that docs/release.md shows the same regexp as the script; that a failed verification fails the script; that it refuses a missing asset, assets split over two directories (with a same-named file in both, so the test fails if the directory check is removed) and its own outputs as input.
- hack/upload-assets.bats, 4 tests, new: the exact upload list and order (the assets, then the SBOM, the checksums and the bundle); that the checksums name every upload but the last two; zero gh calls when verification fails or an asset is missing. The old upload-assets.sh fails all four, a version that signs after uploading fails the last two.
- hack/ci-tool-installs-contract.bats, 8 tests. Putting download-syft back, dropping the sha256sum -c line, letting SYFT_VERSION drift between the workflows, dropping cosign-release from one step, letting it drift, unpinning the installer or giving one step another installer commit each fails it.
- Putting the old any-ref regexp back in the script fails the identity test and the docs tie; putting it back in the docs alone fails the tie.
- The meta-suites bats-strict-setup, bats-no-negated-assert, bats-no-exit-trap, bats-runner-coverage and bats-runner-contract pass: 90 tests across these eight files.
The pinned regexp gives the same accept/reject results under grep -E and Go's regexp (what cosign uses). cosign-installer's install script at the pinned commit, run as written with cosign-release: v3.1.3, verifies the binary's keyless and KMS bundles and installs v3.1.3 (its SHA-256 matches cosign_checksums.txt). --yes, --bundle, --certificate-identity-regexp and --certificate-oidc-issuer are listed, not deprecated, in cosign v3.1.3's help.
The Set up syft step's script, run as written with RUNNER_TEMP/GITHUB_PATH set: the checksum passes and syft version reports 1.54.1. A real syft v1.54.1 run through hack/sign-release-assets.sh (cosign stubbed) over this tree and the v1.6.5 cozypkg-linux-amd64.tar.gz took 10 s and wrote an SPDX 2.3 document with 920 packages: 440 golang, 373 npm, 103 github, 4 without a purl; 60 come from the cozypkg binary.
cosign behaviour behind the docs, with a key-signed legacy bundle whose cert holds the bare public key, against the documented identity regexp (tlog ignored, since an attacker can upload their own entry): v2.6.4 and v3.1.2 print "Verified OK", v2.6.5 and v3.1.3 reject it. --new-bundle-format makes v2.6.4 reject it but not v3.1.2, so the docs give the minimum version and not the flag. A real keyless protobuf bundle (syft's) verifies on v2.6.5 and v3.1.3.
zizmor 1.26.1 --offline reports no findings. actionlint 1.7.12 (with .github/actionlint.yaml) reports the same 13 pre-existing info-level shellcheck notes in tags.yaml/promote-rc.yaml as before, nothing new. shellcheck passes on both scripts. hack/check-commit-trailers.sh passes.
Not checked: a real keyless signature. It needs the OIDC token of a job in this repository, so the first rc after the merge is the end-to-end test. Cutting an -alpha.N first would exercise it without touching an rc.

Screenshots

Downstream repositories

I checked the trigger map file by file. The only entry hack/upload-assets.sh can trigger is renaming a release asset, because the website install pages hardcode asset names. This PR adds three assets and renames none of the released ones, cozypkg-checksums.txt included (the SBOM was renamed to cozystack-source-sbom.spdx.json in review, before any release carried it). The workflow and docs changes match no entry.

Release note

feat(release): releases carry cozystack-source-sbom.spdx.json, an SPDX SBOM of the source tree and the cozypkg binaries (not of the container or Talos images), cozystack-checksums.txt with the SHA-256 of each asset, and cozystack-checksums.txt.sigstore.json, a keyless Sigstore signature over the checksums made by the release workflow; docs/release.md shows how to verify a download with cosign v3.1.3 or later.

Refs #4759

Summary by CodeRabbit

New Features
- Releases now include an SPDX software bill of materials, SHA-256 checksums, and a keyless Sigstore signature bundle.
- Release assets are verified against the expected workflow identity before upload; uploads stop if verification fails.
Documentation
- Added guidance on release signatures, SBOM coverage, asset verification, and which branch dispatches release workflows.
Tests
- Added coverage for generated release files, upload behavior, and signing and verification failures.

coderabbitai · 2026-10-11T04:45:38Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: cozystack/cozystack/.coderabbit.yaml
Review profile: CHILL
Plan: Advanced
Run ID: 1f9c658f-a965-4250-8501-ed92b852a9bc

📥 Commits

Reviewing files that changed from the base of the PR and between 1571346 and bc24b58.

📒 Files selected for processing: 6

.github/workflows/promote-rc.yaml
.github/workflows/tags.yaml
docs/release.md
hack/ci-tool-installs-contract.bats
hack/sign-release-assets.bats
hack/sign-release-assets.sh

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

📝 Walkthrough

Release workflows now install Syft and Cosign. Release scripts generate an SPDX SBOM and SHA-256 checksums, sign and verify the checksum file, then upload release assets with the generated metadata. Documentation and Bats tests cover workflow setup, signing, verification, and upload behavior.

Changes: Release asset signing

Layer / File(s)	Summary
Generate and verify release metadata `hack/sign-release-assets.sh`, `hack/sign-release-assets.bats`	The script validates asset inputs, creates an SPDX SBOM and SHA-256 checksums, then signs and verifies the checksum file. Tests cover generated files, SBOM inputs, workflow identities, verification failure, and rejected inputs.
Prepare release workflows for signing `.github/workflows/promote-rc.yaml`, `.github/workflows/tags.yaml`, `hack/ci-tool-installs-contract.bats`	Both release jobs grant `id-token: write` and install Syft and Cosign. Contract tests check Syft download verification and version declarations, and check that Cosign installers use pinned commits and consistent releases.
Sign and upload release assets `hack/upload-assets.sh`, `hack/upload-assets.bats`, `docs/release.md`	The upload script signs its asset list before uploading it with the SBOM, checksums, and signature bundle. Tests check upload order, checksum coverage, and failure behavior. The documentation describes dispatch refs, artifact contents, and verification.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant UploadAssets
  participant SignReleaseAssets
  participant Syft
  participant Cosign
  participant GHCLI
  UploadAssets->>SignReleaseAssets: pass version and asset paths
  SignReleaseAssets->>Syft: generate SPDX SBOM
  SignReleaseAssets->>Cosign: sign and verify checksum bundle
  SignReleaseAssets-->>UploadAssets: return after verification
  UploadAssets->>GHCLI: upload assets and generated metadata

Merge Risk: ⚪ Minimal · up to bc24b

No actionable release-asset issue remains on the supplied evidence. The first release run should still confirm keyless signing end to end.

Pre-merge checks: 5

Check name	Status	Explanation
Title check		The title clearly summarizes the main changes: signing release assets and attaching a source SBOM.
Docstring Coverage		Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 11 functions across 5 files. (3 skipped: 3…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.

Full details: Docstring Coverage

Explanation

Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 11 functions across 5 files. (3 skipped: 3 unsupported.)

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

myasnikovdaniil

Requesting changes for two things, syft install in the release jobs and no minimum cosign version in the verification docs.

syft is installed by a script fetched at run time

anchore/sbom-action/download-syft is pinned by commit, but at run time it downloads install.sh from raw.githubusercontent.com/anchore/syft/v1.51.1/install.sh and runs it with sh (src/github/SyftGithubAction.ts:269-291 at 3ad7283). It is a tag in other repo, so the job runs whatever the tag points to at that moment.

We already moved flux away from this in 6df5d07 (script "executed on runners that later hold registry credentials"), and promote-rc.yaml:425-428 has the rule written as "runs no script fetched at run time". hack/ci-tool-installs-contract.bats covers only flux, yq and helm-unittest, so nothing catches syft. In tags.yaml:538-540 the step runs after Login to GHCR (tags.yaml:435) and after the app token is put into the remote URL (tags.yaml:490), in promote-rc.yaml:584-585 it is the same after Prepare stable branch (promote-rc.yaml:490). Both jobs have contents: write and packages: write, and this PR adds id-token: write. If the tag or the file behind it is ever replaced, the step can get OIDC token for tags.yaml or promote-rc.yaml identity, which is the signature we tell users to trust, and read the app token and ghcr credentials. It needs upstream compromise, but it is the same class as the flux case.

Please install syft like the yq step in promote-rc.yaml does, download syft_<version>_linux_amd64.tar.gz from the release url and check a committed sha256, in both jobs. hack/ci-tool-installs-contract.bats should fail on download-syft or on a syft download without sha256sum -c. While there, v1.51.1 is in the range of GHSA-mw2c-m758-9v5q (memory exhaustion parsing binary inputs, fixed in v1.52.0), sha256 of syft_1.52.0_linux_amd64.tar.gz is caeedb81fb0491615f1ebd1761e4145d41ee86dd2cc7bf80669f9f5ad9d6133d (from the release checksums file).

cosign version in the verification docs

docs/release.md:339-342 tells users to run cosign verify-blob --bundle ... --certificate-identity-regexp ... --certificate-oidc-issuer ... and names no cosign version. GHSA-fx35-mq7g-6g98 (CVSS 7.4, fixed in v3.1.3 and v2.6.5): a legacy format --bundle with a bare public key in the cert field skips identity and issuer checks, and verification ends with exit 0 if the signature matches that key.

The format is picked from file content (verify_blob.go:117 at v2.6.5, NewBundleFormat || checkNewBundle(BundlePath)) and the documented command has no --new-bundle-format. Bundles we write are protobuf (--new-bundle-format defaults to true in v3.0.6, options/signblob.go:108), so they are not affected, but the user verifies whatever file lies next to the download. Whoever can replace downloads (a mirror, a proxy in the path) can give a legacy bundle with own key, and on cosign 3.1.2 or older and 2.6.4 or older the documented command reports success. The job's own verify-blob is fine, it checks the bundle it just wrote.

Please name the minimum version (v3.1.3 or v2.6.5) before the command. Adding --new-bundle-format to the command should also make cosign refuse a legacy file (verify_blob.go:166-175 at v2.6.5), I didn't run it.

What the SBOM covers (not a blocker, I want your opinion)

The file tells what is inside the cozypkg binaries and the dependencies of the source tree, but not what is inside the container images and the Talos ISO and disk images, and these are what runs in the cluster. By the numbers in the description 139 of 995 packages come from shipped binaries, the rest are lockfile and workflow entries of the source tree (373 npm, 99 GitHub Actions references), and packages/ pins 70 image repositories by digest while the breakdown in the description has no images. #4759 says image SBOMs are complementary, but the title, the description ("every release gets an SBOM"), the release note ("every release carries cozystack-sbom.spdx.json"), the commit message and docs/release.md:332 don't say it, and the planned update of the SBOM row in docs/security/self-assessment.md:13 doesn't say the image caveat stays.

Is this a first step to image SBOMs (#2937, A2 in #3278)? And can you write in docs/release.md, the description and the release note what the file covers and what not, and name it by content, like cozystack-source-sbom.spdx.json (CLOMonitor (?i)sbom still matches), so cozystack-sbom can be the SBOM of the release itself later? Name is cheaper to change before the first release.

Not blocking

docs/release.md:339-343 has the two commands on separate lines, so pasted into a script a rejected signature is followed by sha256sum --check and the block still exits 0. With a cosign stub that rejects the signature and a replacement payload with matching checksum it prints cosign: signature rejected, then metal-amd64.iso: OK, exit 0. With && between them exit is 1, please join.
Certificate identity accepts any ref (hack/sign-release-assets.sh:38, docs/release.md:341). docs/release.md:665-670 says tag rulesets restrict who creates v* tags, but the repo has only two branch rulesets today, so a tag pushed from any branch with edited tags.yaml gives certificate .../tags.yaml@refs/tags/vX that the documented command accepts. Either pin the ref in regexp (script, docs and the test that ties them) or say in docs that the signature proves a workflow run in this repository until tag rulesets exist. promote-rc.yaml may be dispatched from non-main refs, so this is for release owners to decide.
docs/release.md:330 says "Every release carries three files", not true until the first release cut from a tree with the script, and on maintenance lines before a backport. The PR leaves security-insights.yml and docs/security/self-assessment.md:13 alone for this reason, same wording rule should be here.
Closes #4759 closes the only issue that lists the two deferred items (project.documentation.signature-verification in security-insights.yml and the SBOM row in docs/security/self-assessment.md:13). Keep it open or file one for them.
hack/upload-assets.sh has no test. I ran the old and new script against stub gh, syft and cosign: same 18 assets in the same order plus the three new files, and zero gh calls when verify-blob fails, so behaviour is right but nothing pins it. hack/sign-release-assets.bats:192 ("refuses assets spread over two directories") still passes with the directory check removed, because sha256sum fails on the file missing in the first directory. With a file of the same name and different content in both directories the test fails without the guard, and without the guard the script would hash the first directory's bytes for both entries.

Marian Koreniuk (themoriarti) · 2026-10-16T13:34:10Z

myasnikovdaniil thanks, all points taken. Force-pushed as one commit (1571346d4); the body, title and release note are updated.

syft install. Both jobs now download syft_1.54.1_linux_amd64.tar.gz from the release URL into $RUNNER_TEMP, check it against a committed SHA-256 and put it on GITHUB_PATH. I went to v1.54.1 instead of v1.52.0 since it is the latest and outside GHSA-mw2c-m758-9v5q as well. c069905b391cc4c20a5ba65ad5c10be2a7ba074f8ea6ad203e24d14e303dad47 is the entry in syft_1.54.1_checksums.txt, and that file verifies with cosign against https://github.com/anchore/syft/.github/workflows/release.yaml@refs/heads/main. Your 1.52.0 hash matches its checksums file too. hack/ci-tool-installs-contract.bats gets two tests: one fails on anchore/sbom-action, on a syft install.sh or on get.anchore.io, and the other fails on a syft download with no sha256sum -c on SYFT_SHA256 within three lines, and on SYFT_VERSION/SYFT_SHA256 drift between the workflows. I moved the yq awk into a shared helper. Putting download-syft back, dropping the check line or bumping one version alone each fails it.

cosign version. docs/release.md names v3.1.3 or later, or v2.6.5 or later on the v2 line, before the command. I left --new-bundle-format out, after running it. With a key-signed legacy bundle whose cert is the bare public key, against the documented identity regexp (tlog ignored, since an attacker can log their own entry):

cosign	without flag	`--new-bundle-format`
v2.6.4	Verified OK	rejected (`proto: unknown field "base64Signature"`)
v2.6.5	rejected	rejected
v3.1.2	Verified OK	Verified OK
v3.1.3	rejected	rejected, with a deprecation warning for the flag

So on the v3 line the flag doesn't protect, and on v3.1.3 it only adds a warning. The docs explain why the version matters. The job itself keeps cosign-installer's default v3.0.6. It only verifies the bundle it has just written, so I didn't add a version input. Tell me if you would rather it ran v3.1.3 as well.

What the SBOM covers. Renamed to cozystack-source-sbom.spdx.json, in the script, docs, tests, commit message, title, description and release note. docs/release.md and the script header now say what it covers and what not: the container images packages/ pins by digest (#2937, A2 in #3278) and the Talos ISO, disk, kernel and initramfs images. It also says the name leaves cozystack-sbom for an SBOM of the release as a whole. On your question: I don't see it as a step towards image SBOMs in code. Those are a build-side change (A2: SBOM=1 → --sbom=true, then the delivery decision), and this file stays the source and cozypkg layer. It would feed a release-level cozystack-sbom later, for example one that lists the digest-pinned images from packages/ and points to their attestations once A2 settles delivery. The self-assessment follow-up in the description now says the image caveat stays.

Not blocking:

Joined with &&.
I took the docs option. A passing check proves a run of tags.yaml or promote-rc.yaml in this repository, not a release cut through the documented flow. Until the v* tag rulesets exist, that includes a hand-pushed tag whose tree has an edited tags.yaml, and in any case a promote-rc.yaml edited on a branch and dispatched from it. If release owners confirm promote-rc.yaml is dispatched only from main, I'll pin the refs in the script, docs and test.
Reworded: "A release cut from a tree that has hack/sign-release-assets.sh carries three files…". Earlier releases, and maintenance lines before a backport, have none of them.
Closes #4759 → Refs #4759 in the commit and the description, so the issue stays open for signature-verification in security-insights.yml and the self-assessment row.
New hack/upload-assets.bats with 4 tests, using the real sign-release-assets.sh with gh, syft and cosign stubbed. It checks the exact upload list and order (the fixture's 18 assets with two cozypkg tarballs, then the three files), that the checksums name every upload except the checksums and the bundle, and that there are zero gh calls when verification fails or an asset is missing. The old upload-assets.sh fails all four, and a version that signs after uploading fails the last two. The two-directory test now puts a same-named file with different content in both directories and checks that nothing is written; with the guard removed it fails.

Also re-run: a real syft 1.54.1 scan through the script (cosign stubbed) over this tree and the v1.6.5 cozypkg-linux-amd64.tar.gz gives 920 packages (440 golang, 373 npm, 103 github, 60 from the binary). The Set up syft step runs as written. Bats 1.14.0: 89/89 across the three suites and the five meta-suites. zizmor offline is clean, actionlint shows nothing new, shellcheck and check-commit-trailers.sh pass.

myasnikovdaniil

Requesting changes, but only for two small things, the rest is fine. Syft install, cosign version in the docs, SBOM name and the tests are done as I asked. You are right about --new-bundle-format, I reproduced it: v3.1.2 accepts the forged legacy bundle with and without the flag, v3.1.3 rejects both. My suggestion was wrong for v3. The image SBOM answer is fine with me, release level cozystack-sbom can wait for A2.

Pin the refs. promote-rc.yaml is meant to be dispatched from main only, and docs/release.md says it nowhere (only Cut Pre-release Tag says where to dispatch from), so please add that sentence when you pin. Script, docs and the test that ties them need the new regexp, and the reject list needs a branch ref for each workflow.
Run v3.1.3 in the jobs too. sigstore/cosign-installer v4.1.2 takes cosign-release: v3.1.3 and checks the binary and its keyless and KMS bundles with the pinned bootstrap cosign, so it is verified like the default. Docs tell users v3.1.3 or later, the job should verify with what users run, and v3.0.6 is in the advisory range anyway. The flags in sign-release-assets.sh are not deprecated in v3.1.3. If you keep the version next to the other pins, the contract test can hold it equal in both workflows.

myasnikovdaniil · 2026-10-19T08:26:52Z

+          echo "$syft_dir" >> "$GITHUB_PATH"
+      - name: Set up cosign
+        if: needs.build-amd64.outputs.release_exists == 'false'
+        uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2


Please add cosign-release: v3.1.3 under with: here and at promote-rc.yaml:607, see the review body.

Done, cosign-release: v3.1.3 here and in promote-rc.yaml. The contract test holds both equal.

myasnikovdaniil · 2026-10-19T08:26:52Z

+
+# The workflows that publish releases: tags.yaml for a pre-release, promote-rc.yaml
+# for a stable one. docs/release.md shows users the same pattern.
+IDENTITY_REGEXP='^https://github\.com/cozystack/cozystack/\.github/workflows/(tags|promote-rc)\.yaml@'


Pin the refs: '^https://github\.com/cozystack/cozystack/\.github/workflows/(tags\.yaml@refs/tags/v|promote-rc\.yaml@refs/heads/main$)'. I ran it against the accept cases and the fork and tags.yaml.evil rejects in hack/sign-release-assets.bats, and against promote-rc.yaml@refs/heads/release-1.6, tags.yaml@refs/heads/feature and promote-rc.yaml@refs/heads/main-evil, it accepts and rejects as expected. The regexp in docs/release.md has to change with it, the test that ties them checks that.

Done, this regexp is now in the script, the docs and the test. The reject list also has tags.yaml@refs/heads/main, promote-rc.yaml@refs/heads/release-1.6, @refs/heads/main-evil and @refs/tags/v1.2.3.

No release carried a signature or an SBOM, so users could verify none of the ISO, disk images, kernel, initramfs, manifests and cozypkg binaries they download, and the OpenSSF Scorecard Signed-Releases check and CLOMonitor's signed_releases and sbom checks failed. hack/upload-assets.sh now hands its asset list to the new hack/sign-release-assets.sh before the first upload. That script writes cozystack-source-sbom.spdx.json with syft over the committed tree and the cozypkg binaries, cozystack-checksums.txt with the SHA-256 of every asset and the SBOM, and cozystack-checksums.txt.sigstore.json with a keyless cosign signature over the checksums. The SBOM does not describe the container images or the Talos boot images, which is what its name says; image SBOMs are cozystack#2937. The script verifies the bundle against the identities of tags.yaml on a v* tag and promote-rc.yaml on main before returning, so a bad signature fails the job with nothing uploaded, and hack/upload-assets.bats pins the upload list and that order. The two jobs that call make upload_assets get id-token: write, cosign v3.1.3 from the SHA-pinned cosign-installer, and syft from its release tarball checked against a committed SHA-256, because anchore/sbom-action/download-syft runs an install script fetched at run time; hack/ci-tool-installs-contract.bats now covers both. docs/release.md describes the files, what the SBOM covers, the minimum cosign version to verify a download with, what the certificate identity proves, and that promote-rc.yaml is dispatched from main only. Refs cozystack#4759 Assisted-by: LLM Signed-off-by: Marian Koreniuk <[email protected]>

Marian Koreniuk (themoriarti) · 2026-10-19T19:20:22Z

myasnikovdaniil thanks for re-running the --new-bundle-format case. Both points are done, force-pushed as one commit (bc24b588b, delta from 1571346d4: 6 files, +58/−10).

Refs pinned to your regexp, in hack/sign-release-assets.sh, the docs/release.md command and the test. The reject list now also has tags.yaml@refs/heads/main, promote-rc.yaml@refs/heads/release-1.6, promote-rc.yaml@refs/heads/main-evil and promote-rc.yaml@refs/tags/v1.2.3. It gives the same results under grep -E and Go's regexp. "Dispatched from main only" is now in the three places that tell you to run promote-rc.yaml (regular release step 3, patch release step 4, the workflow list) and in the signatures section. The any-ref paragraph is replaced: what a passing check proves now, and the tag-ruleset caveat that is still open for tags.yaml.
cosign v3.1.3 in the jobs: cosign-release: v3.1.3 under both installer steps. I ran the installer's script at the pinned commit with that input. It verified the keyless and KMS bundles with the bootstrap cosign and installed a binary whose SHA-256 matches cosign_checksums.txt. hack/ci-tool-installs-contract.bats gets a test that fails on an installer not pinned by SHA, a step without cosign-release, and the release or the action commit differing between steps. Each of those mutations fails it.

One thing for the release owners, not in this PR: nothing in promote-rc.yaml refuses a non-main dispatch up front. Such a dispatch now fails at its upload step, on the job's own signature check. By then the staging branch, the promotion-* candidate and the empty draft release already exist. A re-dispatch from main reuses the draft and force-pushes the staging branch, and retention is left to clean up the abandoned candidate. A github.ref == 'refs/heads/main' check in parse would stop it before the first write. I can send that separately if you want it.

Re-checked: Bats 1.14.0, 90/90 across the three suites and the five meta-suites. zizmor offline is clean, actionlint shows nothing new, shellcheck and check-commit-trailers.sh pass.

myasnikovdaniil

Thanks, both points look good, approving. Yes to the github.ref == 'refs/heads/main' check in parse, please send it as a separate PR.

A release has signed its assets since cozystack#4861, but not the container images a cluster installed from it pulls, and the SBOM each image carries since cozystack#4997 sat in the registry only, readable with buildx and signed by nothing. A user could verify the ISO they booted and not the platform it installed. hack/sign-release-images.sh signs every first-party digest the repinned tree holds with cosign, keylessly, in prepare-release after the stitch. It skips a digest that already verifies, so a re-run stacks no second signature, and verifies each one against the identity docs/release.md names before hack/upload-assets.sh runs, so nothing is uploaded over an unsigned image. The signature over an index covers the attestation manifest the index lists, so the SBOM needs none of its own, and cosign stores it as a referrer of the digest in the repository hack/promote-retag.sh copies the digest within, so promote-rc.yaml only verifies, with the rc tree's script, before it stages the stable release. hack/collect-image-sboms.sh reads the SBOM attestation of every such image back into cozystack-images-sbom.tar.gz, one SPDX document per platform, and fails on an image that lacks one for any platform its index serves. With cozystack-images.txt, the digests the signatures are bound to, both join the asset list hack/upload-assets.sh hands to hack/sign-release-assets.sh, so the existing checksums signature covers them. docs/release.md gains the verification recipe for a whole release; the bats files pin the scripts, the recipe, the upload list and the two workflow steps. Refs cozystack#2937, cozystack#3278 Assisted-by: LLM Signed-off-by: Marian Koreniuk <[email protected]>

## What this PR does Backports to `release-1.7` from `main`, picked by the rules of the [backport policy](https://github.com/cozystack/cozystack/blob/main/docs/release/backports.md) in merge order. A merged pull request comes as its own commits, picked with `git cherry-pick -x` so each keeps its author, message and sign-offs, and a merge commit that carries the original's merge message with a `Backported-from:` trailer naming it; a squash-merged one comes as one pick. `backport-batch verify` checks that shape. A pick listed as changed differs from the commit it came from; `git range-diff <merge>^1..<merge>^2 <batch merge>^1..<batch merge>^2` shows how. A conflict or a failing check here is a [backport task](https://github.com/cozystack/cozystack/blob/main/docs/release/backport-tasks/README.md). - [#4756](#4756) `fix(backupstrategy-controller): run backup clients on a package-owned image`, 2 commits, changed in the pick: `48d0cb2958a5` - [#4867](#4867) `fix(e2e): keep chainsaw asserts polling through transient API errors`, 1 commit - [#4888](#4888) `fix(ci): dispatch frozen-line builds with GITHUB_TOKEN`, 1 commit - [#4985](#4985) `fix(etcd): stop counting canceled watch streams as failed gRPC requests`, 1 commit - [#5005](#5005) `fix(tcp-balancer): cap HAProxy fds and threads by the pod's own limits`, 1 commit - [#4917](#4917) `fix(kubernetes): run Proxmox CCM v0.16.1 and pin the Proxmox images by digest`, 2 commits - [#4974](#4974) `fix(kubernetes-nodes): plan autoscaler capacity on the cores the VM actually gets`, 1 commit - [#4971](#4971) `fix(e2e): keep the Proxmox tenant kubeconfig per run and remove it`, 2 commits - [#4970](#4970) `fix(e2e): read the KubernetesNodes pool release in the HelmRelease guard`, 3 commits - [#4972](#4972) `fix(e2e): validate the leak allowlist against every suite layout`, 3 commits - [#5082](#5082) `fix(marketplace): pin a tapped repository to a digest instead of a mutable tag`, 15 commits, changed in the pick: `a31ae9090928`, `b326e0badd62`, `a3d883b4bf5b`, `f8509450f7ca` - [#4004](#4004) `fix(dashboard): link a tenant row to its detail page`, 2 commits - [#5212](#5212) `fix(cilium): pass the CiliumNode list to jq on stdin in the IPAM hook`, 1 commit - [#5213](#5213) `fix(kubernetes): move the OIDC users out of the bootstrap Job's script`, 1 commit - [#5208](#5208) `fix(clickhouse): restore the archive a Backup was taken as`, 3 commits - [#5230](#5230) `fix(backups): re-render the default backup objects when the S3 endpoint changes`, 8 commits - [#5133](#5133) `fix(securitygroups): project a membership-only group to a policy Cilium accepts`, 1 commit - [#5091](#5091) `fix(postgres): render the declared databases in a stable order`, 1 commit - [#5123](#5123) `fix(mariadb-operator): derive the cluster name from the platform domain`, 2 commits - [#5151](#5151) `fix(redis): run one Sentinel when Redis runs a single pod`, 5 commits - [#5124](#5124) `fix(lineage, kubernetes): resolve CAPK node VMs to their cluster`, 2 commits - [#5184](#5184) `fix(api): read pool usage under the quota key the controller counts`, 1 commit - [#5190](#5190) `fix(platform): remove the dead registries value and patch-containerd Secret`, 2 commits - [#5211](#5211) `fix(postgres): fail a backup job as soon as its cnpg Backup fails`, 1 commit - [#5155](#5155) `fix(postgres): refuse system databases and never drop them`, 1 commit - [#5207](#5207) `fix(mariadb): fail backup and restore jobs whose operator Job failed`, 5 commits - [#5024](#5024) `fix(kubernetes, monitoring): send tenant cluster metrics to the tenant's vminsert`, 1 commit ### Maintainer overrides - [#5083](#5083) `fix(workloadmonitor): stop doubling the bucket- prefix in bucket Workload names`: `backport/skip`, left out - [#4973](#4973) `fix(kubernetes-nodes): refuse a partial resources spec on every pool`: `backport/skip`, left out - [#5180](#5180) `fix(tests): keep two unit tests green after the release image restamp`: `backport/skip`, left out ### Skipped - Rule 1, already on `release-1.7`: 19 - Rule 2, the title type is not `fix` or `security`, or the scope is `deps`: 26 - Rule 3, a breaking change: 5 - Rule 4, touches platform migrations, `targetVersion` or a CRD, or makes a sizeable API change: 2 <details><summary>Rule 1, already on `release-1.7`</summary> - [#4815](#4815) `fix(vm-disk): name the importer error in the disk's WorkloadsReady message`, `its 2 commits picked, the last in 4323363` - [#4293](#4293) `fix(fluxcd): keep Helm uninstall working during flux-plunger recovery`, `its 1 commit picked, the last in 0210407` - [#4791](#4791) `fix(kubernetes): keep NFS publish from hanging on a dead server`, `its 1 commit picked, the last in fa88bd1` - [#4830](#4830) `fix(multus): drop the CPU limit and make resources configurable`, `its 2 commits picked, the last in 363c842` - [#4865](#4865) `fix(kafka): pass the container limits to the Kafka JVMs`, `its 3 commits picked, the last in fc38a54` - [#4866](#4866) `fix(linstor): mark volumes restored from a snapshot DRBD-initialized`, `its 1 commit picked, the last in f9550b7` - [#4819](#4819) `fix(seaweedfs): build SeaweedFS in-tree with a bounded S3 read-ahead`, `its 5 commits picked, the last in 7f4c346` - [#3427](#3427) `fix(backups): prepare standalone VMDisk restore targets and add VM backup/restore e2e round-trips`, `its 3 commits picked, the last in 4ac0cd5` - [#4889](#4889) `fix(monitoring): send alerts to every Alertmanager replica`, `its 1 commit picked, the last in 5033d53` - [#4929](#4929) `fix(backups): gate every default Strategy CR on its CRD`, `its 2 commits picked, the last in dee1eef` - [#4914](#4914) `fix(fluxcd): close the flux-plunger review follow-ups for the hook, drain and Warnings`, `its 7 commits picked, the last in a8e4916` - [#4939](#4939) `fix(cluster-api): resync capi-operator every minute`, `its 1 commit picked, the last in cc8d6a4` - [#5027](#5027) `fix(hack): keep the backup stack in the vminstance selective install`, `its 1 commit picked, the last in 8ba8386` - [#4955](#4955) `fix(vm-disk): let CDI adopt the PVC a Velero restore recreates`, `its 2 commits picked, the last in 691d7bc` - [#5071](#5071) `fix(marketplace): do not install a connected repository's components on connect`, `its 2 commits picked, the last in 25f8f11` - [#5086](#5086) `fix(tenant): stop the gateway release waiting on TenantGateway listeners and certificates`, `its 1 commit picked, the last in 18b2249` - [#5087](#5087) `fix(controller): carry healthCheckExprs and waitStrategy to existing releases`, `its 1 commit picked, the last in d2a452e` - [#5088](#5088) `fix(ingress-nginx): drop the admission-webhook policy a rebase brought back`, `its 1 commit picked, the last in cf00b41` - [#5142](#5142) `fix(release): keep release bodies within GitHub's size limit`, `its 1 commit picked, the last in c740515` </details> <details><summary>Rule 2, the title type is not `fix` or `security`, or the scope is `deps`</summary> - [#4845](#4845) `docs(governance): welcome new contributors and link LF sanctions guidance`, `type docs` - [#4881](#4881) `feat(bucket): add a ui flag to skip the s3manager web UI`, `type feat` - [#4899](#4899) `chore: link the FOSSA license scan for CLOMonitor`, `type chore` - [#4898](#4898) `feat(backups): back up Valkey with the Redis strategy`, `type feat` - [#4940](#4940) `docs(release): add changelog for v1.6.5`, `type docs` - [#4861](#4861) `feat(release): sign the release assets and attach a source SBOM`, `type feat` - [#3780](#3780) `docs: move the operator guides out of docs/ to where readers find them`, `type docs` - [#4936](#4936) `docs(release): check changelog claims against the code`, `type docs` - [#4959](#4959) `docs(release): point changelog check B at the Talos installer profile`, `type docs` - [#3777](#3777) `docs(agents): define where a new document belongs`, `type docs` - [#5025](#5025) `test(harbor): pin the resources of the nginx proxy and the portal`, `type test` - [#5039](#5039) `feat(keycloak): load additional CA files from Secret or ConfigMap keys`, `type feat` - [#4860](#4860) `feat(kubernetes-nodes): trust a factory CA in the Proxmox template script`, `type feat` - [#5090](#5090) `ci(e2e): end the Proxmox lane's SSH key with a newline`, `type ci` - [#4969](#4969) `test(kubernetes): pin RetryOnFailure on the Proxmox csi release`, `type test` - [#5000](#5000) `test(kubernetes): name the Proxmox fixture's node pool <cluster>-<pool>`, `type test` - [#5168](#5168) `docs(agents): steer agents away from Helm lookup`, `type docs` - [#4997](#4997) `feat(release): attach an SBOM to every release image`, `type feat` - [#3440](#3440) `ci(api-review-gate): publish the gate as a commit status`, `type ci` - [#5034](#5034) `docs: move the repo-only guides onto the website`, `type docs` - [#5215](#5215) `feat(monitoring-agents): export PVC labels from kube-state-metrics`, `type feat` - [#5217](#5217) `feat(monitoring-agents): keep application identity in container log streams`, `type feat` - [#5171](#5171) `feat(api): expose HelmRelease failure state on Application status`, `type feat` - [#5237](#5237) `feat(operator): let the platform source trust a private registry CA`, `type feat` - [#5106](#5106) `feat(release): make backports automatic with backport-batch`, `type feat` - [#4983](#4983) `feat(api): add the reservation evaluator for tenant quotas`, `type feat` </details> <details><summary>Rule 3, a breaking change</summary> - [#4790](#4790) `fix(platform)!: refuse a networking variant change on a live cluster`, `! in the title` - [#4796](#4796) `fix(vm-disk)!: refuse a golden-image clone CDI would leave unbound`, `! in the title` - [#5085](#5085) `fix(kubernetes, opensearch, foundationdb)!: restrict managed application images`, `! in the title` - [#4818](#4818) `fix(keycloak)!: make the KMS proxy keep its keys across restarts`, `! in the title` - [#5167](#5167) `fix(monitoring-agents)!: isolate merged log payload under log_processed`, `! in the title` </details> <details><summary>Rule 4, touches platform migrations, `targetVersion` or a CRD, or makes a sizeable API change</summary> - [#4941](#4941) `fix(backups): keep the backupstrategy-controller Helm release under the Secret size limit`, `a sizeable API change (breaking-change: [v1alpha1] spec.spec.template: field "metadata" was removed, and 5 more), a CRD` - [#5223](#5223) `fix(backups): report sizeBytes for MariaDB and CNPG backup artifacts`, `a CRD` </details> ```backport-batch {"line":"release-1.7","deferred":[]} ``` ### Screenshots See the original pull requests. ### Downstream repositories - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(backupstrategy-controller): the ClickHouse (Altinity) and RabbitMQ backup strategy Pods now run a dedicated `backup-http-client` image (alpine + curl + jq + ca-certificates, non-root, ~7 MB) instead of the 349 MB `platform-migrations` image. Air-gapped mirrors need the new `ghcr.io/cozystack/cozystack/backup-http-client` repository. fix(e2e): a chainsaw assert in the e2e suites keeps polling through a transient API error such as an etcd leader change, instead of failing the suite long before its timeout. fix(ci): use GITHUB_TOKEN to start the initial packages build when freezing a release line fix(etcd): the tenant etcd alert etcdHighNumberOfFailedGRPCRequests now counts only the gRPC codes that the etcd mixin and the platform copy of the alert count, so the Canceled watch streams of a Kubernetes API server that stops no longer fire it on every etcd member. fix(tcp-balancer): HAProxy now takes its fd and thread caps from the replica's memory and CPU limits instead of the node, so a t1.nano balancer is no longer OOMKilled on nodes with many CPUs. maxconn follows the memory limit at about 16 connections per MiB, 2035 on t1.nano, and the pods roll once on upgrade. fix(kubernetes): the Proxmox cloud-controller-manager now runs v0.16.1, and the Proxmox CCM and CSI driver images are pinned by digest. The v0.15.0 tag referenced before was re-pushed upstream to a build that panics on start, which left every new Proxmox worker with the uninitialized taint. fix(marketplace): a tapped repository is pinned to a digest instead of following a mutable tag, so moving the tag upstream no longer changes what runs in the cluster; connect accepts oci://...＠sha256:... and an optional keyless cosign verify policy (anchored to an exact identity and preserved across re-connects), and with verify flags the CLI verifies the signature and pins the verified digest at tap time. fix(dashboard): tenant name in the tenant list links to its detail page now (where Delete is), and top level tenants get their Edit button back. fix(cilium): the IPAM migration hook no longer fails with "Argument list too long" on clusters with more than a few nodes, which blocked the cilium upgrade and every package that depends on networking. fix(kubernetes): the OIDC bootstrap Job no longer fails to start for a tenant cluster with about 240 or more oidc.users, the bindings now come from a ConfigMap the Job mounts. fix(clickhouse): a RestoreJob now restores the archive its Backup was taken as, not the newest one. New Backups record the archive name, Backups taken before this release are restored best effort from the newest archive stamped no later than their takenAt, with a warning in the restore pod log. A ClickHouse BackupJob created again under a name whose Backup still exists now fails instead of reporting that Backup as its result. fix(backups): the backup controller now re-renders the Velero BSL, the default backup strategies and the platform bucket's credentials Secret when their S3 endpoint is stale, so after the platform bucket's BucketAccess is re-granted for a new publishing.host, backups move to the new endpoint without forcing releases by hand. fix(securitygroups): a SecurityGroup with attachments and no rules no longer leaves an invalid CiliumNetworkPolicy behind; its backing policy is now accepted by Cilium and still allows or denies nothing. fix(postgres): the init script renders the declared databases in a stable order, so an upgrade that leaves the databases unchanged no longer rewrites the init-script Secret. fix(mariadb-operator): the MariaDB operator now uses the platform's networking.clusterDomain instead of a hard-coded cozy.local, so replication and User/Database/Grant reconciliation work on clusters with a non-default cluster domain. fix(redis): an instance with `replicas: 1` now runs one Sentinel instead of three, without a Sentinel PodDisruptionBudget; on upgrade, the PDB the operator created earlier is deleted. Instances with two replicas or more are unchanged. fix(lineage, kubernetes): the virt-launcher pods of Kubernetes worker nodes are now resolved to their Kubernetes application and marked as managed by Cozystack, so a SecurityGroup attached to a Kubernetes cluster now applies to its worker nodes, not only to its control plane. fix(api): tenant quota admission now counts the parent pool's usage of extended resources and hugepages, so a child tenant can no longer get a share the parent has already used. After upgrading, updates to existing tenants may be rejected if their quota for these resources exceeds the parent's remaining budget. This includes updates that leave the quota unchanged. fix(postgres): a Postgres BackupJob now fails as soon as its CloudNativePG Backup fails, with the recorded error, instead of after 30 minutes, and a BackupJob waiting on a CloudNativePG Backup reports a WaitingForCNPGBackup condition with the Backup's phase. fix(postgres): databases now refuses the PostgreSQL system databases postgres, template0 and template1, and the init job never runs DROP DATABASE on them. A system database a previous version took over is handed back to the superuser on the next upgrade (event trigger removed, ownership and public schema restored, its <db>_admin and <db>_readonly roles dropped). A release that declares one of them must remove the entry before upgrading. fix(mariadb): a MariaDB BackupJob or RestoreJob whose operator Job failed now ends Failed instead of Succeeded, and a failed backup no longer creates a Ready Backup without an archive. Backups created by failed runs before this fix are not removed; restoring from one can fail or select another dump from the shared prefix. fix(kubernetes, monitoring): the vmagent of tenant Kubernetes clusters now writes to the tenant's vminsert and labels series with the owning tenant instead of `cozy-monitoring`. Previously no tenant cluster metric reached the tenant's Monitoring. ```

## What this PR does Backports to `release-1.7` from `main`, picked by the rules of the [backport policy](https://github.com/cozystack/cozystack/blob/main/docs/release/backports.md) in merge order. A merged pull request comes as its own commits, picked with `git cherry-pick -x` so each keeps its author, message and sign-offs, and a merge commit that carries the original's merge message with a `Backported-from:` trailer naming it; a squash-merged one comes as one pick. `backport-batch verify` checks that shape. A pick listed as changed differs from the commit it came from; `git range-diff <merge>^1..<merge>^2 <batch merge>^1..<batch merge>^2` shows how. A conflict or a failing check here is a [backport task](https://github.com/cozystack/cozystack/blob/main/docs/release/backport-tasks/README.md). - [#5284](#5284) `fix(seaweedfs): bump the COSI driver to v0.4.0 for the lost-grant fix`, 1 commit ### Maintainer overrides - [#5083](#5083) `fix(workloadmonitor): stop doubling the bucket- prefix in bucket Workload names`: `backport/skip`, left out - [#4973](#4973) `fix(kubernetes-nodes): refuse a partial resources spec on every pool`: `backport/skip`, left out - [#5180](#5180) `fix(tests): keep two unit tests green after the release image restamp`: `backport/skip`, left out ### Skipped - Rule 1, already on `release-1.7`: 46 - Rule 2, the title type is not `fix` or `security`, or the scope is `deps`: 31 - Rule 3, a breaking change: 5 - Rule 4, touches platform migrations, `targetVersion` or a CRD, or makes a sizeable API change: 2 <details><summary>Rule 1, already on `release-1.7`</summary> - [#4815](#4815) `fix(vm-disk): name the importer error in the disk's WorkloadsReady message`, `its 2 commits picked, the last in 4323363` - [#4756](#4756) `fix(backupstrategy-controller): run backup clients on a package-owned image`, `its 2 commits picked, the last in 4643366` - [#4293](#4293) `fix(fluxcd): keep Helm uninstall working during flux-plunger recovery`, `its 1 commit picked, the last in 0210407` - [#4791](#4791) `fix(kubernetes): keep NFS publish from hanging on a dead server`, `its 1 commit picked, the last in fa88bd1` - [#4830](#4830) `fix(multus): drop the CPU limit and make resources configurable`, `its 2 commits picked, the last in 363c842` - [#4865](#4865) `fix(kafka): pass the container limits to the Kafka JVMs`, `its 3 commits picked, the last in fc38a54` - [#4866](#4866) `fix(linstor): mark volumes restored from a snapshot DRBD-initialized`, `its 1 commit picked, the last in f9550b7` - [#4867](#4867) `fix(e2e): keep chainsaw asserts polling through transient API errors`, `its 1 commit picked, the last in df75adc` - [#4819](#4819) `fix(seaweedfs): build SeaweedFS in-tree with a bounded S3 read-ahead`, `its 5 commits picked, the last in 7f4c346` - [#3427](#3427) `fix(backups): prepare standalone VMDisk restore targets and add VM backup/restore e2e round-trips`, `its 3 commits picked, the last in 4ac0cd5` - [#4889](#4889) `fix(monitoring): send alerts to every Alertmanager replica`, `its 1 commit picked, the last in 5033d53` - [#4929](#4929) `fix(backups): gate every default Strategy CR on its CRD`, `its 2 commits picked, the last in dee1eef` - [#4914](#4914) `fix(fluxcd): close the flux-plunger review follow-ups for the hook, drain and Warnings`, `its 7 commits picked, the last in a8e4916` - [#4888](#4888) `fix(ci): dispatch frozen-line builds with GITHUB_TOKEN`, `its 1 commit picked, the last in bf81e2d` - [#4939](#4939) `fix(cluster-api): resync capi-operator every minute`, `its 1 commit picked, the last in cc8d6a4` - [#5027](#5027) `fix(hack): keep the backup stack in the vminstance selective install`, `its 1 commit picked, the last in 8ba8386` - [#4955](#4955) `fix(vm-disk): let CDI adopt the PVC a Velero restore recreates`, `its 2 commits picked, the last in 691d7bc` - [#4985](#4985) `fix(etcd): stop counting canceled watch streams as failed gRPC requests`, `its 1 commit picked, the last in 38e4089` - [#5005](#5005) `fix(tcp-balancer): cap HAProxy fds and threads by the pod's own limits`, `its 1 commit picked, the last in 7a3d0b0` - [#5071](#5071) `fix(marketplace): do not install a connected repository's components on connect`, `its 2 commits picked, the last in 25f8f11` - [#5086](#5086) `fix(tenant): stop the gateway release waiting on TenantGateway listeners and certificates`, `its 1 commit picked, the last in 18b2249` - [#5087](#5087) `fix(controller): carry healthCheckExprs and waitStrategy to existing releases`, `its 1 commit picked, the last in d2a452e` - [#5088](#5088) `fix(ingress-nginx): drop the admission-webhook policy a rebase brought back`, `its 1 commit picked, the last in cf00b41` - [#5142](#5142) `fix(release): keep release bodies within GitHub's size limit`, `its 1 commit picked, the last in c740515` - [#4917](#4917) `fix(kubernetes): run Proxmox CCM v0.16.1 and pin the Proxmox images by digest`, `its 2 commits picked, the last in 8ff6dca` - [#4974](#4974) `fix(kubernetes-nodes): plan autoscaler capacity on the cores the VM actually gets`, `its 1 commit picked, the last in d68a163` - [#4971](#4971) `fix(e2e): keep the Proxmox tenant kubeconfig per run and remove it`, `its 2 commits picked, the last in 77227c5` - [#4970](#4970) `fix(e2e): read the KubernetesNodes pool release in the HelmRelease guard`, `its 3 commits picked, the last in 6b17a76` - [#4972](#4972) `fix(e2e): validate the leak allowlist against every suite layout`, `its 3 commits picked, the last in 7ec550b` - [#5082](#5082) `fix(marketplace): pin a tapped repository to a digest instead of a mutable tag`, `its 15 commits picked, the last in 9cff86f` - [#4004](#4004) `fix(dashboard): link a tenant row to its detail page`, `its 2 commits picked, the last in bd5e0b5` - [#5212](#5212) `fix(cilium): pass the CiliumNode list to jq on stdin in the IPAM hook`, `its 1 commit picked, the last in b28a2e2` - [#5213](#5213) `fix(kubernetes): move the OIDC users out of the bootstrap Job's script`, `its 1 commit picked, the last in 02dd8e5` - [#5208](#5208) `fix(clickhouse): restore the archive a Backup was taken as`, `its 3 commits picked, the last in 9ded5c5` - [#5230](#5230) `fix(backups): re-render the default backup objects when the S3 endpoint changes`, `its 8 commits picked, the last in 490eab5` - [#5133](#5133) `fix(securitygroups): project a membership-only group to a policy Cilium accepts`, `its 1 commit picked, the last in b7c50a5` - [#5091](#5091) `fix(postgres): render the declared databases in a stable order`, `its 1 commit picked, the last in b658dff` - [#5123](#5123) `fix(mariadb-operator): derive the cluster name from the platform domain`, `its 2 commits picked, the last in 17702dd` - [#5151](#5151) `fix(redis): run one Sentinel when Redis runs a single pod`, `its 5 commits picked, the last in 5524a41` - [#5124](#5124) `fix(lineage, kubernetes): resolve CAPK node VMs to their cluster`, `its 2 commits picked, the last in c908560` - [#5184](#5184) `fix(api): read pool usage under the quota key the controller counts`, `its 1 commit picked, the last in 18c17af` - [#5190](#5190) `fix(platform): remove the dead registries value and patch-containerd Secret`, `its 2 commits picked, the last in 617f889` - [#5211](#5211) `fix(postgres): fail a backup job as soon as its cnpg Backup fails`, `its 1 commit picked, the last in 6645e21` - [#5155](#5155) `fix(postgres): refuse system databases and never drop them`, `its 1 commit picked, the last in 6f1bb80` - [#5207](#5207) `fix(mariadb): fail backup and restore jobs whose operator Job failed`, `its 5 commits picked, the last in ef3de55` - [#5024](#5024) `fix(kubernetes, monitoring): send tenant cluster metrics to the tenant's vminsert`, `its 1 commit picked, the last in 423a75b` </details> <details><summary>Rule 2, the title type is not `fix` or `security`, or the scope is `deps`</summary> - [#4845](#4845) `docs(governance): welcome new contributors and link LF sanctions guidance`, `type docs` - [#4881](#4881) `feat(bucket): add a ui flag to skip the s3manager web UI`, `type feat` - [#4899](#4899) `chore: link the FOSSA license scan for CLOMonitor`, `type chore` - [#4898](#4898) `feat(backups): back up Valkey with the Redis strategy`, `type feat` - [#4940](#4940) `docs(release): add changelog for v1.6.5`, `type docs` - [#4861](#4861) `feat(release): sign the release assets and attach a source SBOM`, `type feat` - [#3780](#3780) `docs: move the operator guides out of docs/ to where readers find them`, `type docs` - [#4936](#4936) `docs(release): check changelog claims against the code`, `type docs` - [#4959](#4959) `docs(release): point changelog check B at the Talos installer profile`, `type docs` - [#3777](#3777) `docs(agents): define where a new document belongs`, `type docs` - [#5025](#5025) `test(harbor): pin the resources of the nginx proxy and the portal`, `type test` - [#5039](#5039) `feat(keycloak): load additional CA files from Secret or ConfigMap keys`, `type feat` - [#4860](#4860) `feat(kubernetes-nodes): trust a factory CA in the Proxmox template script`, `type feat` - [#5090](#5090) `ci(e2e): end the Proxmox lane's SSH key with a newline`, `type ci` - [#4969](#4969) `test(kubernetes): pin RetryOnFailure on the Proxmox csi release`, `type test` - [#5000](#5000) `test(kubernetes): name the Proxmox fixture's node pool <cluster>-<pool>`, `type test` - [#5168](#5168) `docs(agents): steer agents away from Helm lookup`, `type docs` - [#4997](#4997) `feat(release): attach an SBOM to every release image`, `type feat` - [#3440](#3440) `ci(api-review-gate): publish the gate as a commit status`, `type ci` - [#5034](#5034) `docs: move the repo-only guides onto the website`, `type docs` - [#5215](#5215) `feat(monitoring-agents): export PVC labels from kube-state-metrics`, `type feat` - [#5217](#5217) `feat(monitoring-agents): keep application identity in container log streams`, `type feat` - [#5171](#5171) `feat(api): expose HelmRelease failure state on Application status`, `type feat` - [#5237](#5237) `feat(operator): let the platform source trust a private registry CA`, `type feat` - [#5106](#5106) `feat(release): make backports automatic with backport-batch`, `type feat` - [#4983](#4983) `feat(api): add the reservation evaluator for tenant quotas`, `type feat` - [#5264](#5264) `feat(release): keep a line's backport record and reports across batch pull requests`, `type feat` - [#5260](#5260) `ci(e2e): skip final cleanup on disposable PR runners`, `type ci` - [#5046](#5046) `feat(backups): record the RDB snapshot location and size for Redis backups`, `type feat` - [#5214](#5214) `feat(observability-proxy): serve tenant namespace logs through an authorizing proxy`, `type feat` - [#5282](#5282) `test(release): isolate report deduplication from workflow attempts`, `type test` </details> <details><summary>Rule 3, a breaking change</summary> - [#4790](#4790) `fix(platform)!: refuse a networking variant change on a live cluster`, `! in the title` - [#4796](#4796) `fix(vm-disk)!: refuse a golden-image clone CDI would leave unbound`, `! in the title` - [#5085](#5085) `fix(kubernetes, opensearch, foundationdb)!: restrict managed application images`, `! in the title` - [#4818](#4818) `fix(keycloak)!: make the KMS proxy keep its keys across restarts`, `! in the title` - [#5167](#5167) `fix(monitoring-agents)!: isolate merged log payload under log_processed`, `! in the title` </details> <details><summary>Rule 4, touches platform migrations, `targetVersion` or a CRD, or makes a sizeable API change</summary> - [#4941](#4941) `fix(backups): keep the backupstrategy-controller Helm release under the Secret size limit`, `a sizeable API change (breaking-change: [v1alpha1] spec.spec.template: field "metadata" was removed, and 5 more), a CRD` - [#5223](#5223) `fix(backups): report sizeBytes for MariaDB and CNPG backup artifacts`, `a CRD` </details> ```backport-batch {"version":2,"line":"release-1.7","built":{"main":"76a34d3e53a14c39d8df63eec6271c2eaae5ffea","line":"3cd9d4c220b519972677c7ff756ae6c4db07ef4a","newer":"","policy":"76a34d3e53a14c39d8df63eec6271c2eaae5ffea","batch":"e27d6630a0008381f0bc105a87d117dfcc29680f","tool":"76a34d3e53a14c39d8df63eec6271c2eaae5ffea","run":"https://github.com/cozystack/cozystack/actions/runs/43209950053","at":"2026-11-12T11:33:16Z","options":{"noWorkflowFiles":true,"mode":"new"}},"report":"9ca0285a0bb30a09736f6a430878b3ce0d32a2811639861e80daa549f208d7b3","closing":null,"deferred":[],"dropped":[],"stale":[]} ``` ```backport-batch-report {"format":1,"id":"9ca0285a0bb30a09736f6a430878b3ce0d32a2811639861e80daa549f208d7b3","line":"release-1.7","built":{"main":"76a34d3e53a14c39d8df63eec6271c2eaae5ffea","line":"3cd9d4c220b519972677c7ff756ae6c4db07ef4a","newer":"","policy":"76a34d3e53a14c39d8df63eec6271c2eaae5ffea","batch":"e27d6630a0008381f0bc105a87d117dfcc29680f","tool":"76a34d3e53a14c39d8df63eec6271c2eaae5ffea","run":"https://github.com/cozystack/cozystack/actions/runs/43209950053","at":"2026-11-12T11:33:16Z","options":{"noWorkflowFiles":true,"mode":"new"}},"overrides":{"backport/include":[],"backport/skip":[4973,5083,5180]},"primary":{"picked":[5284],"missing":[],"deferred":[],"parked":[],"skipped":{"on-line":[3427,4004,4293,4756,4791,4815,4819,4830,4865,4866,4867,4888,4889,4914,4917,4929,4939,4955,4970,4971,4972,4974,4985,5005,5024,5027,5071,5082,5086,5087,5088,5091,5123,5124,5133,5142,5151,5155,5184,5190,5207,5208,5211,5212,5213,5230],"before-start":[],"type":[3440,3777,3780,4845,4860,4861,4881,4898,4899,4936,4940,4959,4969,4983,4997,5000,5025,5034,5039,5046,5090,5106,5168,5171,5214,5215,5217,5237,5260,5264,5282],"breaking":[4790,4796,4818,5085,5167],"touches":[4941,5223],"newer":[],"skip-label":[4973,5083,5180],"dropped":[]}},"overlays":{"dropped":[],"stale":[],"changed":[]},"diag":{},"other":[]} ``` ### Screenshots See the original pull requests. ### Downstream repositories - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(seaweedfs): bucket users granted together could get S3 keys that never work (403 InvalidAccessKeyId). COSI driver is bumped to v0.4.0, which writes each grant to per-identity IAM file. ```

github-actions Bot added area/release Issues or PRs related to release tooling (changelog, backport, release pipeline) kind/feature Categorizes issue or PR as related to a new feature size/L This PR changes 100-499 lines, ignoring generated files labels Oct 11, 2026

Marian Koreniuk (themoriarti) marked this pull request as ready for review October 15, 2026 09:23

Marian Koreniuk (themoriarti) requested review from IvanHunters, Andrei Kvapil (kvaps), Aleksei Sviridkin (lexfrei), Timofei Larkin (lllamnyp), myasnikovdaniil and Timur Tukaev (tym83) as code owners October 15, 2026 09:23

myasnikovdaniil requested changes Oct 16, 2026

View reviewed changes

Marian Koreniuk (themoriarti) force-pushed the feat/release-signing-sbom branch from 20df0a4 to 1571346 Compare October 16, 2026 13:33

Marian Koreniuk (themoriarti) changed the title ~~feat(release): sign the release assets and attach an SBOM~~ feat(release): sign the release assets and attach a source SBOM Oct 16, 2026

github-actions Bot added size/XL This PR changes 500-999 lines, ignoring generated files and removed size/L This PR changes 100-499 lines, ignoring generated files labels Oct 16, 2026

Marian Koreniuk (themoriarti) requested a review from myasnikovdaniil October 16, 2026 19:28

myasnikovdaniil requested changes Oct 19, 2026

View reviewed changes

Marian Koreniuk (themoriarti) force-pushed the feat/release-signing-sbom branch from 1571346 to bc24b58 Compare October 19, 2026 19:20

Marian Koreniuk (themoriarti) requested a review from myasnikovdaniil October 19, 2026 19:26

myasnikovdaniil approved these changes Oct 20, 2026

View reviewed changes

myasnikovdaniil merged commit d1d4e92 into cozystack:main Oct 20, 2026
21 of 22 checks passed

This was referenced Oct 20, 2026

[Tracking] Remaining CI / build / release / E2E automation work #3278

Open

Finish the CI image-build refactor: SBOM emission and pr-* tag retention #2937

Open

feat(release): attach an SBOM to every release image #4997

Merged

This was referenced Oct 21, 2026

feat(release): sign the release images and attach their SBOMs #5042

Open

Sign release assets and attach an SBOM to each release #4759

Open

Marian Koreniuk (themoriarti) mentioned this pull request Oct 31, 2026

[Backport release-1.7] feat(release): sign the release assets and attach a source SBOM #5172

Open

1 of 12 tasks

Marian Koreniuk (themoriarti) mentioned this pull request Nov 2, 2026

ci(release): refuse a Promote RC dispatch from any ref but main #5181

Open

4 tasks done

cozystack-ci Bot mentioned this pull request Nov 9, 2026

fix(backport): batch for release-1.7, 2026-11-09 #5246

Merged

12 tasks

cozystack-ci Bot mentioned this pull request Nov 12, 2026

fix(backport): batch for release-1.7, 2026-11-12 #5290

Merged

12 tasks

cozystack-ci Bot mentioned this pull request Nov 13, 2026

fix(backport): batch for release-1.7, 2026-11-13 #5311

Draft

12 tasks

Navigation Menu

Repository navigation

feat(release): sign the release assets and attach a source SBOM - #4861

feat(release): sign the release assets and attach a source SBOM#4861
myasnikovdaniil merged 1 commit into
cozystack:mainfrom
themoriarti:feat/release-signing-sbom

Marian Koreniuk (themoriarti) commented Oct 11, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Oct 11, 2026 •

edited

Loading

Sequence Diagram(s)

Uh oh!

myasnikovdaniil left a comment

Uh oh!

Marian Koreniuk (themoriarti) commented Oct 16, 2026

Uh oh!

myasnikovdaniil left a comment

Uh oh!

myasnikovdaniil Oct 19, 2026

Uh oh!

Marian Koreniuk (themoriarti) Oct 19, 2026

Uh oh!

myasnikovdaniil Oct 19, 2026

Uh oh!

Marian Koreniuk (themoriarti) Oct 19, 2026

Uh oh!

Marian Koreniuk (themoriarti) commented Oct 19, 2026

Uh oh!

myasnikovdaniil left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Marian Koreniuk (themoriarti) commented Oct 11, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does

Decisions for the release owners

How it was checked

Screenshots

Downstream repositories

Release note

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Sequence Diagram(s)

Uh oh!

myasnikovdaniil left a comment

Choose a reason for hiding this comment

syft is installed by a script fetched at run time

cosign version in the verification docs

What the SBOM covers (not a blocker, I want your opinion)

Not blocking

Uh oh!

Marian Koreniuk (themoriarti) commented Oct 16, 2026

Uh oh!

myasnikovdaniil left a comment

Choose a reason for hiding this comment

Uh oh!

myasnikovdaniil Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

Marian Koreniuk (themoriarti) Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

myasnikovdaniil Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

Marian Koreniuk (themoriarti) Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

Marian Koreniuk (themoriarti) commented Oct 19, 2026

Uh oh!

myasnikovdaniil left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Marian Koreniuk (themoriarti) commented Oct 11, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Oct 11, 2026 •

edited

Loading