Skip to content

Commit 5e32ef3

Browse files
authored
feat(build): build Talos, Harbor, the Velero KubeVirt plugin and the unlisted images for arm64 (#4552)
## What this PR does This brings together four approved PRs that give the remaining first-party images an arm64 build. Each of them lost its approval when #4498 merged and GitHub retargeted it to main, so I merged them into one branch to review and approve once. Each original PR has the full description and review history, and I'll close them in favour of this one. - #4507 builds the Talos installer and matchbox for arm64. Releases also publish the arm64 ISO, disk images, kernel and initramfs. - #4515 builds the KubeVirt Velero plugin in-tree for amd64 and arm64. - #4525 adds flux-plunger, keycloak-operator, kilo and migration-controller to the root `build:` list, so CI rebuilds them like every other image. A test keeps any new image target from being left out. - #4528 rebuilds the Harbor v2.15.1 images for amd64 and arm64, because upstream publishes v2.15 for amd64 only. Release builds still run with `PLATFORM=linux/amd64`, except the Talos installer, so these images ship arm64 only once #4505 adds the arm64 leg. This PR makes that leg buildable. A few commits are new on top of the originals. The velero test no longer pins the plugin's image string, because the release build stamps its own rebuild over it. It now checks the plugin init container by repository name. A promote-retag test comment now says what it guards, and the image-refs doc no longer calls its fixed gaps open. The merge with main keeps the Harbor portal in the list of stages whose output has no architecture. ### Screenshots Not a UI change. ### Downstream repositories I left every box empty and opened nothing in other repositories. Two repositories are touched, and neither follow-up is needed before this merges: - cozystack/website names the amd64 assets in `content/en/docs/next/install/talos/iso.md` and `install/providers/oracle-cloud.md`. The install pages could offer the arm64 downloads once a release carries them. No asset is renamed, so the current links keep working. - cozystack/talm points at `ghcr.io/cozystack/cozystack/talos:<version>` in `charts/cozystack/values.yaml`. Once a release publishes that tag as a two-arch index, the same reference works on arm64 nodes, so nothing there has to change. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(talos): the Talos installer image is built for amd64 and arm64, with arm64 profiles that carry drbd and zfs, and the bootbox matchbox image network-boots arm64 machines as well as amd64 ones. Releases publish the arm64 Talos ISO, disk images, kernel and initramfs. feat(velero): the KubeVirt Velero plugin is now built in-tree for amd64 and arm64, so the velero server starts on arm64 clusters. build: flux-plunger, keycloak-operator, kilo and migration-controller are now rebuilt by CI like every other first-party image, so their pinned images pick up source patches and base-image updates and are published for amd64 and arm64. feat(harbor): the Harbor application now runs on arm64 nodes. Its component images are rebuilt from the pinned upstream v2.15.1 release for amd64 and arm64, because upstream publishes v2.15 images for amd64 only. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Talos images and boot assets now support both AMD64 and ARM64, including architecture-specific installation and cloud images. * Added build support for Harbor components and the KubeVirt Velero plugin. * Nightly builds now include ARM64 release assets. * **Improvements** * Harbor uses pinned chart and image versions, with built image references recorded automatically. * Added checks to catch image packages missing from the main build and to verify multi-architecture installer output. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 689b7ee + cb035f8 commit 5e32ef3

40 files changed

Lines changed: 944 additions & 151 deletions

‎.github/renovate.json‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,13 @@
7272
"groupName": "container image & action digests",
7373
"schedule": ["before 6am on monday"]
7474
},
75+
{
76+
"description": "The Harbor rebuilds reproduce one upstream release, so their build bases follow upstream's Makefile at that tag (GOBUILDIMAGE, NODEBUILDIMAGE) rather than the newest image. A golang tag bump builds Harbor with a toolchain upstream never shipped it with, and a node major breaks the Angular portal build. Tag bumps are disabled here; digest refreshes of the same tag still arrive through the digest group above. Move the tags by hand with HARBOR_VERSION.",
77+
"matchManagers": ["dockerfile"],
78+
"matchFileNames": ["packages/system/harbor/images/**"],
79+
"matchUpdateTypes": ["major", "minor", "patch"],
80+
"enabled": false
81+
},
7582
{
7683
"description": "kubevirt.io/csi-driver is pinned to an untagged pseudo-version commit and publishes no release tags, so Renovate's go datasource returns no-result and the lookup failure lingers on the Dependency Dashboard. Disable it; this module is advanced deliberately alongside the kubevirt-csi-driver image build.",
7784
"matchManagers": ["gomod"],

‎.github/workflows/nightly.yaml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -208,6 +208,9 @@ jobs:
208208
# rebuild of any cozystack image.
209209
- name: Build nocloud disk
210210
run: make -C packages/core/talos talos-nocloud
211+
env:
212+
# The e2e sandbox boots amd64 only.
213+
PLATFORM: linux/amd64
211214
- name: Publish disk to GHCR
212215
env:
213216
VERSION: ${{ needs.mirror.outputs.version }}

‎.github/workflows/pull-requests.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -524,6 +524,10 @@ jobs:
524524
env:
525525
DOCKER_CONFIG: ${{ runner.temp }}/.docker
526526
BUILDER: ${{ steps.buildx.outputs.name }}
527+
# amd64 only, as in the build job. The arm64 installer needs no QEMU,
528+
# but nothing a PR runs boots it. matchbox still gets the arm64 kernel
529+
# and initramfs.
530+
PLATFORM: linux/amd64
527531
IMAGE_TAG: pr-${{ github.event.pull_request.number }}-${{ github.event.pull_request.head.sha }}
528532
PUBLISH_VERSIONED: '0'
529533
PUBLISH_FLOATING: '0'

‎.github/workflows/tags.yaml‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -181,7 +181,11 @@ jobs:
181181
# Build project artifacts
182182
- name: Build
183183
if: steps.check_release.outputs.release_exists == 'false'
184-
run: make build
184+
run: |
185+
make build
186+
# The arm64 assets published below install this image, and building
187+
# its arm64 half needs no QEMU, so it is pushed again as both arches.
188+
make -C packages/core/talos image-talos PLATFORM=linux/amd64,linux/arm64
185189
env:
186190
DOCKER_CONFIG: ${{ runner.temp }}/.docker
187191
# Cache manifests are registry-portable (oci-mediatypes=true) — read

‎Makefile‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,13 +19,16 @@ build: build-deps
1919
make -C packages/system/backup-controller image
2020
make -C packages/system/backupstrategy-controller image
2121
make -C packages/system/lineage-controller-webhook image
22+
make -C packages/system/migration-controller image
2223
make -C packages/system/flux-shard-operator image
24+
make -C packages/system/flux-plunger image
2325
make -C packages/system/cilium image
2426
make -C packages/system/linstor image
2527
make -C packages/system/linstor-gui image
2628
make -C packages/system/kubeovn image
2729
make -C packages/system/kubeovn-webhook image
2830
make -C packages/system/kubeovn-plunger image
31+
make -C packages/system/kilo image
2932
make -C packages/system/dashboard image
3033
make -C packages/system/metallb image
3134
make -C packages/system/talos-log-collector image
@@ -38,7 +41,10 @@ build: build-deps
3841
make -C packages/system/securitygroup-controller image
3942
make -C packages/system/grafana-operator image
4043
make -C packages/system/redis-operator image
44+
make -C packages/system/harbor image
45+
make -C packages/system/velero image
4146
make -C packages/system/opensearch-operator image
47+
make -C packages/system/keycloak-operator image
4248
make -C packages/core/testing image
4349
make -C packages/core/talos image
4450
make -C packages/core/platform image

‎docs/agents/image-refs.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ If a package stamps a reference somewhere the two globs do not cover, add that f
4444

4545
- **Every first-party reference is tag+digest.** A bare tag with no digest is not reproducible and must not be committed. `securitygroup-controller:v0.0.0` is a live violation.
4646
- **Exactly one producer stamps each reference.** A reference no `Makefile` writes will never be refreshed by anything, including release preparation — `chBackupClientImage` in `packages/system/backupstrategy-controller/values.yaml` is a live violation, still pinned to `platform-migrations:v1.4.0-rc.2`.
47-
- **A package that stamps a reference is in the root `Makefile` `build:` list.** `build-matrix.sh`, the main build and release preparation all iterate that one list, so a package outside it is never rebuilt by any path and its pin freezes permanently. `flux-plunger`, `keycloak-operator` and `kilo` are live violations (tracked in #3416).
47+
- **A package that stamps a reference is in the root `Makefile` `build:` list.** `build-matrix.sh`, the main build and release preparation all iterate that one list, so a package outside it is never rebuilt by any path and its pin freezes permanently. `hack/build-matrix_test.bats` computes the set of packages whose Makefile runs `docker buildx build` and fails on any the list misses.
4848
- **Every consumer of references reads all three storage shapes**, via `hack/lib/image-refs.sh`.
4949
- **Scan wider than you rewrite.** A rewrite is deliberately narrow to avoid corrupting vendored defaults, so the check that follows it must not share that blind spot. `hack/promote-rewrite-tags.sh` rewrites the enumerated files, then greps the whole tree and fails if any reference to the release candidate survives.
5050

@@ -65,13 +65,13 @@ Two rows do not source the shared library, for opposite reasons. `hack/overlay-m
6565

6666
## Known gaps
6767

68-
These are real, currently unfixed, and predate the shared enumeration. They are recorded here so nobody rediscovers them as surprises.
68+
These predate the shared enumeration. Each entry says whether the tree works around it today; they are recorded here so nobody rediscovers them as surprises.
6969

70-
**The nightly host rewrite cannot reach a host split across two keys.** The primary pass is a literal `<src-registry>/` substring replace, so it only rewrites a reference whose host sits contiguously in front of the repository. Two layouts defeat it, and only one is now handled.
70+
**The nightly host rewrite cannot reach a host split across two keys.** The primary pass is a literal `<src-registry>/` substring replace, so it only rewrites a reference whose host sits contiguously in front of the repository. Two layouts in the tree defeated it; one is handled by a second expression and the other was removed from the tree.
7171

7272
`kubeovn` puts the whole host in `global.registry.address` with the bare repository in a sibling key, so there is no trailing slash to match. This one **is** rewritten now, by a second, end-of-line-anchored expression that fires only when the source registry is the complete scalar value — a contiguous `<src>/<repo>@<digest>` ref continues past the host, so the two expressions can never both fire on one reference. It had to be fixed when kubeovn's image build moved into this repository: until then the image was built and pushed to the public registry by cozystack/kubeovn-chart, the baked ref kept the committed public-registry host, and the ownership filter skipped it. The moment a CI build stamps the build-registry host into that key, the image is mirrored but the host is not rewritten, and the published tree points at the private build registry. Covered by `hack/nightly-mirror_test.bats`.
7373

74-
`keycloak-operator` remains unfixed. It splits at a different boundary — `registry: iad.ocir.io` beside `repository: idyksih5sir9/cozystack/<name>` — so neither key holds the source registry as a whole and no anchored replace can reach it; rewriting it needs a structure-aware pass that knows how the destination host splits. The gap stays dormant because no CI path rebuilds the package (#3416), and turns live the moment one does.
74+
`keycloak-operator` split at a different boundary: `registry: ghcr.io` beside `repository: cozystack/cozystack/<name>`, with an `image:` target that stamped only `tag`. A CI build would have rendered the committed public host with a digest that exists only in the build registry, and stamping the build host into `registry` instead would spread the source registry across two keys (`iad.ocir.io` and `idyksih5sir9/cozystack/<name>`), where no anchored replace can reach it. Rather than a structure-aware rewrite pass, the package now sets `registry: ""` and keeps the whole host in `repository`, which renders the same image string because the upstream chart joins the two keys, and its `image:` target stamps `repository` the same way. The rewrite still cannot reach a host split at that boundary, so a package must not stamp one: keep the host inside `repository`, or make it the complete value of its key as kubeovn does. Covered by `hack/nightly-mirror_test.bats`.
7575

7676
Collection understands both shapes — the host is rejoined, so a split-key ref carrying the source-registry host is still selected and mirrored. This is a defect in the host dimension; the version dimension the promote rewrite handles is unaffected, because the version always lives in a `tag` key regardless of where the host is.
7777

‎docs/agents/overview.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,7 @@ Build environment variables:
9494
- `REGISTRY` — Docker registry (default `ghcr.io/cozystack/cozystack`).
9595
- `PUSH=1` / `LOAD=0` — control buildx push/load behaviour.
9696
- `LOAD=1 PUSH=0` — load images locally instead of pushing. Builds for the host architecture only, because the classic docker image store cannot load a multi-platform index.
97-
- `PLATFORM` — the buildx platforms. Defaults to `linux/amd64,linux/arm64` for anything but `LOAD=1`, so a push from either architecture publishes an index both can run. Needs a `docker-container` builder (`BUILDER`) or a docker daemon on the containerd image store, with QEMU registered for the non-native half. Compiling builder stages run natively on `$BUILDPLATFORM` and cross-compile for the target, but every `RUN` step of a stage that runs on the target platform (package installs, `setcap`, install scripts, patch steps) still runs emulated. `packages/core/talos` and `packages/core/testing` pin `linux/amd64`, because they carry amd64-only payloads: the Talos assets and an x86 KVM sandbox.
97+
- `PLATFORM` — the buildx platforms. Defaults to `linux/amd64,linux/arm64` for anything but `LOAD=1`, so a push from either architecture publishes an index both can run. Needs a `docker-container` builder (`BUILDER`) or a docker daemon on the containerd image store, with QEMU registered for the non-native half. Compiling builder stages run natively on `$BUILDPLATFORM` and cross-compile for the target, but every `RUN` step of a stage that runs on the target platform (package installs, `setcap`, install scripts, patch steps) still runs emulated. `packages/core/testing` pins `linux/amd64`, because it carries an x86 KVM sandbox. `packages/core/talos` builds the Talos installer and release assets for the arches in `PLATFORM` with the upstream imager, which needs no QEMU for either; its matchbox image carries the amd64 and arm64 kernel and initramfs whatever `PLATFORM` says, because it network-boots machines of either arch.
9898

9999
### Values schema generation
100100

‎hack/build-matrix_test.bats‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,3 +156,26 @@
156156
actual=$(echo "$out" | jq 'length')
157157
[ "$expected" -eq "$actual" ]
158158
}
159+
160+
@test "every package that builds an image is in the build list" {
161+
# The build list is maintained by hand, and a package missing from it is
162+
# rebuilt by no CI path: its Dockerfile and patches keep changing while the
163+
# pinned image stays whatever was last pushed by hand. Derive the set from
164+
# the package Makefiles so a new image target cannot be forgotten.
165+
listed=$(sed -n '/^build:/,/^[^[:space:]]/p' Makefile \
166+
| grep -oE 'make -C packages/[A-Za-z0-9._/-]+ image' \
167+
| sed -E 's/^make -C (packages[^ ]+) image$/\1/')
168+
missing=""
169+
checked=0
170+
for m in packages/*/*/Makefile; do
171+
grep -q 'docker buildx build' "$m" || continue
172+
checked=$((checked + 1))
173+
d=${m%/Makefile}
174+
printf '%s\n' "$listed" | grep -qx "$d" || missing="$missing $d"
175+
done
176+
[ "$checked" -gt 0 ]
177+
if [ -n "$missing" ]; then
178+
echo "image-building packages missing from the root Makefile build: list:$missing" >&2
179+
false
180+
fi
181+
}

‎hack/common-envs_test.bats‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,3 +119,17 @@
119119
echo "$out" | grep -q -- '--load=1'
120120
if echo "$out" | grep -q -- '--platform'; then echo "FAIL: LOAD=1 passes a platform, so buildx builds an index it cannot load"; false; fi
121121
}
122+
123+
@test "talos pushes one installer index for PLATFORM and matchbox boots both arches" {
124+
# -B prints every asset build whether or not a local _out/assets has it.
125+
out=$(make -n -B -C packages/core/talos image IMAGE_TAG=pr-1-abc COZYSTACK_VERSION=0 BUILDER=b)
126+
echo "$out" | grep -q 'for a in amd64 arm64; do skopeo copy'
127+
echo "$out" | grep -q 'hack/installer-index.sh [^ ]* amd64 arm64;'
128+
echo "$out" | grep -q 'skopeo copy --all "$SRC" docker://[^ ]*/talos:pr-1-abc'
129+
# The installer follows PLATFORM, but matchbox serves machines of either arch
130+
# wherever it runs, so it still needs the arm64 kernel and initramfs.
131+
out=$(make -n -B -C packages/core/talos image IMAGE_TAG=pr-1-abc COZYSTACK_VERSION=0 BUILDER=b PLATFORM=linux/amd64)
132+
if echo "$out" | grep -q 'installer-arm64'; then echo "FAIL: an amd64 build builds the arm64 installer"; false; fi
133+
echo "$out" | grep -q 'profiles/kernel-arm64.yaml'
134+
echo "$out" | grep -q 'profiles/initramfs-arm64.yaml'
135+
}

‎hack/image-builders-cross-compile.bats‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -27,11 +27,12 @@ REPO_ROOT="$(cd "$(dirname "${BATS_TEST_FILENAME:-$0}")/.." && pwd)"
2727

2828
# Outputs with no architecture, one entry per line: a whole file, or
2929
# "<file>:<stage>" when only that stage is neutral and the file's other
30-
# stages still have to target TARGETARCH. The console and shadowbox's webpack
31-
# stage build static JS; piraeus-server builds Java .debs that
32-
# linstor-server's debian/control declares `Architecture: all`.
30+
# stages still have to target TARGETARCH. The console, the Harbor portal and
31+
# shadowbox's webpack stage build static JS; piraeus-server builds Java .debs
32+
# that linstor-server's debian/control declares `Architecture: all`.
3333
ARCH_NEUTRAL="packages/apps/vpn/images/shadowbox/Dockerfile:app
3434
packages/system/dashboard/images/console/Containerfile
35+
packages/system/harbor/images/harbor-portal/Dockerfile
3536
packages/system/linstor/images/piraeus-server/Dockerfile"
3637

3738
# is_neutral <file> <stage> <list>: 1 when the list names the file or that stage of it.

0 commit comments

Comments
 (0)