Skip to content

Commit 2ddd55c

Browse files
authored
fix(build): honour PUSH and LOAD from the environment and in image-talos (cozystack#4654)
## What this PR does A local build that asks to load images into docker no longer pushes them. `hack/common-envs.mk` set `PUSH` and `LOAD` with plain `:=`, and a makefile assignment beats the environment, so `LOAD=1 PUSH=0 make image` quietly pushed a multi-arch build. Both now take the environment value, and an empty one falls back to the default. I also made `PUSH` default to 0 when `LOAD=1`, so `LOAD=1` alone only loads. Before this, `make image LOAD=1` pushed and loaded at once. An explicit `PUSH` from the environment or the command line still wins, and `OCI_EXPORT_DIR` still forces both off. Every buildx target gets the values through `BUILDX_ARGS`, so the fix lives in that one file. The Talos installer was the one image that ignored `PUSH`. It is published with skopeo, not buildx, so it never saw `--push=$(PUSH)`. Its registry copies now run only when `PUSH=1`. A build that pushes nothing no longer writes `PUSHED_TAGS_LOG`. That log tells the multi-arch stitch which tags to move, so it must hold only tags that were really pushed. The cache helper lost an argument nobody used. No caller passes the second argument of `cache-args`, and an explicit tag there would bypass `CACHE_TAG`, so the amd64 and arm64 builds of that image would share one cache ref. CI sets neither `PUSH` nor `LOAD`, so CI builds do the same thing as before. Each fix has a `make --dry-run` test in `hack/common-envs_test.bats`. Closes cozystack#4499 Closes cozystack#4504 Closes cozystack#4551 ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The ccp skills pass `PUSH=1` explicitly, which still works. The website docs do not mention `PUSH`, `LOAD` or `cache-args`. ### Release note ```release-note fix(build): `LOAD` and `PUSH` set in the environment are now honoured, `LOAD=1` no longer pushes unless `PUSH=1` is also set, and the Talos installer is no longer published when `PUSH=0`. ```
2 parents 2af8b0c + c603327 commit 2ddd55c

4 files changed

Lines changed: 56 additions & 22 deletions

File tree

‎hack/common-envs.mk‎

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ CACHE_REGISTRY ?= $(REGISTRY)
1818
# that does not collide between concurrent builds:
1919
# * pull-requests.yaml -> pr-<N>-<sha>
2020
# * tags.yaml -> <ref_name> (e.g. v1.5.0)
21-
# * local -> dev (with PUSH=0 by default)
21+
# * local -> dev (pushes unless PUSH=0; LOAD=1 defaults PUSH to 0)
2222
IMAGE_TAG ?= dev
2323

2424
# Opt-in extra tags. Workflows set these explicitly; defaults are off so a
@@ -34,8 +34,11 @@ PUBLISH_FLOATING ?= 0
3434
# never race on the cache manifest (the 409 collisions PR #2711 fixed for tags).
3535
WRITE_CACHE ?= 0
3636

37-
PUSH := 1
38-
LOAD := 0
37+
# Environment and command line both override these, and an empty value from the
38+
# environment counts as unset. LOAD=1 loads into the local docker store, so PUSH
39+
# follows it to 0 unless PUSH is set too.
40+
LOAD := $(or $(strip $(LOAD)),0)
41+
PUSH := $(or $(strip $(PUSH)),$(if $(filter 1,$(LOAD)),0,1))
3942

4043
# OCI_EXPORT_DIR: when set, build every image to a per-image OCI archive
4144
# (<dir>/<name>.oci.tar) instead of pushing or loading it. Fork PRs use this:
@@ -116,28 +119,27 @@ oci-output = $(if $(strip $(OCI_EXPORT_DIR)), --output type=oci$(comma)dest=$(OC
116119
# line. hack/stitch-multiarch.sh moves exactly those tags onto the multi-arch
117120
# index; no ref in the tree names a versioned tag, so this is how it learns
118121
# one was pushed. The append happens when the recipe is expanded, `make -n`
119-
# included, and is skipped under OCI_EXPORT_DIR, which pushes nothing.
122+
# included, and is skipped when the build pushes nothing (PUSH other than 1, or
123+
# OCI_EXPORT_DIR).
120124
image-tag-refs = $(REGISTRY)/$(1):$(IMAGE_TAG)$(if $(strip $(OCI_EXPORT_DIR)),,$(if $(filter 1,$(PUBLISH_VERSIONED)),$(if $(filter-out $(IMAGE_TAG),$(strip $(2))), $(REGISTRY)/$(1):$(strip $(2))))$(if $(filter 1,$(PUBLISH_FLOATING)), $(REGISTRY)/$(1):latest))
121125

122126
define image-tags
123-
$(foreach r,$(call image-tag-refs,$(1),$(2)),--tag $(r))$(call oci-output,$(1))$(if $(and $(strip $(PUSHED_TAGS_LOG)),$(if $(strip $(OCI_EXPORT_DIR)),,1)),$(shell printf '%s\n' $(call image-tag-refs,$(1),$(2)) >>'$(PUSHED_TAGS_LOG)'))
127+
$(foreach r,$(call image-tag-refs,$(1),$(2)),--tag $(r))$(call oci-output,$(1))$(if $(and $(strip $(PUSHED_TAGS_LOG)),$(filter 1,$(PUSH)),$(if $(strip $(OCI_EXPORT_DIR)),,1)),$(shell printf '%s\n' $(call image-tag-refs,$(1),$(2)) >>'$(PUSHED_TAGS_LOG)'))
124128
endef
125129

126-
# cache-args <image-name> [<cache-tag>]
130+
# cache-args <image-name>
127131
# Expands to buildx cache flags for one image:
128132
# --cache-from is always emitted (a missing cache 404s harmlessly -> cold build)
129133
# --cache-to is emitted only when WRITE_CACHE=1 (main/release), writing mode=max
130134
# so ALL stages are cached -- including the multistage `builder` layers that
131135
# `--cache-to type=inline` could never export. oci-mediatypes + image-manifest
132136
# keep the cache manifest portable across registries (OCIR/ghcr/ECR).
133-
# <cache-tag> defaults to $(CACHE_TAG); pass an explicit tag for images that build
134-
# a distinct artifact per loop iteration (e.g. ubuntu-container-disk per k8s ver).
135137
# CACHE_TAG lets a build for another platform keep a cache of its own
136138
# (buildcache-arm64) instead of overwriting the amd64 one with every write.
137139
# $(comma) (defined above) escapes the literal commas in the --cache-to value so
138140
# make does not mis-parse them as $(if ...) argument separators.
139141
CACHE_TAG ?= buildcache
140-
cache-args = --cache-from type=registry,ref=$(CACHE_REGISTRY)/$(1):$(if $(2),$(2),$(CACHE_TAG))$(if $(filter 1,$(WRITE_CACHE)), --cache-to type=registry$(comma)ref=$(CACHE_REGISTRY)/$(1):$(if $(2),$(2),$(CACHE_TAG))$(comma)mode=max$(comma)oci-mediatypes=true$(comma)image-manifest=true)
142+
cache-args = --cache-from type=registry,ref=$(CACHE_REGISTRY)/$(1):$(CACHE_TAG)$(if $(filter 1,$(WRITE_CACHE)), --cache-to type=registry$(comma)ref=$(CACHE_REGISTRY)/$(1):$(CACHE_TAG)$(comma)mode=max$(comma)oci-mediatypes=true$(comma)image-manifest=true)
141143

142144
ifeq ($(COZYSTACK_VERSION),)
143145
$(shell git remote add upstream https://github.com/cozystack/cozystack.git || true)

‎hack/common-envs_test.bats‎

Lines changed: 37 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -121,7 +121,35 @@
121121
if echo "$out" | grep -q -- '--platform'; then echo "FAIL: LOAD=1 passes a platform, so buildx builds an index it cannot load"; false; fi
122122
}
123123

124-
@test "CACHE_TAG moves the default cache ref and an explicit cache tag still wins" {
124+
@test "LOAD and PUSH from the environment reach buildx, and LOAD=1 alone does not push" {
125+
out=$(LOAD=1 PUSH=0 make -n -C packages/system/cozystack-controller image IMAGE_TAG=pr-1-abc COZYSTACK_VERSION=0 BUILDER=b)
126+
echo "$out" | grep -q -- '--push=0 --load=1' || { echo "FAIL: environment LOAD=1 PUSH=0 ignored"; false; }
127+
if echo "$out" | grep -q -- '--platform'; then echo "FAIL: environment LOAD=1 still builds a multi-arch index"; false; fi
128+
out=$(LOAD=1 make -n -C packages/system/cozystack-controller image IMAGE_TAG=pr-1-abc COZYSTACK_VERSION=0 BUILDER=b)
129+
echo "$out" | grep -q -- '--push=0 --load=1' || { echo "FAIL: LOAD=1 alone pushes"; false; }
130+
# An explicit PUSH still wins over the LOAD-derived default.
131+
out=$(LOAD=1 PUSH=1 make -n -C packages/system/cozystack-controller image IMAGE_TAG=pr-1-abc COZYSTACK_VERSION=0 BUILDER=b)
132+
echo "$out" | grep -q -- '--push=1 --load=1'
133+
# The fork export must keep beating an environment PUSH=1.
134+
out=$(PUSH=1 make -n -C packages/system/cozystack-controller image IMAGE_TAG=pr-1-abc COZYSTACK_VERSION=0 BUILDER=b OCI_EXPORT_DIR=/tmp/ocitest)
135+
echo "$out" | grep -q -- '--push=0 --load=0'
136+
}
137+
138+
@test "an empty PUSH or LOAD in the environment falls back to the default" {
139+
out=$(PUSH= LOAD= make -n -C packages/system/cozystack-controller image IMAGE_TAG=pr-1-abc COZYSTACK_VERSION=0 BUILDER=b)
140+
echo "$out" | grep -q -- '--push=1 --load=0' || { echo "FAIL: empty PUSH/LOAD reach buildx as empty flags"; false; }
141+
out=$(PUSH= LOAD=1 make -n -C packages/system/cozystack-controller image IMAGE_TAG=pr-1-abc COZYSTACK_VERSION=0 BUILDER=b)
142+
echo "$out" | grep -q -- '--push=0 --load=1'
143+
}
144+
145+
@test "talos publishes nothing to the registry with PUSH=0" {
146+
out=$(make -n -B -C packages/core/talos image-talos PUSH=0 PUBLISH_VERSIONED=1 PUBLISH_FLOATING=1 IMAGE_TAG=pr-1-abc COZYSTACK_VERSION=0)
147+
echo "$out" | grep -q 'docker://'
148+
pushes=$(echo "$out" | grep 'docker://' | grep -vF '[ "0" = "1" ]' || true)
149+
if [ -n "$pushes" ]; then echo "FAIL: talos pushes with PUSH=0: $pushes"; false; fi
150+
}
151+
152+
@test "CACHE_TAG moves the cache ref of every image" {
125153
# The arm64 leg writes its own mode=max cache. Sharing the amd64 ref would
126154
# make every write from one leg evict the other's layers.
127155
out=$(make -n -C packages/system/cozystack-controller image IMAGE_TAG=pr-1-abc COZYSTACK_VERSION=0 BUILDER=b WRITE_CACHE=1)
@@ -130,14 +158,13 @@
130158
out=$(make -n -C packages/system/cozystack-controller image IMAGE_TAG=pr-1-abc COZYSTACK_VERSION=0 BUILDER=b WRITE_CACHE=1 CACHE_TAG=buildcache-arm64)
131159
echo "$out" | grep -q -- '--cache-from type=registry,ref=[^ ]*/cozystack-controller:buildcache-arm64 '
132160
echo "$out" | grep -q -- '--cache-to type=registry,ref=[^ ]*/cozystack-controller:buildcache-arm64,'
133-
# The second argument of cache-args names a per-iteration cache; CACHE_TAG
134-
# must not override it.
161+
# No per-call tag may bypass CACHE_TAG, or both legs write one cache ref.
135162
tmp=$(mktemp -d)
136163
printf 'include %s/hack/common-envs.mk\nprobe:\n\t@echo $(call cache-args,img,percall)\n' "$(pwd)" > "$tmp/Makefile"
137164
out=$(make -s -C "$tmp" probe COZYSTACK_VERSION=0 WRITE_CACHE=1 CACHE_TAG=buildcache-arm64)
138165
rm -rf "$tmp"
139-
echo "$out" | grep -q -- '--cache-from type=registry,ref=[^ ]*/img:percall '
140-
echo "$out" | grep -q -- '--cache-to type=registry,ref=[^ ]*/img:percall,'
166+
echo "$out" | grep -q -- '--cache-from type=registry,ref=[^ ]*/img:buildcache-arm64 '
167+
echo "$out" | grep -q -- '--cache-to type=registry,ref=[^ ]*/img:buildcache-arm64,'
141168
}
142169

143170
@test "the kamaji provider image is pushed under the build's IMAGE_TAG like every other image" {
@@ -159,7 +186,7 @@
159186
if echo "$tags" | grep -q -- '-$'; then echo "FAIL: a tag ends in '-': $tags"; false; fi
160187
}
161188

162-
@test "image-tags records every tag it pushes in PUSHED_TAGS_LOG, and nothing when unset or exporting" {
189+
@test "image-tags records every tag it pushes in PUSHED_TAGS_LOG, and nothing when unset, exporting or not pushing" {
163190
# The stitch moves exactly these tags onto the multi-arch index. A component
164191
# version pushed under PUBLISH_VERSIONED=1 is named by no ref in the tree, so
165192
# a tag missing here would stay on the amd64-only image.
@@ -174,6 +201,10 @@
174201
make -n -C packages/system/cozystack-controller image IMAGE_TAG=pr-1-abc BUILDER=b \
175202
OCI_EXPORT_DIR=/tmp/ocitest PUSHED_TAGS_LOG="$tmp/export-log" >/dev/null
176203
[ ! -e "$tmp/export-log" ]
204+
# A build that does not push must not hand its tags to the stitch.
205+
LOAD=1 make -n -C packages/system/cozystack-controller image IMAGE_TAG=pr-1-abc BUILDER=b \
206+
PUSHED_TAGS_LOG="$tmp/load-log" >/dev/null
207+
if [ -s "$tmp/load-log" ]; then echo "FAIL: LOAD=1 logs tags it never pushed: $(cat "$tmp/load-log")"; false; fi
177208
rm -rf "$tmp"
178209
}
179210

‎hack/stitch-multiarch.sh‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,10 +21,11 @@
2121
# if it points at A, so none is left naming the amd64 half alone: <tag>,
2222
# the tag <t> the ref carries, and every <repo>:<tag> line in the file
2323
# PUSHED_TAGS_LOG names. image-tags in hack/common-envs.mk writes that
24-
# log when the variable is set, and it is how the component version
25-
# pushed beside <image-tag> under PUBLISH_VERSIONED=1 is found, since
26-
# most refs name only <image-tag>. These are the only tags read; a tag of
27-
# another build that sits on the same bytes is not this build's to move.
24+
# log when the variable is set and PUSH=1, and it is how the component
25+
# version pushed beside <image-tag> under PUBLISH_VERSIONED=1 is found,
26+
# since most refs name only <image-tag>. These are the only tags read;
27+
# a tag of another build that sits on the same bytes is not this build's
28+
# to move.
2829
# PUSHED_TAGS_LOG set to a file that does not exist fails the run.
2930
# 3. A is replaced with the index digest in the files hack/lib/image-refs.sh
3031
# enumerates, and nowhere else.

‎packages/core/talos/Makefile‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,9 @@ image-talos: talos-installer
3030
for a in $(TALOS_ARCHES); do skopeo copy docker-archive:$(ASSETS)/installer-$$a.tar oci:$(ASSETS)/installer-oci:$$a; done; \
3131
hack/installer-index.sh $(ASSETS)/installer-oci $(TALOS_ARCHES); \
3232
SRC=oci:$(ASSETS)/installer-oci:index; \
33-
$(if $(strip $(OCI_EXPORT_DIR)),mkdir -p "$(OCI_EXPORT_DIR)"; skopeo copy --all "$$SRC" "oci-archive:$(OCI_EXPORT_DIR)/talos.oci.tar:$(IMAGE_TAG)",skopeo copy --all "$$SRC" docker://$(REGISTRY)/talos:$(IMAGE_TAG)); \
34-
if [ -z "$(OCI_EXPORT_DIR)" ] && [ "$(PUBLISH_VERSIONED)" = "1" ]; then skopeo copy --all "$$SRC" docker://$(REGISTRY)/talos:$(TALOS_VERSION); fi; \
35-
if [ -z "$(OCI_EXPORT_DIR)" ] && [ "$(PUBLISH_FLOATING)" = "1" ]; then skopeo copy --all "$$SRC" docker://$(REGISTRY)/talos:latest; fi
33+
$(if $(strip $(OCI_EXPORT_DIR)),mkdir -p "$(OCI_EXPORT_DIR)"; skopeo copy --all "$$SRC" "oci-archive:$(OCI_EXPORT_DIR)/talos.oci.tar:$(IMAGE_TAG)",if [ "$(PUSH)" = "1" ]; then skopeo copy --all "$$SRC" docker://$(REGISTRY)/talos:$(IMAGE_TAG); fi); \
34+
if [ -z "$(OCI_EXPORT_DIR)" ] && [ "$(PUSH)" = "1" ] && [ "$(PUBLISH_VERSIONED)" = "1" ]; then skopeo copy --all "$$SRC" docker://$(REGISTRY)/talos:$(TALOS_VERSION); fi; \
35+
if [ -z "$(OCI_EXPORT_DIR)" ] && [ "$(PUSH)" = "1" ] && [ "$(PUBLISH_FLOATING)" = "1" ]; then skopeo copy --all "$$SRC" docker://$(REGISTRY)/talos:latest; fi
3636
rm -rf $(ASSETS)/installer-oci
3737

3838
image-matchbox: $(call assets,kernel-%,$(BOOT_ARCHES)) $(call assets,initramfs-metal-%.xz,$(BOOT_ARCHES))

0 commit comments

Comments
 (0)