Skip to content

Commit c582cc5

Browse files
committed
fix(cdi): stop make update from deleting first-party templates
The update recipe in kubevirt-cdi opened with `rm -rf templates` and then refetched one upstream manifest. The directory keeps two more files -- the upload-proxy Ingress and TLSRoute -- that are written by hand and never refetched, so the next CDI bump would drop upload-proxy exposure from the chart while `make update` still exited 0. The clean slate bought nothing either way: wget -O truncates the file it writes. The recipe now creates the directory with `mkdir -p` and leaves the rest of it alone, which is what kubevirt-operator moved to when it was last bumped. kubevirt-cdi-operator keeps its wipe. The same change there would be for symmetry rather than for a loss, and it is not free: both its patch calls resolve, so a hunk upstream has moved out from under leaves templates/cdi-operator.yaml.rej, and gawk's inplace edit leaves templates/cdi-operator.yaml.gawk.XXXXXX if it dies abnormally. Helm renders every file under templates/, so the first fails the package loudly and the second renders a duplicate object at exit 0. The wipe is what takes both. Removing it wants that recipe restructured to transform outside templates/ the way kubevirt-operator now does, which is its own change. Until then the scan below is what watches that package, with the limit the scan already documents: it reports a tracked template whose stem the Makefile never mentions, so a name that collides with something already written there, templates/ tolerations.yaml against the patches/tolerations.diff line, is exempted. It catches the loss it was written for, not every loss. The bats case scans the package Makefiles and fails when a recipe that clears its own templates/ leaves behind a tracked file it never names. The search skips each Makefile's NAME and NAMESPACE however the assignment is written, or a package called foo-bar would satisfy the check for its own templates/bar.yaml through its identity alone. It matches the variable name rather than a list of spellings, because the tree already uses three of them and the hole is the same in each. Nothing the scan reaches is exempted that way today, so the exclusion is there for the first package that would be. A removal reaching into another package is outside the scan's range, and the header records the case in the tree that this leaves uncovered. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent afc003c commit c582cc5

2 files changed

Lines changed: 358 additions & 2 deletions

File tree

‎hack/package-update-templates.bats‎

Lines changed: 348 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,348 @@
1+
#!/usr/bin/env bats
2+
# -----------------------------------------------------------------------------
3+
# Invariant for the package `update` recipes that clear templates/ wholesale.
4+
#
5+
# `update` refetches a package's vendored upstream manifests, and several
6+
# recipes open by deleting templates/ to guarantee a clean slate. That is safe
7+
# only while every file under templates/ comes back from the recipe. templates/
8+
# is an ordinary chart directory, so a package can also keep first-party
9+
# manifests there -- kubevirt-cdi keeps the CDI upload-proxy Ingress and
10+
# TLSRoute -- and the recipe knows nothing about those. The wipe takes them and
11+
# `make update` exits 0, so the deletion lands in the diff with nothing marking
12+
# it. Whether anything downstream notices depends on the package carrying a
13+
# helm-unittest suite that names the file, and where none does the loss is
14+
# silent all the way to a cluster. That is a property of the package, not of
15+
# the tree, so it is worth re-deriving rather than reading: at the time of
16+
# writing kubevirt-cdi has such a suite and a deletion there fails `make test`
17+
# as well, while three of the packages this scans declare no `test` target at
18+
# all. Adding one to any of them changes that sentence and not this one.
19+
#
20+
# The invariant: a recipe that clears templates/ must NAME every file tracked
21+
# under it. Both halves are approximations of the shell the recipe would run,
22+
# and each is argued at its helper below: the wipe is a text match over the
23+
# recipe's lines rather than a parse, and "names" means the file's basename
24+
# minus its extension appears somewhere in the Makefile -- a mention, not proof
25+
# of a write. What that catches is the shape kubevirt-cdi had, a file the
26+
# Makefile does not refer to at all: every recipe that clears its own
27+
# templates/ writes each path it puts back there, so a name appearing nowhere
28+
# in the file is a name nothing refetches. A recipe that unpacked an archive or
29+
# globbed a stream into templates/ would produce files it never names and break
30+
# that reading. cert-manager's `mv charts/.../crd-*.yaml` is that shape, and it
31+
# is out of range for the separate reason below: the directory it fills belongs
32+
# to another package.
33+
#
34+
# The tree is scanned rather than a list of packages pinned, so a package that
35+
# grows its first first-party manifest under a wiping recipe of its own is
36+
# caught by the commit that adds it. The packages that clear their own
37+
# templates/ today refetch all of it, which is what keeps this green;
38+
# kubevirt-cdi stopped clearing it, which is what this file was written for.
39+
# kubevirt-cdi-operator still clears its own, and is scanned like the rest.
40+
#
41+
# What that leaves uncovered is a recipe clearing SOMEONE ELSE'S templates/, and
42+
# the tree has one: packages/system/cert-manager/Makefile runs
43+
# `rm -rf ../cert-manager-crds/templates/*` and moves the CRDs it pulled into
44+
# that directory, while cert-manager-crds declares no update target at all. A
45+
# first-party manifest added there is deleted by the next `make update` in
46+
# cert-manager and nothing below reports it. Covering that means resolving the
47+
# removal's target directory rather than reading the wiping package's own, which
48+
# is a second helper; the pattern below is deliberately not widened to accept a
49+
# path prefix, because that alone would fire on cert-manager and then list the
50+
# templates of a directory it does not have -- the removal names the sibling's
51+
# path, so the lookup has to follow it.
52+
#
53+
# The scan alone would also be green against a detector that finds nothing, so
54+
# the control below runs the same two helpers over a fixture that does hold an
55+
# unrefetched file, and over that fixture with the removal taken out.
56+
#
57+
# Constraints from cozytest.sh: it rewrites the closing brace of every function
58+
# in this file into `return 0`, so a helper cannot report through its exit
59+
# status -- both report through stdout -- and it injects `set -e`, so statuses
60+
# are captured explicitly.
61+
#
62+
# Run with: hack/cozytest.sh hack/package-update-templates.bats
63+
# -----------------------------------------------------------------------------
64+
65+
# Prints the recipe line that clears templates/ wholesale, for a package
66+
# Makefile whose `update` target has one. Silent otherwise, including for a
67+
# Makefile with no `update` target at all.
68+
#
69+
# The recipe is read as the run of tab-indented lines after `update:`. make also
70+
# ignores blank lines inside a recipe and this stops at the first one, so such a
71+
# recipe would be read short; none in the tree is written that way.
72+
#
73+
# The match accepts `rm -rf`, `rm -fr` and `rm -r` against `templates` and
74+
# `./templates`, bare, with a trailing slash, or with a `/*` glob -- the three
75+
# spellings that empty the directory. It deliberately does not accept
76+
# `rm -rf templates/one-file.yml`, which takes a single file the recipe goes on
77+
# to refetch (rabbitmq-operator), nor a narrower glob such as
78+
# `rm -rf templates/*.yaml`, which takes a subset the same way.
79+
#
80+
# It is a text match and not a shell parse, so a quoted `rm -rf "templates"`, a
81+
# removal reached through a variable, a `find -delete` and a `cd` into another
82+
# directory first are all invisible to it, as is a path-prefixed target -- the
83+
# cert-manager case in the header is one, and the prefix is the only thing
84+
# keeping it out: the `/*` it ends with is accepted. The enumeration is only as
85+
# complete as the last reader made it, so extend it in the same edit as the
86+
# pattern.
87+
wipe_command() {
88+
awk '/^update:/ { r = 1; next } r && /^\t/ { print; next } r { exit }' "$1" |
89+
grep -E 'rm[[:space:]]+-(rf|fr|r)[[:space:]]+(\./)?templates(/\*?)?([[:space:]]|;|&|$)' || true
90+
}
91+
92+
# Prints every file tracked under a package's templates/ that its Makefile
93+
# never mentions, one path per line. Whether that matters is the caller's call:
94+
# it matters for a package whose recipe clears the directory, and for no other.
95+
#
96+
# The listing is the index, so only a tracked file is at stake -- it is the one
97+
# the next `make update` deletes out from under a reviewer who has no reason to
98+
# look. Reading the working tree instead would fail a package over the
99+
# developer's own litter: an untracked .orig from a hand-applied patch, a
100+
# half-written template, a .DS_Store. A guard that goes red on scratch files is
101+
# a guard that gets switched off.
102+
#
103+
# The extension is stripped before matching, because a recipe may assemble the
104+
# filename from a variable holding the bare stem -- etcd-operator-crds writes
105+
# templates/$$c.yaml from a CRDS list of names without one. A basename whose
106+
# only dot is its first character keeps the whole name instead: stripping there
107+
# leaves the empty string, which grep matches against every Makefile, and the
108+
# file would pass without being looked at. The search covers the whole Makefile
109+
# rather than the recipe alone for the same reason as the stem: the variable is
110+
# assigned above the target. Both make the match generous, so a stem that
111+
# appears for an unrelated reason satisfies it; this reports the files nothing in
112+
# the Makefile refers to, not every file the recipe fails to write.
113+
#
114+
# Comment lines are dropped before the search, so a filename written into one is
115+
# not a mention: a rationale comment above a recipe, naming the very file it is
116+
# about, would otherwise exempt that package from its own guard. Comments
117+
# naming a template are not rare in the tree, so this is the difference between
118+
# a package being read and a package being taken at its word.
119+
#
120+
# The NAME and NAMESPACE assignments go the same way. They are the package's
121+
# identity rather than a reference to any file, and a package called foo-bar
122+
# whose recipe writes templates/bar.yaml has that file's stem inside its own
123+
# name: left in, they let a package vouch for its own templates without any
124+
# recipe touching them. The match is on the variable name rather than on one
125+
# way of writing the assignment, because the hole does not care how it was
126+
# spelled: `export` is optional, whitespace either side of the operator is
127+
# optional, and make's assignment forms (`=`, `:=`, `::=`, `?=`, `+=`) are all
128+
# accepted. The tree uses at least three of these -- `export NAME=`, the bare
129+
# `NAME=`, and `NAME := ` in packages/core/testing -- and pinning the list of
130+
# spellings is what this deliberately avoids. Two names are named rather than a
131+
# `[A-Z_]+=` shape for the opposite reason: a variable holding content is what
132+
# the whole-file search is there to find, and etcd-operator-crds names its
133+
# templates through nothing but its CRDS list, so a shape that broad would be
134+
# one unspaced assignment away from blinding the scan to five CRDs. That list
135+
# is written spaced, which is why the broader pattern would not reach it today
136+
# and why no case below can pin this choice; two names cannot reach it at any
137+
# spacing. Precautionary like the comment strip above: no package the scan
138+
# reaches is exempted this way today, and the line is what keeps the first one
139+
# that tries from being.
140+
unnamed_templates() {
141+
_mk=$1
142+
_dir=$(dirname "$_mk")
143+
_code=$(grep -vE '^([[:space:]]*@?#|(export[[:space:]]+)?(NAME|NAMESPACE)[[:space:]]*[:?+]*=)' "$_mk") || true
144+
git -C "$_dir" ls-files -- templates | while IFS= read -r _f; do
145+
_base=${_f##*/}
146+
_stem=${_base%.*}
147+
[ -n "$_stem" ] || _stem=$_base
148+
printf '%s\n' "$_code" | grep -Fq -- "$_stem" || printf '%s\n' "$_dir/$_f"
149+
done
150+
}
151+
152+
@test "no package update recipe clears templates/ over a file it never refetches" {
153+
examined=0
154+
offenders=""
155+
156+
for mk in packages/*/*/Makefile; do
157+
[ -f "$mk" ] || continue
158+
examined=$((examined + 1))
159+
[ -n "$(wipe_command "$mk")" ] || continue
160+
found=$(unnamed_templates "$mk")
161+
if [ -n "$found" ]; then
162+
offenders="$offenders
163+
$mk clears templates/ but names none of:
164+
$found"
165+
fi
166+
done
167+
168+
# A glob that matched nothing -- packages/ moved, or the suite run from
169+
# somewhere other than the repository root -- would otherwise report success
170+
# having looked at no Makefile at all.
171+
if [ "$examined" -eq 0 ]; then
172+
echo "no packages/*/*/Makefile found: run this from the repository root" >&2
173+
exit 1
174+
fi
175+
176+
# The index is where the scan reads a package's templates from, so an answer
177+
# of nothing here means it could not have reported anything either.
178+
if [ -z "$(git ls-files -- packages | head -1)" ]; then
179+
echo "git ls-files reported no tracked file under packages/: this test" >&2
180+
echo "cannot see a template directory, so it measured nothing" >&2
181+
exit 1
182+
fi
183+
184+
if [ -n "$offenders" ]; then
185+
echo "a package update recipe deletes templates/ over a file it does not refetch:" >&2
186+
echo "$offenders" >&2
187+
echo "" >&2
188+
echo "templates/ holds first-party manifests alongside the vendored ones and" >&2
189+
echo "'rm -rf templates' takes both, silently: the next 'make update' drops the" >&2
190+
echo "first-party ones and still exits 0. Either keep 'mkdir -p templates'" >&2
191+
echo "without the removal -- wget -O and kustomize > overwrite what they fetch" >&2
192+
echo "anyway -- or write the local file from the recipe too, where the clean" >&2
193+
echo "slate is load-bearing (etcd-operator-crds stages into a tmpdir first)." >&2
194+
exit 1
195+
fi
196+
}
197+
198+
@test "the scan reports a template file that a wiping recipe does not name" {
199+
# Negative control. The test above passes on a tree with no offender and on
200+
# a detector that cannot see one; this separates the two by handing the same
201+
# helpers a package that is broken in exactly the way kubevirt-cdi was.
202+
tmp=$(mktemp -d)
203+
mkdir -p "$tmp/pkg/templates"
204+
: > "$tmp/pkg/templates/fetched.yaml"
205+
: > "$tmp/pkg/templates/local-extra.yaml"
206+
# A tracked dotfile. Stripping its extension leaves nothing and an empty
207+
# pattern matches every Makefile, so without the guard in the helper this
208+
# file is skipped in silence rather than reported.
209+
: > "$tmp/pkg/templates/.gitkeep"
210+
# Two files whose stems sit inside the package's own identity, the shape
211+
# every `foo-bar` package writing templates/bar.yaml has. One is covered by
212+
# NAME and one by NAMESPACE, and the Makefile below writes the first as
213+
# `export NAME=` and the second as `NAMESPACE := `, so between them the
214+
# export prefix, the bare form, spacing around the operator and the `:=`
215+
# spelling are all exercised. Left in the search, each of these matches
216+
# with nothing writing it.
217+
: > "$tmp/pkg/templates/name-echo.yaml"
218+
: > "$tmp/pkg/templates/bare-echo.yaml"
219+
# The comment names the file the recipe does not write, which is what a
220+
# rationale comment above a real recipe looks like. It must not count as the
221+
# recipe naming it.
222+
printf 'export NAME=demo-name-echo\nNAMESPACE := cozy-bare-echo\n# local-extra.yaml is written by hand\nupdate:\n\trm -rf templates\n\tmkdir templates\n\twget -O templates/fetched.yaml https://example.invalid/f.yaml\n' \
223+
> "$tmp/pkg/Makefile"
224+
# An index is what the helper reads, so the fixture needs one. Staged rather
225+
# than committed: nothing here needs a commit object, and a commit would want
226+
# an identity and a signature from whoever runs the suite.
227+
git -C "$tmp" init --quiet
228+
git -C "$tmp" add --all
229+
230+
# Untracked litter, written after the staging so it stays untracked. Neither
231+
# file is at stake and neither may be reported.
232+
: > "$tmp/pkg/templates/scratch.yaml"
233+
: > "$tmp/pkg/templates/.DS_Store"
234+
235+
if [ -z "$(wipe_command "$tmp/pkg/Makefile")" ]; then
236+
echo "the matcher did not see 'rm -rf templates' in the fixture recipe" >&2
237+
rm -rf "$tmp"
238+
exit 1
239+
fi
240+
241+
# Exactly the tracked files the recipe does not name. Reporting the fetched
242+
# one too would fail every wiping package in the tree, reporting none of them
243+
# would pass all of them, and reporting the litter would fail a package over
244+
# a file nobody is going to miss.
245+
expected=$(printf '%s\n%s\n%s\n%s' "$tmp/pkg/templates/.gitkeep" "$tmp/pkg/templates/bare-echo.yaml" "$tmp/pkg/templates/local-extra.yaml" "$tmp/pkg/templates/name-echo.yaml")
246+
found=$(unnamed_templates "$tmp/pkg/Makefile")
247+
if [ "$found" != "$expected" ]; then
248+
echo "expected only the tracked unrefetched files to be reported:" >&2
249+
printf '%s\n' "$expected" >&2
250+
echo "got:" >&2
251+
printf '%s\n' "$found" >&2
252+
rm -rf "$tmp"
253+
exit 1
254+
fi
255+
256+
# The trailing-slash spelling removes exactly as much, so it has to be seen
257+
# as well. Spelled with the leading ./ at the same time, which is the other
258+
# optional half of the pattern; the fixture above carries neither.
259+
printf 'update:\n\trm -rf ./templates/\n\twget -O templates/fetched.yaml https://example.invalid/f.yaml\n' \
260+
> "$tmp/pkg/Makefile"
261+
if [ -z "$(wipe_command "$tmp/pkg/Makefile")" ]; then
262+
echo "the matcher missed 'rm -rf ./templates/', which clears the directory" >&2
263+
rm -rf "$tmp"
264+
exit 1
265+
fi
266+
267+
# And the glob spelling, which empties the directory without naming it as
268+
# the argument. It is what cert-manager's recipe writes, so it is the line a
269+
# package copies when it wants to clear its own templates.
270+
printf 'update:\n\trm -rf templates/*\n\twget -O templates/fetched.yaml https://example.invalid/f.yaml\n' \
271+
> "$tmp/pkg/Makefile"
272+
if [ -z "$(wipe_command "$tmp/pkg/Makefile")" ]; then
273+
echo "the matcher missed 'rm -rf templates/*', which empties the directory" >&2
274+
rm -rf "$tmp"
275+
exit 1
276+
fi
277+
278+
# The two flag spellings the header claims to accept. Nothing in the tree
279+
# writes either today, so they are precautionary in the same way the
280+
# identity strip is -- and the file's standard is that a precautionary
281+
# branch gets pinned, or the claim in the header is prose nothing checks.
282+
for flags in -fr -r; do
283+
printf 'update:\n\trm %s templates\n\twget -O templates/fetched.yaml https://example.invalid/f.yaml\n' \
284+
"$flags" > "$tmp/pkg/Makefile"
285+
if [ -z "$(wipe_command "$tmp/pkg/Makefile")" ]; then
286+
echo "the matcher missed 'rm $flags templates', which the header says it accepts" >&2
287+
rm -rf "$tmp"
288+
exit 1
289+
fi
290+
done
291+
292+
# Scoping. The helper claims to read only the `update` recipe, and to stay
293+
# silent for a Makefile that has no `update` target at all. The tree cannot
294+
# test either, because every wipe in it already sits inside `update`, so
295+
# without these two a matcher that searched the whole file would look
296+
# identical. The consequence of losing this is a false positive that never
297+
# clears: a package growing `clean: rm -rf templates` reported forever.
298+
printf 'clean:\n\trm -rf templates\nupdate:\n\tmkdir -p templates\n\twget -O templates/fetched.yaml https://example.invalid/f.yaml\n' \
299+
> "$tmp/pkg/Makefile"
300+
if [ -n "$(wipe_command "$tmp/pkg/Makefile")" ]; then
301+
echo "the matcher read a removal from a target other than update" >&2
302+
rm -rf "$tmp"
303+
exit 1
304+
fi
305+
306+
printf 'clean:\n\trm -rf templates\n' > "$tmp/pkg/Makefile"
307+
if [ -n "$(wipe_command "$tmp/pkg/Makefile")" ]; then
308+
echo "the matcher reported on a Makefile with no update target" >&2
309+
rm -rf "$tmp"
310+
exit 1
311+
fi
312+
313+
# The other side of that boundary: removing one file inside templates/ is
314+
# rabbitmq-operator's shape, it takes only what the recipe refetches, and a
315+
# pattern loose enough to call it a wipe would report that package forever.
316+
printf 'update:\n\trm -rf templates/fetched.yaml\n\twget -O templates/fetched.yaml https://example.invalid/f.yaml\n' \
317+
> "$tmp/pkg/Makefile"
318+
if [ -n "$(wipe_command "$tmp/pkg/Makefile")" ]; then
319+
echo "the matcher read a single-file removal as clearing the directory" >&2
320+
rm -rf "$tmp"
321+
exit 1
322+
fi
323+
324+
# The narrower glob is the other rejection the header claims and the last
325+
# of that class without a case. It takes a subset the recipe refetches, so
326+
# calling it a wipe would report the package forever -- and it
327+
# discriminates: widening the tail to accept any glob, the obvious "also
328+
# match globs" edit, slips past the single-file case above but not this one.
329+
printf 'update:\n\trm -rf templates/*.yaml\n\twget -O templates/fetched.yaml https://example.invalid/f.yaml\n' \
330+
> "$tmp/pkg/Makefile"
331+
if [ -n "$(wipe_command "$tmp/pkg/Makefile")" ]; then
332+
echo "the matcher read a narrower glob as clearing the directory" >&2
333+
rm -rf "$tmp"
334+
exit 1
335+
fi
336+
337+
# And the shape kubevirt-cdi moved to switches the check off, so the scan
338+
# is measuring the removal rather than the package.
339+
printf 'update:\n\tmkdir -p templates\n\twget -O templates/fetched.yaml https://example.invalid/f.yaml\n' \
340+
> "$tmp/pkg/Makefile"
341+
if [ -n "$(wipe_command "$tmp/pkg/Makefile")" ]; then
342+
echo "the matcher fired on a recipe that only creates the directory" >&2
343+
rm -rf "$tmp"
344+
exit 1
345+
fi
346+
347+
rm -rf "$tmp"
348+
}

0 commit comments

Comments
 (0)