Skip to content

Commit a38843c

Browse files
test(seaweedfs): run the db-adopt migrations in the shell they ship with
Address review feedback from IvanHunters on packages/core/platform/images/migrations/migrations/43:23: The suite invoked the migrations as `sh "$MIG_DIR/<n>"`. On the CI runner /bin/sh is dash, which has no `set -o pipefail`, so both scripts aborted on their `set` line with "set: Illegal option -o pipefail" (exit 2) before doing any work, and the job went red. Production was unaffected — the migrations image is FROM alpine, so /bin/sh there is busybox ash — which is exactly why checking the runtime image missed it. Rather than change a shebang or drop pipefail, run the migrations the way run-migrations.sh does: by path, inside the image's own pinned alpine base, read out of the migrations Dockerfile so the interpreter under test cannot drift from the one they ship on. pipefail is load-bearing here — lib/cozystack-version.sh pipes the rendered manifest into `kubectl apply`, so without it a failed render stamps the version from empty input — and a shell that lacks it can prove nothing about that path. run_migration() returns the container's status explicitly: cozytest.sh's awk generator rewrites every bare `}` in column 0 into `return 0` + `}`, so a helper that falls off its own end reports success no matter what it ran, and every fail-closed assertion would have passed vacuously. 15/15 pass. Verified the interpreter under test is BusyBox v1.37.0 ash with pipefail supported, and that a fail-open mutation in lib/seaweedfs-db-adopt.sh turns the suite red. Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent 10af926 commit a38843c

1 file changed

Lines changed: 77 additions & 28 deletions

File tree

‎hack/migration-seaweedfs-db-adopt.bats‎

Lines changed: 77 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -34,19 +34,26 @@
3434
# These drive the real migration scripts end-to-end against a fake kubectl
3535
# (hack/testdata/migration-seaweedfs-db/), mocking only the cluster boundary.
3636
#
37-
# SHELL. Production runs these under /bin/sh = busybox ash (the migrations image
38-
# is FROM alpine, run-migrations.sh is #!/bin/sh), where `set -euo pipefail` and
39-
# errexit-in-function semantics differ from bash. Asserting fail-closed in a shell
40-
# that never runs it would be asserting nothing, so the scripts are invoked here
41-
# via `sh` rather than `bash`, and the fake kubectl is POSIX sh. On a developer box
42-
# /bin/sh is often bash, so this alone does not PROVE ash compatibility — that was
43-
# verified directly against the production base image, and can be re-run with:
37+
# SHELL. Production runs these under /bin/sh = busybox ash: the migrations image
38+
# is FROM alpine, and run-migrations.sh execs `/migrations/<n>` BY PATH, so the
39+
# kernel honours the `#!/bin/sh` shebang. `set -euo pipefail` and
40+
# errexit-in-function semantics differ from bash, and pipefail is load-bearing —
41+
# lib/cozystack-version.sh pipes the rendered manifest into `kubectl apply`, so
42+
# without it a failed render stamps the version from empty input. Asserting
43+
# fail-closed in a shell that never runs it would be asserting nothing.
4444
#
45-
# docker run --rm -v "$PWD/packages/core/platform/images/migrations/migrations:/m:ro" \
46-
# alpine:3.24 sh -c 'for f in /m/43 /m/53 /m/lib/*.sh; do ash -n "$f" || exit 1; done'
45+
# So these tests do not invoke the migrations through the runner's shell at all:
46+
# run_migration() executes them by path inside the image's own pinned base, read
47+
# from the migrations Dockerfile. Same base image, same interpreter, same
48+
# invocation form — production, rather than an approximation of it. This needs a
49+
# working docker; there is deliberately no host-shell fallback, because every
50+
# fallback available is a shell production never uses.
4751
#
48-
# (busybox 1.37.0 on alpine:3.24 supports `set -o pipefail`, and every script here
49-
# passes `ash -n`; the fail-closed path was exercised end-to-end under it.)
52+
# What that replaced was `sh "$MIG_DIR/<n>"`, which is neither production nor
53+
# bash: on the GitHub ubuntu runner /bin/sh is dash, which has no `set -o
54+
# pipefail` and aborts on line 1 with "set: Illegal option -o pipefail" before
55+
# the script does any work, while on a developer box /bin/sh is often bash, which
56+
# runs green and proves nothing about ash.
5057
#
5158
# cozytest.sh's awk parser recognizes only @test blocks and a bare `}` on its
5259
# own line; there is no bats `run`/`$status`/`setup`. Assertions are direct
@@ -58,13 +65,55 @@
5865
FAKEBIN="$PWD/hack/testdata/migration-seaweedfs-db"
5966
MIG_DIR="$PWD/packages/core/platform/images/migrations/migrations"
6067

61-
# prep resets PATH/env to a clean scenario. Tests set FAKE_* afterwards.
68+
# The production base image, read out of the migrations Dockerfile rather than
69+
# repeated here, so the interpreter under test cannot drift from the one the
70+
# migrations actually ship on when that pin is bumped.
71+
ALPINE=$(sed -n 's/^FROM \(alpine:[^ ]*\).*$/\1/p' \
72+
"$PWD/packages/core/platform/images/migrations/Dockerfile" | head -1)
73+
74+
# run_migration <n> -- run migrations/<n> the way run-migrations.sh does.
75+
#
76+
# By path, not `sh <file>`: that is what makes the shebang, and therefore the
77+
# interpreter, part of what is under test. The fake kubectl goes on PATH inside
78+
# the container and $WORK is bind-mounted, so $FAKE_CMDLOG is the same file the
79+
# assertions read back on the host. --network none because nothing here may
80+
# reach a real cluster; --user keeps $WORK removable by the test afterwards.
81+
#
82+
# The explicit `return` is load-bearing: cozytest.sh's awk generator rewrites
83+
# every bare `}` in column 0 into `return 0` + `}`, so a helper that falls off
84+
# its own end returns 0 no matter what it ran, and every fail-closed assertion
85+
# below would pass vacuously. Capture the status and return it by hand.
86+
run_migration() {
87+
_run_migration_rc=0
88+
docker run --rm --network none \
89+
--user "$(id -u):$(id -g)" \
90+
-v "$MIG_DIR:/migrations:ro" \
91+
-v "$FAKEBIN:/fakebin:ro" \
92+
-v "$WORK:/work" \
93+
-e PATH=/fakebin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
94+
-e FAKE_CMDLOG=/work/cmdlog \
95+
-e NAMESPACE="${NAMESPACE-}" \
96+
-e FAKE_CLUSTERS="${FAKE_CLUSTERS-}" \
97+
-e FAKE_LIST_FAIL="${FAKE_LIST_FAIL-}" \
98+
-e FAKE_GET_FAIL="${FAKE_GET_FAIL-}" \
99+
-e FAKE_ANNOTATE_FAIL="${FAKE_ANNOTATE_FAIL-}" \
100+
"$ALPINE" "/migrations/$1" || _run_migration_rc=$?
101+
return "$_run_migration_rc"
102+
}
103+
104+
# prep resets env to a clean scenario. Tests set FAKE_* afterwards.
62105
prep() {
106+
# Fail here rather than at the first docker run, so the reason is legible.
107+
docker info >/dev/null 2>&1 || {
108+
echo "docker is required: these tests run the migrations inside $ALPINE," >&2
109+
echo "the base image of the migrations image, so that they exercise busybox" >&2
110+
echo "ash — the interpreter run-migrations.sh actually gives them." >&2
111+
return 1
112+
}
63113
chmod +x "$FAKEBIN/kubectl"
64114
WORK=$(mktemp -d)
65115
export FAKE_CMDLOG="$WORK/cmdlog"
66116
: > "$FAKE_CMDLOG"
67-
export PATH="$FAKEBIN:$PATH"
68117
export NAMESPACE=cozy-system
69118
export FAKE_CLUSTERS=""
70119
unset FAKE_LIST_FAIL FAKE_GET_FAIL FAKE_ANNOTATE_FAIL || true
@@ -76,7 +125,7 @@ prep() {
76125
prep
77126
export FAKE_CLUSTERS="tenant-root seaweedfs-system -"
78127
rc=0
79-
sh "$MIG_DIR/43" >"$WORK/out" 2>&1 || rc=$?
128+
run_migration 43 >"$WORK/out" 2>&1 || rc=$?
80129
cat "$WORK/out"; cat "$FAKE_CMDLOG"
81130
[ "$rc" -eq 0 ]
82131
grep -qF -- "ANNOTATE tenant-root release-name=seaweedfs-db resource-policy=keep" "$FAKE_CMDLOG"
@@ -93,7 +142,7 @@ prep() {
93142
prep
94143
export FAKE_CLUSTERS="tenant-named foo-system -"
95144
rc=0
96-
sh "$MIG_DIR/43" >"$WORK/out" 2>&1 || rc=$?
145+
run_migration 43 >"$WORK/out" 2>&1 || rc=$?
97146
cat "$WORK/out"; cat "$FAKE_CMDLOG"
98147
[ "$rc" -eq 0 ]
99148
grep -qF -- "ANNOTATE tenant-named release-name=foo-db resource-policy=keep" "$FAKE_CMDLOG"
@@ -104,7 +153,7 @@ prep() {
104153
prep
105154
export FAKE_CLUSTERS="tenant-named foo-system -"
106155
rc=0
107-
sh "$MIG_DIR/53" >"$WORK/out" 2>&1 || rc=$?
156+
run_migration 53 >"$WORK/out" 2>&1 || rc=$?
108157
cat "$WORK/out"; cat "$FAKE_CMDLOG"
109158
[ "$rc" -eq 0 ]
110159
grep -qF -- "ANNOTATE tenant-named release-name=foo-db resource-policy=keep" "$FAKE_CMDLOG"
@@ -120,7 +169,7 @@ tenant-dsplit seaweedfs-system -
120169
tenant-l seaweedfs-system -
121170
tenant-named foo-system -"
122171
rc=0
123-
sh "$MIG_DIR/53" >"$WORK/out" 2>&1 || rc=$?
172+
run_migration 53 >"$WORK/out" 2>&1 || rc=$?
124173
cat "$WORK/out"; cat "$FAKE_CMDLOG"
125174
[ "$rc" -eq 0 ]
126175
grep -qF -- "ANNOTATE tenant-root release-name=seaweedfs-db resource-policy=keep" "$FAKE_CMDLOG"
@@ -142,7 +191,7 @@ tenant-named foo-system -"
142191
prep
143192
export FAKE_CLUSTERS="tenant-named foo-db -"
144193
rc=0
145-
sh "$MIG_DIR/53" >"$WORK/out" 2>&1 || rc=$?
194+
run_migration 53 >"$WORK/out" 2>&1 || rc=$?
146195
cat "$WORK/out"; cat "$FAKE_CMDLOG"
147196
[ "$rc" -eq 0 ]
148197
grep -qF -- "ANNOTATE tenant-named release-name=<unset> resource-policy=keep" "$FAKE_CMDLOG"
@@ -153,7 +202,7 @@ tenant-named foo-system -"
153202
prep
154203
export FAKE_CLUSTERS="tenant-fresh seaweedfs-db -"
155204
rc=0
156-
sh "$MIG_DIR/53" >"$WORK/out" 2>&1 || rc=$?
205+
run_migration 53 >"$WORK/out" 2>&1 || rc=$?
157206
cat "$WORK/out"; cat "$FAKE_CMDLOG"
158207
[ "$rc" -eq 0 ]
159208
grep -qF -- "ANNOTATE tenant-fresh release-name=<unset> resource-policy=keep" "$FAKE_CMDLOG"
@@ -165,7 +214,7 @@ tenant-named foo-system -"
165214
export FAKE_CLUSTERS="tenant-root seaweedfs-db keep
166215
tenant-named foo-db keep"
167216
rc=0
168-
sh "$MIG_DIR/53" >"$WORK/out" 2>&1 || rc=$?
217+
run_migration 53 >"$WORK/out" 2>&1 || rc=$?
169218
cat "$WORK/out"; cat "$FAKE_CMDLOG"
170219
[ "$rc" -eq 0 ]
171220
! grep -q 'ANNOTATE' "$FAKE_CMDLOG"
@@ -179,7 +228,7 @@ tenant-named foo-db keep"
179228
# an unowned SeaweedFS database must not pass silently.
180229
export FAKE_CLUSTERS="tenant-manual - -"
181230
rc=0
182-
sh "$MIG_DIR/53" >"$WORK/out" 2>&1 || rc=$?
231+
run_migration 53 >"$WORK/out" 2>&1 || rc=$?
183232
cat "$WORK/out"; cat "$FAKE_CMDLOG"
184233
[ "$rc" -eq 0 ]
185234
! grep -q 'ANNOTATE' "$FAKE_CMDLOG"
@@ -191,7 +240,7 @@ tenant-named foo-db keep"
191240
prep
192241
export FAKE_CLUSTERS="tenant-x some-other-release -"
193242
rc=0
194-
sh "$MIG_DIR/53" >"$WORK/out" 2>&1 || rc=$?
243+
run_migration 53 >"$WORK/out" 2>&1 || rc=$?
195244
cat "$WORK/out"; cat "$FAKE_CMDLOG"
196245
[ "$rc" -eq 0 ]
197246
! grep -q 'ANNOTATE' "$FAKE_CMDLOG"
@@ -206,7 +255,7 @@ tenant-named foo-db keep"
206255
# meant to protect it.
207256
export FAKE_CLUSTERS="tenant-weird -system -"
208257
rc=0
209-
sh "$MIG_DIR/53" >"$WORK/out" 2>&1 || rc=$?
258+
run_migration 53 >"$WORK/out" 2>&1 || rc=$?
210259
cat "$WORK/out"; cat "$FAKE_CMDLOG"
211260
[ "$rc" -eq 0 ]
212261
! grep -q 'ANNOTATE' "$FAKE_CMDLOG"
@@ -221,7 +270,7 @@ tenant-named foo-db keep"
221270
export FAKE_CLUSTERS="tenant-named foo-system -"
222271
export FAKE_LIST_FAIL="Error from server (Timeout): the server was unable to return a response in the time allotted"
223272
rc=0
224-
sh "$MIG_DIR/53" >"$WORK/out" 2>&1 || rc=$?
273+
run_migration 53 >"$WORK/out" 2>&1 || rc=$?
225274
cat "$WORK/out"; cat "$FAKE_CMDLOG"
226275
# Must propagate: the Job retries rather than advancing the version.
227276
[ "$rc" -ne 0 ]
@@ -235,7 +284,7 @@ tenant-named foo-db keep"
235284
export FAKE_CLUSTERS="tenant-named foo-system -"
236285
export FAKE_GET_FAIL="Error from server (Forbidden): clusters.postgresql.cnpg.io \"seaweedfs-db\" is forbidden"
237286
rc=0
238-
sh "$MIG_DIR/53" >"$WORK/out" 2>&1 || rc=$?
287+
run_migration 53 >"$WORK/out" 2>&1 || rc=$?
239288
cat "$WORK/out"; cat "$FAKE_CMDLOG"
240289
[ "$rc" -ne 0 ]
241290
grep -qF -- "cannot read the Helm owner" "$WORK/out"
@@ -248,7 +297,7 @@ tenant-named foo-db keep"
248297
export FAKE_CLUSTERS="tenant-named foo-system -"
249298
export FAKE_ANNOTATE_FAIL="Error from server (Conflict): the object has been modified"
250299
rc=0
251-
sh "$MIG_DIR/53" >"$WORK/out" 2>&1 || rc=$?
300+
run_migration 53 >"$WORK/out" 2>&1 || rc=$?
252301
cat "$WORK/out"; cat "$FAKE_CMDLOG"
253302
[ "$rc" -ne 0 ]
254303
! grep -q 'STAMP' "$FAKE_CMDLOG"
@@ -262,7 +311,7 @@ tenant-named foo-db keep"
262311
prep
263312
export FAKE_LIST_FAIL="error: the server doesn't have a resource type \"cluster\" in group \"postgresql.cnpg.io\""
264313
rc=0
265-
sh "$MIG_DIR/53" >"$WORK/out" 2>&1 || rc=$?
314+
run_migration 53 >"$WORK/out" 2>&1 || rc=$?
266315
cat "$WORK/out"; cat "$FAKE_CMDLOG"
267316
[ "$rc" -eq 0 ]
268317
grep -qF -- "resource type is not served" "$WORK/out"
@@ -275,7 +324,7 @@ tenant-named foo-db keep"
275324
prep
276325
export FAKE_CLUSTERS=""
277326
rc=0
278-
sh "$MIG_DIR/53" >"$WORK/out" 2>&1 || rc=$?
327+
run_migration 53 >"$WORK/out" 2>&1 || rc=$?
279328
cat "$WORK/out"; cat "$FAKE_CMDLOG"
280329
[ "$rc" -eq 0 ]
281330
! grep -q 'ANNOTATE' "$FAKE_CMDLOG"

0 commit comments

Comments
 (0)