Skip to content

Commit 062207f

Browse files
myasnikovdaniilmyasnikovdaniil
committed
test(hack): run the unit bats suite under bats(1)
hack/cozytest.sh serves two populations that want opposite things from a runner: 32 hermetic unit tests of hack/*.sh, and 3 live-cluster imperative tests. This moves the unit half to real bats and leaves the live half where it is; the hack/e2e-*.bats filter already in BATS_UNIT_FILES is what keeps them apart. The unit files need no edits. They pass under vanilla bats 1.13 unchanged because their authors wrote to the intersection of both runners -- no run, no $status, no skip, repo-root-relative paths. Measured on this tree: 328 tests, 0 failures, 26s under `bats -j 8` against cozytest's 63s serial. What this buys, none of which cozytest offers: --filter-status failed to rerun only the previous failures, JUnit so CI annotates the failing test instead of requiring a log read, -j, and deletion of a bespoke awk parser whose rewrite rules void helper exit codes, corrupt heredoc fixtures on a column-0 brace, and collide test names after sanitization. Two consequences worth naming, neither covered by the issue: bats is a new dependency. cozytest is pure bash+awk, so no runner image or contributor machine has ever needed anything else. The checks job installs it from a pinned tag rather than the distro package, whose version varies by base image, and the make target says so plainly when it is absent. GNU parallel stays optional -- BATS_JOBS resolves to 1 without it, since `bats -j` exits non-zero rather than degrading. The pre-commit lane is the second consumer and does NOT get bats. That job lints changed YAML and Markdown on a runner with no bats, and the new hook's path filter covers .github/workflows/, so a workflow edit would drag the whole 328-test shell suite into a lint lane and duplicate the checks job that already runs `make unit-tests`. It is skipped there by id, keeping the hook a local-developer gate. The tested shell changes. cozytest is #!/bin/sh, which is dash on the CI runner, and bats runs bash. Seven POSIX scripts that tests source rather than execute -- seaweedfs-naming-audit.sh, promote-rewrite-tags.sh, lib/image-refs.sh, the three e2e-capture scripts and cozystack-version.sh -- therefore lose a runtime dash check they had by accident. Four shell=bash files under e2e-chainsaw/_lib gain correct fidelity for the same reason. The static replacement is the shellcheck gate in item 5 of #3453, which does not exist yet. set -u is not restored here. bats enforces -e but not -u, and there is no runner-level way to inject it: --setup-suite-file does not reach test files, and SHELLOPTS=nounset is exported, so it leaks into the scripts under test and drops two tests. Restoring it needs a per-file load and is the layer above this one. Refs: #3453 Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent 2823b76 commit 062207f

4 files changed

Lines changed: 69 additions & 7 deletions

File tree

‎.github/workflows/pre-commit.yml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,14 @@ jobs:
5656
curl -sSL https://github.com/cozystack/cozyvalues-gen/releases/download/v1.6.0/cozyvalues-gen-linux-amd64.tar.gz | tar -xzvf- -C /usr/local/bin/ cozyvalues-gen
5757
5858
- name: Run pre-commit hooks
59+
# bats-unit-tests is a local-developer gate, skipped here. This job
60+
# lints changed YAML/Markdown, and the hook's path filter covers
61+
# .github/workflows/, so a workflow edit would otherwise drag the whole
62+
# 328-test shell suite into a lint lane -- on a runner with no bats,
63+
# and duplicating the `checks` job in pull-requests.yaml that already
64+
# runs `make unit-tests`.
65+
env:
66+
SKIP: bats-unit-tests
5967
run: |
6068
git fetch origin main || git fetch origin master
6169
base_commit=$(git rev-parse --verify origin/main || git rev-parse --verify origin/master || echo "")

‎.github/workflows/pull-requests.yaml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,11 +107,32 @@ jobs:
107107
# Ephemeral runners don't carry the helm-unittest plugin the persistent
108108
# self-hosted runner accreted. Install if absent (idempotent).
109109
- name: Set up test toolchain
110+
env:
111+
BATS_VERSION: v1.13.0
110112
run: |
113+
set -euo pipefail
111114
helm plugin list 2>/dev/null | grep -q unittest \
112115
|| helm plugin install https://github.com/helm-unittest/helm-unittest
116+
# The hack/*.bats unit suite moved from hack/cozytest.sh (pure bash,
117+
# no dependency) to bats(1) in #3453, so the runner image now needs
118+
# bats. Installed from a pinned tag rather than the distro package,
119+
# whose version varies by base image. GNU parallel is optional --
120+
# the Makefile falls back to a serial run when it is absent.
121+
if ! command -v bats >/dev/null 2>&1; then
122+
git clone --depth 1 --branch "$BATS_VERSION" \
123+
https://github.com/bats-core/bats-core.git /tmp/bats-core
124+
sudo /tmp/bats-core/install.sh /usr/local
125+
fi
126+
bats --version
113127
- name: Run unit tests
114128
run: make unit-tests
129+
- name: Upload unit test report
130+
if: always()
131+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
132+
with:
133+
name: bats-unit-report
134+
path: _out/test-reports/
135+
if-no-files-found: ignore
115136
- name: Run controller Go tests
116137
run: make test-controllers
117138

‎.pre-commit-config.yaml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,22 @@ repos:
2626
'
2727
language: system
2828
files: ^.*$
29+
# The hack/*.bats unit suite, hermetic and parallel-safe, runs in ~26s on
30+
# 8 cores under bats -j (#3453) -- cheap enough to gate a commit, and
31+
# faster than the two hooks above. Requires bats-core on PATH; the make
32+
# target says so explicitly when it is missing.
33+
#
34+
# The path filter is the union of what the suite reads, not just the
35+
# scripts under test: several tests assert cross-tree consistency
36+
# (image-pin-consistency over packages/, downstream-trigger-map over
37+
# workflows, md-no-hardwrap over .claude/hooks/), so narrowing this to
38+
# hack/ would let exactly the regressions they exist to catch through.
39+
- id: bats-unit-tests
40+
name: Run hack/*.bats unit tests
41+
entry: make bats-unit-tests
42+
language: system
43+
files: ^(hack/|Makefile$|\.github/workflows/|packages/|\.claude/hooks/)
44+
pass_filenames: false
2945
# Lints changed GitHub Actions workflows for unpinned actions and excessive token
3046
# permissions (audit scope is set in .github/zizmor.yml). Runs offline so it needs
3147
# no GitHub token locally. The same gate runs tree-wide in .github/workflows/zizmor.yml.

‎Makefile‎

Lines changed: 24 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -150,9 +150,16 @@ test-controllers:
150150
test-check-readiness:
151151
go test ./test/check-readiness/ -count=1
152152

153-
# Discover every hack/*.bats file that is NOT an e2e test and run it
154-
# through cozytest.sh. Drop a new *.bats file in hack/ and it is picked
155-
# up automatically on the next `make unit-tests` run.
153+
# Discover every hack/*.bats file that is NOT an e2e test and run it under
154+
# bats(1). Drop a new *.bats file in hack/ and it is picked up automatically
155+
# on the next `make unit-tests` run.
156+
#
157+
# These are hermetic unit tests of hack/*.sh, and they run under real bats
158+
# rather than hack/cozytest.sh (#3453). The live-cluster suite
159+
# (hack/e2e-*.bats) stays on cozytest, whose streaming trace and snapshot
160+
# behaviour earn their place over a 15-minute test; bats buffers a test's
161+
# output until it completes. Filtering by the e2e- prefix is what keeps the
162+
# two runners apart.
156163
#
157164
# Caveat: $(wildcard ...) returns space-separated names, so a filename
158165
# containing a literal space would split into multiple tokens here. All
@@ -161,15 +168,25 @@ test-check-readiness:
161168
# (e.g. to use `find ... -print0 | xargs -0`).
162169
BATS_UNIT_FILES := $(filter-out hack/e2e-%.bats,$(wildcard hack/*.bats))
163170

171+
# `bats -j` needs GNU parallel and exits non-zero rather than degrading when
172+
# it is missing, so resolve the job count to 1 unless parallel is present.
173+
BATS_JOBS ?= $(shell command -v parallel >/dev/null 2>&1 && nproc 2>/dev/null || echo 1)
174+
175+
# JUnit XML so CI annotates the failing test instead of requiring a log read.
176+
# _out is gitignored.
177+
BATS_REPORT_DIR ?= _out/test-reports
178+
164179
bats-unit-tests:
165180
@if [ -z "$(BATS_UNIT_FILES)" ]; then \
166181
echo "ERROR: no hack/*.bats unit test files found"; \
167182
exit 1; \
168183
fi
169-
@for f in $(BATS_UNIT_FILES); do \
170-
echo "--- running $$f ---"; \
171-
hack/cozytest.sh "$$f" || exit 1; \
172-
done
184+
@command -v bats >/dev/null 2>&1 || { \
185+
echo "ERROR: bats not found. Install bats-core >= 1.5 — https://bats-core.readthedocs.io"; \
186+
exit 1; \
187+
}
188+
@mkdir -p $(BATS_REPORT_DIR)
189+
bats -j $(BATS_JOBS) --report-formatter junit -o $(BATS_REPORT_DIR) $(BATS_UNIT_FILES)
173190

174191
# Operator-facing host preflight check. Warns about a standalone
175192
# containerd.service or docker.service running alongside the embedded

0 commit comments

Comments
 (0)