Skip to content

Commit 8fae615

Browse files
committed
test(cozystack-basics): make the version-coverage guard prove it still compares
The guard added alongside the TLSRoute version fix only ever ran against a tree that already satisfied it. A passing check that is checking nothing looks exactly like a passing check that is checking something, so a later refactor could empty out the comparison and the suite would stay green forever. That is the same shape as the negated assertions that cannot fail elsewhere in this repository: the assertion survives, the guarantee does not. Drive the checker with synthetic input instead of only with the live tree, the way hack/bats-no-exit-trap.bats and hack/md-no-hardwrap.bats feed their checkers a broken sample and require a complaint. Two layers need it separately. The comparison itself moves into uncovered() so a fixture can call it with a served set the named set does not cover. Above that, assert_covers runs in a subshell with both readers replaced, so the wiring, the empty-set guards and the exit status are exercised too -- covering only the inner function leaves the outer one free to become a no-op while every other case in the file stays green, because the live-tree cases are its only callers and the tree they read is correct. Each fixture requires the complaint it expects rather than any non-zero exit. A renamed helper exits 127 and an unbound variable under set -u exits 1, and either would read as a successful rejection if only the status were checked; a rejection for the wrong reason is the same false comfort as no fixture, and harder to notice because the red looks earned. Each way of breaking the guard now fails at its own case. Emptying uncovered(), reducing assert_covers to a no-op, and deleting the empty-set guards each turn a different fixture red while the live-tree cases stay green, and changing only the wording of assert_covers' complaint -- which still rejects, just for an unstated reason -- turns the omission fixture red on the message rather than passing on the exit status. Select a resourceRules entry by comparing each resource name for equality, and match versions as fixed strings rather than patterns. yq's contains() compares array elements by substring, so a rule scoped to a subresource would have satisfied a search for the parent: a rule naming tlsroutes/status answers a query for tlsroutes and donates its apiVersions to the set the guard checks coverage against. That direction is the wrong one for a fail-closed check -- it makes coverage succeed where the real rule falls short. No rule in this policy is scoped that way today. Also pin apiGroups on the TLSRoute document in the chart suite. The HTTPRoute document has always had that assertion; the asymmetry meant the case named after both kinds only checked the group for one of them. Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent 4a09504 commit 8fae615

2 files changed

Lines changed: 95 additions & 10 deletions

File tree

‎hack/route-hostname-policy-version-coverage.bats‎

Lines changed: 91 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,26 @@ policy_versions() {
6060
| yq eval-all \
6161
"select(.kind == \"ValidatingAdmissionPolicy\" and .metadata.name == \"$1\")
6262
| .spec.matchConstraints.resourceRules[]
63-
| select(.resources[] == \"$2\") | .apiVersions[]" - | sort -u
63+
| select(.resources[] | select(. == \"$2\")) | .apiVersions[]" - | sort -u
64+
}
65+
66+
# Echoes, space-prefixed, each served version the policy does not name. The
67+
# comparison lives here rather than inline so the fixtures at the bottom can
68+
# drive it with synthetic input; neutering it turns them red.
69+
#
70+
# Membership by loop rather than comm(1) with process substitution: the
71+
# cozytest.sh harness rejects <(...) with a syntax error, which exits non-zero
72+
# and would read as a genuine failure.
73+
uncovered() {
74+
served=$1
75+
named=$2
76+
out=
77+
for version in $served; do
78+
if ! echo "$named" | grep -qxF "$version"; then
79+
out="$out $version"
80+
fi
81+
done
82+
echo "$out"
6483
}
6584

6685
# Fails on an empty set from either side: a yq expression that stops matching
@@ -83,15 +102,7 @@ assert_covers() {
83102
exit 1
84103
fi
85104

86-
# Membership by loop rather than comm(1) with process substitution: the
87-
# cozytest.sh harness rejects <(...) with a syntax error, which exits
88-
# non-zero and would read as a genuine failure.
89-
missing=
90-
for version in $served; do
91-
if ! echo "$named" | grep -qx "$version"; then
92-
missing="$missing $version"
93-
fi
94-
done
105+
missing=$(uncovered "$served" "$named")
95106

96107
if [ -n "$missing" ]; then
97108
echo "$policy_name does not name every served version of $crd_name" >&2
@@ -111,3 +122,73 @@ assert_covers() {
111122
assert_covers httproutes.gateway.networking.k8s.io \
112123
cozystack-route-hostname-policy httproutes
113124
}
125+
126+
# The two cases above only ever see a tree that is already correct, so on their
127+
# own they cannot tell a working comparison from one that stopped comparing.
128+
# These drive it with synthetic input, the way hack/bats-no-exit-trap.bats and
129+
# hack/md-no-hardwrap.bats feed their checkers a broken sample and require a
130+
# complaint.
131+
#
132+
# Both layers need their own fixture. Driving uncovered() alone leaves
133+
# assert_covers unprotected -- emptying it to a no-op keeps every other case in
134+
# this file green, because the live-tree cases are the only callers and the
135+
# tree they read is correct. So the cases below feed assert_covers itself, in a
136+
# subshell with the two readers redefined.
137+
#
138+
# Each requires the complaint it expects, not merely a non-zero exit. A renamed
139+
# helper exits 127 and an unbound variable under `set -u` exits 1, and either
140+
# would read as a successful rejection if only the status were checked -- red
141+
# for the wrong reason is the same false comfort as no fixture at all.
142+
assert_covers_rejects() {
143+
expected=$1
144+
rc=0
145+
out=$(
146+
served_versions() { printf '%s\n' "$FIXTURE_SERVED"; }
147+
policy_versions() { printf '%s\n' "$FIXTURE_NAMED"; }
148+
assert_covers fixture.crd fixture-policy fixtureroutes 2>&1
149+
) || rc=$?
150+
[ "$rc" != "0" ]
151+
case "$out" in
152+
*"$expected"*) ;;
153+
*) echo "rejected, but not for the expected reason" >&2
154+
echo "wanted: $expected" >&2
155+
echo "got: $out" >&2
156+
exit 1 ;;
157+
esac
158+
}
159+
160+
@test "assert_covers rejects a policy that omits a served version" {
161+
FIXTURE_SERVED='v1
162+
v1alpha2'
163+
FIXTURE_NAMED='v1alpha2'
164+
export FIXTURE_SERVED FIXTURE_NAMED
165+
assert_covers_rejects "does not name every served version"
166+
}
167+
168+
@test "assert_covers rejects an empty served set rather than passing vacuously" {
169+
FIXTURE_SERVED=''
170+
FIXTURE_NAMED='v1'
171+
export FIXTURE_SERVED FIXTURE_NAMED
172+
assert_covers_rejects "no served versions found"
173+
}
174+
175+
@test "assert_covers rejects an empty named set rather than passing vacuously" {
176+
FIXTURE_SERVED='v1'
177+
FIXTURE_NAMED=''
178+
export FIXTURE_SERVED FIXTURE_NAMED
179+
assert_covers_rejects "no apiVersions found"
180+
}
181+
182+
@test "the coverage check names a served version the policy omits" {
183+
missing=$(uncovered 'v1
184+
v1alpha2
185+
v1alpha3' 'v1alpha2')
186+
[ "$missing" = " v1 v1alpha3" ]
187+
}
188+
189+
@test "the coverage check stays silent when every served version is named" {
190+
missing=$(uncovered 'v1
191+
v1beta1' 'v1
192+
v1beta1')
193+
[ -z "$missing" ]
194+
}

‎packages/system/cozystack-basics/tests/route-hostname-policy_test.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,10 @@ tests:
104104
equal:
105105
path: spec.matchConstraints.resourceRules[0].resources
106106
value: ["httproutes"]
107+
- documentIndex: 2
108+
equal:
109+
path: spec.matchConstraints.resourceRules[0].apiGroups
110+
value: ["gateway.networking.k8s.io"]
107111
- documentIndex: 2
108112
equal:
109113
path: spec.matchConstraints.resourceRules[0].resources

0 commit comments

Comments
 (0)