Skip to content

Commit abd331b

Browse files
lexfreilexfrei
authored
test(system): anchor index-addressed negative asserts to what they name (#3608)
## What this PR does Some negative assertions in these suites can't fail for the reason their test names give. `contains` and `notContains` compare list elements for equality, so `--pinned-tenants=` never equals a rendered `--pinned-tenants=tenant-a=shard0`, and `{name: GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET}` never matches an env entry carrying a `valueFrom` beside the name. Others forbid one rendered value under a name that claims the flag itself, so `notContains --compatibility=cilium` leaves the kilo default render free to carry `--compatibility=flannel` off the same values key. Flag guards become `notMatchRegex` on `args[*]`, which matches per element, so any value fails and so does the bare form. Mapping guards get `any: true`. flux-shard-operator gets an `equal` on the whole default arg list instead, and `--shard-cpu-limit=` is dropped because that list covers it. The vendored ouroboros templates keep per-flag guards: a whole-list pin there would go red on any upstream flag change, under a test name about one flag. Several of these address `containers[0]` with nothing pinning that index, so a sidecar rendered ahead of the workload turns them green while the flag is live on the real container. Those get an `equal` on the container name first. A Cilium deny rule has no name, so that one pins the selector plus the rule and `toPorts` counts. Where a positive `contains` already sits on the same path it holds the index by itself and gets no extra anchor. Assertions of the same shape remain in files this PR doesn't touch, including one in a package it does. ## Tests Every dead guard was confirmed dead first: make the forbidden content render, watch the old form stay green. `compatibility: flannel` in kilo's values, `--gateway-api=true` in place of the bare flag, `--proxy-fqdn` baked beside the composition flags, the OIDC client secret in the `mode=None` grafana render. Each one turns the new assertion red. Anchors checked the same way, with a sidecar rendered ahead of the workload. `hack/helm-unit-tests.sh` is green. ### Screenshots None. This changes test files. ### Downstream repositories Every changed file is under a `tests/` directory. No templates, no values, no rendered output. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note NONE ```
2 parents 9dc49de + d73c102 commit abd331b

7 files changed

Lines changed: 132 additions & 59 deletions

File tree

‎packages/system/cilium-networkpolicy/tests/networkpolicy_test.yaml‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,35 @@ tests:
8181
# bound by anything cozystack ships, so listing it in the deny rule is
8282
# misleading.
8383
asserts:
84+
# Anchor the rule before asserting about it. notContains passes whenever
85+
# its path resolves to something that lacks the content, and a different
86+
# deny rule at [0] qualifies, so reordering spec.ingressDeny would leave
87+
# this green while 7473 came back on the rule it names. A deny rule has no
88+
# name, so the anchor is its selector, which is what identifies it rather
89+
# than one of the ports it happens to list.
90+
#
91+
# The selector identifies it only while there is one rule to identify: a
92+
# second world-scoped deny inserted ahead of this one would satisfy the
93+
# selector and shift the real rule to [1], which is the reordering above.
94+
# lengthEqual is what makes the selector sufficient.
95+
# Both indexes in the path, not just the outer one. A second toPorts entry
96+
# on the same rule carries its own ports list, so 7473 could come back
97+
# there with the rule count and the selector both still satisfied.
98+
- lengthEqual:
99+
path: spec.ingressDeny
100+
count: 1
101+
- lengthEqual:
102+
path: spec.ingressDeny[0].toPorts
103+
count: 1
104+
- contains:
105+
path: spec.ingressDeny[0].fromEntities
106+
content: world
107+
# any: true matches on the port alone. Without it the comparison is a
108+
# deep-equal against the whole entry, so 7473 re-added with a protocol
109+
# beside it, which is how the neighbouring 7946 entry is written, would
110+
# slip past a guard that looks like it forbids the port outright.
84111
- notContains:
85112
path: spec.ingressDeny[0].toPorts[0].ports
113+
any: true
86114
content:
87115
port: "7473"

‎packages/system/flux-shard-operator/tests/workload_test.yaml‎

Lines changed: 31 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -17,48 +17,31 @@ tests:
1717
- equal:
1818
path: spec.template.spec.serviceAccountName
1919
value: flux-shard-operator
20-
- contains:
21-
path: spec.template.spec.containers[0].args
22-
content: --leader-elect
23-
- contains:
24-
path: spec.template.spec.containers[0].args
25-
content: --flux-namespace=cozy-fluxcd
26-
- contains:
27-
path: spec.template.spec.containers[0].args
28-
content: --shard-count=auto
29-
- contains:
30-
path: spec.template.spec.containers[0].args
31-
content: --shard-concurrent=5
32-
- contains:
33-
path: spec.template.spec.containers[0].args
34-
content: --rebalance-threshold=0.25
35-
- contains:
36-
path: spec.template.spec.containers[0].args
37-
content: --shard-cpu-request=100m
38-
- contains:
39-
path: spec.template.spec.containers[0].args
40-
content: --shard-memory-request=64Mi
41-
- contains:
42-
path: spec.template.spec.containers[0].args
43-
content: --shard-memory-limit=1Gi
44-
- notContains:
45-
path: spec.template.spec.containers[0].args
46-
content: --shard-cpu-limit=
47-
- contains:
20+
# The whole list, so the pin fails on an argument added, removed or
21+
# swapped rather than only on the ones someone thought to enumerate.
22+
# This template is first-party, so a failure here is a deliberate change
23+
# asking to be read, not upstream churn arriving on its own schedule.
24+
#
25+
# --metrics-bind-address and --metrics-secure are load-bearing among
26+
# these: the README-advertised cozy_flux_shard_* gauges, autosizing
27+
# recommendation included, are the operator's observability surface.
28+
- equal:
4829
path: spec.template.spec.containers[0].args
49-
content: --zap-log-level=info
30+
value:
31+
- --leader-elect
32+
- --metrics-bind-address=:8080
33+
- --metrics-secure=false
34+
- --flux-namespace=cozy-fluxcd
35+
- --shard-count=auto
36+
- --shard-concurrent=5
37+
- --rebalance-threshold=0.25
38+
- --shard-cpu-request=100m
39+
- --shard-memory-request=64Mi
40+
- --shard-memory-limit=1Gi
41+
- --zap-log-level=info
5042
- equal:
5143
path: spec.template.spec.volumes[0].secret.secretName
5244
value: flux-shard-operator-cert
53-
# The metrics endpoint must stay reachable: the README-advertised
54-
# cozy_flux_shard_* gauges (incl. the autosizing recommendation) are the
55-
# operator's observability surface.
56-
- contains:
57-
path: spec.template.spec.containers[0].args
58-
content: --metrics-bind-address=:8080
59-
- contains:
60-
path: spec.template.spec.containers[0].args
61-
content: --metrics-secure=false
6245
- equal:
6346
path: spec.template.metadata.annotations["prometheus.io/scrape"]
6447
value: "true"
@@ -72,9 +55,16 @@ tests:
7255
- it: does not render --pinned-tenants by default
7356
template: templates/workload.yaml
7457
asserts:
75-
- notContains:
76-
path: spec.template.spec.containers[0].args
77-
content: --pinned-tenants=
58+
- equal:
59+
path: spec.template.spec.containers[0].name
60+
value: flux-shard-operator
61+
# Match the flag name per element, not one whole element against one
62+
# value: contains and notContains compare elements for equality, so
63+
# `--pinned-tenants=` never equals a rendered
64+
# `--pinned-tenants=tenant-a=shard0` and the negative could not fail.
65+
- notMatchRegex:
66+
path: spec.template.spec.containers[0].args[*]
67+
pattern: "^--pinned-tenants"
7868

7969
- it: renders shardCount and pinned tenants from values
8070
template: templates/workload.yaml

‎packages/system/keycloak/tests/encryption_test.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -445,6 +445,13 @@ tests:
445445
root-host: example.org
446446
cluster-domain: cozy.local
447447
asserts:
448+
# Anchor the index before asserting on it. notContains passes whenever its
449+
# path resolves to something that lacks the content, and a different
450+
# container at [0] qualifies, so without this the guard goes green if a
451+
# sidecar is ever placed ahead of the proxy.
452+
- equal:
453+
path: spec.template.spec.containers[0].name
454+
value: proxy
448455
- notContains:
449456
path: spec.template.spec.containers[0].env
450457
any: true

‎packages/system/kilo/tests/kilo_test.yaml‎

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,19 @@ tests:
6161

6262
- it: "no --compatibility arg in the default render"
6363
asserts:
64-
- notContains:
65-
path: spec.template.spec.containers[0].args
66-
content: "--compatibility=cilium"
64+
# Anchor the index before asserting on it. A negative assert passes
65+
# whenever its path resolves to something that does not match, and a
66+
# different container at [0] qualifies — as does an index resolving to
67+
# nothing at all — so without this the guard goes green if a sidecar is
68+
# ever placed ahead of kilo.
69+
- equal:
70+
path: spec.template.spec.containers[0].name
71+
value: kilo
72+
# Match the flag name per element, not one whole element against one
73+
# value: contains and notContains compare elements for equality, and the
74+
# chart renders --compatibility=<value> off a single values key, so an
75+
# equality check against the cilium value leaves every other value
76+
# unforbidden.
77+
- notMatchRegex:
78+
path: spec.template.spec.containers[0].args[*]
79+
pattern: "^--compatibility"

‎packages/system/monitoring/tests/oidc_test.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,12 +79,25 @@ tests:
7979
asserts:
8080
- notExists:
8181
path: spec.config["auth.generic_oauth"]
82+
# Anchor the index before asserting on it. notContains passes whenever its
83+
# path resolves to something that lacks the content, and a different
84+
# container at [0] qualifies, so without this both guards go green if a
85+
# sidecar is ever placed ahead of grafana in the CR's deployment spec.
86+
- equal:
87+
path: spec.deployment.spec.template.spec.containers[0].name
88+
value: grafana
89+
# any: true matches on the element name alone. Without it the comparison
90+
# is a deep-equal against the whole entry, and the real ones carry a
91+
# valueFrom or a value alongside the name, so {name: ...} matches nothing
92+
# and the guard passes whether or not the variable is rendered.
8293
- notContains:
8394
path: spec.deployment.spec.template.spec.containers[0].env
95+
any: true
8496
content:
8597
name: GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET
8698
- notContains:
8799
path: spec.deployment.spec.template.spec.containers[0].env
100+
any: true
88101
content:
89102
name: GF_PATHS_CUSTOM_INI
90103

‎packages/system/ouroboros/tests/controller_test.yaml‎

Lines changed: 28 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -22,16 +22,32 @@ tests:
2222
- contains:
2323
path: spec.template.spec.containers[0].args
2424
content: --proxy-service-namespace=cozy-ouroboros
25-
- notContains:
26-
path: spec.template.spec.containers[0].args
27-
content: --proxy-fqdn=ouroboros-proxy.cozy-ouroboros.svc.cluster.local.
25+
# Match the flag name per element, not one whole element against one
26+
# value: contains and notContains compare elements for equality, and the
27+
# baked FQDN carries whatever controller.clusterDomain is set to, so an
28+
# equality check against the form the default cluster-domain produces
29+
# leaves every other value unforbidden.
30+
- notMatchRegex:
31+
path: spec.template.spec.containers[0].args[*]
32+
pattern: "^--proxy-fqdn"
2833

2934
- it: leaves Gateway-API watching off at the parent default (consumer enables)
3035
template: charts/ouroboros/templates/controller-deployment.yaml
3136
asserts:
32-
- notContains:
33-
path: spec.template.spec.containers[0].args
34-
content: --gateway-api
37+
# Anchor the index before asserting on it. A negative assert passes
38+
# whenever its path resolves to something that does not match, and a
39+
# different container at [0] qualifies — as does an index resolving to
40+
# nothing at all — so without this the guard goes green if a sidecar is
41+
# ever placed ahead of the controller.
42+
- equal:
43+
path: spec.template.spec.containers[0].name
44+
value: controller
45+
# Match the flag name per element. An equality check against the bare
46+
# form the chart renders today would go on passing if it ever rendered
47+
# --gateway-api=true, with watching on.
48+
- notMatchRegex:
49+
path: spec.template.spec.containers[0].args[*]
50+
pattern: "^--gateway-api"
3551

3652
- it: emits chart-default --resync=30s — host scenario inherits the upstream chart default (changed from 10m in upstream v0.7.x). The reduction is the controller's only recovery path when an Ingress watch silently dies, and 30s caps worst-case event-to-reconcile latency under broken-watch conditions. Pinning here catches a future upstream bump back to the longer value before it lands as a CI surprise.
3753
template: charts/ouroboros/templates/controller-deployment.yaml
@@ -151,15 +167,15 @@ tests:
151167
# too — this is the mode tenants actually run, so a future refactor
152168
# in the upstream controller-deployment template that drops the
153169
# composition block from this branch only would break tenants
154-
# silently. The negative-assert pins that --proxy-fqdn does NOT
155-
# bake the cluster-domain at render time when no clusterDomain is
156-
# configured (the platform-agnostic default).
170+
# silently. The negative assert pins that no --proxy-fqdn is baked at
171+
# render time when no clusterDomain is configured (the
172+
# platform-agnostic default).
157173
- contains:
158174
path: spec.template.spec.containers[0].args
159175
content: --proxy-service-name=ouroboros-proxy
160176
- contains:
161177
path: spec.template.spec.containers[0].args
162178
content: --proxy-service-namespace=cozy-ouroboros
163-
- notContains:
164-
path: spec.template.spec.containers[0].args
165-
content: --proxy-fqdn=ouroboros-proxy.cozy-ouroboros.svc.cluster.local.
179+
- notMatchRegex:
180+
path: spec.template.spec.containers[0].args[*]
181+
pattern: "^--proxy-fqdn"

‎packages/system/ouroboros/tests/cozystack_overrides_test.yaml‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -27,9 +27,15 @@ tests:
2727
- contains:
2828
path: spec.template.spec.containers[0].args
2929
content: --target-service-namespace=cozy-ingress-nginx
30-
- notContains:
31-
path: spec.template.spec.containers[0].args
32-
content: --target-host=
30+
# Match the flag name per element, not one whole element against one
31+
# value: contains and notContains compare elements for equality, so
32+
# `--target-host=` never equals a rendered `--target-host=<value>` and
33+
# that negative could not fail. A length assertion would not do either,
34+
# since an upstream bump that adds the flag while consolidating another
35+
# argument keeps the count.
36+
- notMatchRegex:
37+
path: spec.template.spec.containers[0].args[*]
38+
pattern: "^--target-host"
3339

3440
- it: controller image is pinned to the digest the values.yaml ships
3541
template: charts/ouroboros/templates/controller-deployment.yaml

0 commit comments

Comments
 (0)