Skip to content

Commit dd2f89d

Browse files
test(sdn): port the securitygroup and serviceexposure suites to Chainsaw
Main added securitygroup.bats (#2922 membership model) and serviceexposure.bats (#3081 ExposureClass/ServiceExposure) after the Chainsaw migration branch was cut. - securitygroup: two Tests — the aggregated-API projection round-trip (backing CNP marker label, empty-value membership endpointSelector, lineage fromApp + membership fromSG peers, 1:1 rule translation, the view hiding storage internals, unmarked-CNP invisibility via an error op, delete propagation) and the controller membership-label stamp/clear lifecycle. Only the attachments annotation stays a jq script (it is a JSON string). Chainsaw auto-cleanup covers every fixture even on mid-test failure — the bats had no teardown hook and leaked its fixtures. - serviceexposure: the externalIPs backend resolve/Ready/assignedIPs path with the no-MetalLB-pool negative as an error op (ordered after resolvedBackend so it cannot false-pass), and the finalizer-removal test collapsed onto the delete op itself, which waits for full disappearance. - selection: kubernetes-application now also maps to the two OIDC render-side suites (they exercise the same app chart — a gap main's bats selection had too); the new dirs enter all_apps discovery automatically. Selector unit tests extended and green (13/13 under cozytest). - README: suites list updated. Assisted-By: Claude <[email protected]> Co-Authored-By: Claude Fable 5 <[email protected]> Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent cac07db commit dd2f89d

10 files changed

Lines changed: 433 additions & 3 deletions

File tree

‎hack/e2e-chainsaw/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ hack/e2e-chainsaw/
1818
└── ...
1919
```
2020

21-
Suites: `postgres`, `bucket`, `mariadb`, `mongodb`, `redis`, `qdrant`, `clickhouse`, `kafka`, `etcd`, `openbao`, `harbor`, `foundationdb`, `external-dns`, `kuberture`, `vminstance`, `gateway`, `kubernetes-latest`, `kubernetes-previous`.
21+
Suites: `postgres`, `bucket`, `mariadb`, `mongodb`, `redis`, `qdrant`, `clickhouse`, `kafka`, `etcd`, `openbao`, `harbor`, `foundationdb`, `external-dns`, `kuberture`, `vminstance`, `gateway`, `kubernetes-latest`, `kubernetes-previous`, `kubernetes-oidc-system`, `kubernetes-oidc-customconfig`, `securitygroup`, `serviceexposure`.
2222

2323
## What Chainsaw buys over the BATS suite
2424

Lines changed: 195 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,195 @@
1+
# Chainsaw port of hack/e2e-apps/securitygroup.bats.
2+
#
3+
# Exercises the SecurityGroup membership model against a live cluster. Two
4+
# things unit tests cannot prove: that the in-tree CiliumNetworkPolicy mirror
5+
# (membership endpointSelector, app/SG peer selectors, attachments annotation)
6+
# round-trips against the real cilium.io/v2 CRD, and that the
7+
# securitygroup-controller actually stamps the membership label onto an
8+
# attached application's pods.
9+
#
10+
# Everything both Tests create is chainsaw-applied, so auto-cleanup reclaims
11+
# it even on mid-test failure — strictly better than the bats, which had no
12+
# teardown hook and leaked its fixtures on failure. The one operator-created
13+
# object (the backing CNP) is the same storage object as the tracked
14+
# SecurityGroup, so no finally block is needed anywhere.
15+
apiVersion: chainsaw.kyverno.io/v1alpha1
16+
kind: Test
17+
metadata:
18+
name: securitygroup-projection
19+
spec:
20+
catch:
21+
- events: {}
22+
- describe:
23+
apiVersion: cilium.io/v2
24+
kind: CiliumNetworkPolicy
25+
name: sg-e2e
26+
steps:
27+
- name: create-securitygroup
28+
try:
29+
- apply:
30+
file: securitygroup.yaml
31+
32+
- name: assert-backing-cnp
33+
try:
34+
# The backing CNP must materialise with the marker label, the group's own
35+
# membership label as endpointSelector (the EMPTY value must round-trip
36+
# against the real cilium.io/v2 CRD), fromApp projected to the three
37+
# lineage labels, fromSG to the peer group's membership label (index
38+
# order fromApp-then-fromSG is code-guaranteed by projectIngress), and
39+
# the rule list translated 1:1 (port is a string in the CRD).
40+
- assert:
41+
timeout: 1m
42+
resource:
43+
apiVersion: cilium.io/v2
44+
kind: CiliumNetworkPolicy
45+
metadata:
46+
name: sg-e2e
47+
labels:
48+
sdn.cozystack.io/securitygroup: "true"
49+
spec:
50+
endpointSelector:
51+
matchLabels:
52+
securitygroup.sdn.cozystack.io/sg-e2e: ""
53+
(ingress[0].fromEndpoints[0].matchLabels):
54+
apps.cozystack.io/application.group: apps.cozystack.io
55+
apps.cozystack.io/application.kind: Kubernetes
56+
apps.cozystack.io/application.name: web
57+
(ingress[0].fromEndpoints[1].matchLabels):
58+
securitygroup.sdn.cozystack.io/sg-frontend-e2e: ""
59+
(ingress[0].toPorts[0].ports[0].port): "5432"
60+
(ingress[0].toPorts[0].ports[0].protocol): TCP
61+
# The attachments annotation is a JSON *string* on the backing policy —
62+
# parsed with jq, exactly like the bats (no reliance on JMESPath JSON
63+
# parsing in chainsaw v0.2.15).
64+
- script:
65+
timeout: 1m
66+
content: |
67+
set -eu
68+
att=$(kubectl -n "$NAMESPACE" get ciliumnetworkpolicy sg-e2e \
69+
-o jsonpath='{.metadata.annotations.sdn\.cozystack\.io/attachments}')
70+
echo "$att" | jq -e '.[0].kind == "Postgres" and .[0].name == "db"' >/dev/null
71+
check:
72+
($error == null): true
73+
74+
- name: assert-securitygroup-view
75+
try:
76+
# The SecurityGroup view reconstructs attachments and peers, and HIDES the
77+
# internal marker label + storage annotation.
78+
- assert:
79+
timeout: 1m
80+
resource:
81+
apiVersion: sdn.cozystack.io/v1alpha1
82+
kind: SecurityGroup
83+
metadata:
84+
name: sg-e2e
85+
spec:
86+
(attachments[0].kind): Postgres
87+
(attachments[0].name): db
88+
(ingress[0].fromApp[0].name): web
89+
(ingress[0].fromSG[0]): sg-frontend-e2e
90+
(metadata.labels."sdn.cozystack.io/securitygroup" == null): true
91+
(metadata.annotations."sdn.cozystack.io/attachments" == null): true
92+
93+
- name: unmarked-cnp-invisible
94+
try:
95+
- apply:
96+
file: plain-cnp.yaml
97+
# The aggregated API filters by the marker label synchronously, so
98+
# non-existence of the view is deterministic.
99+
- error:
100+
timeout: 30s
101+
resource:
102+
apiVersion: sdn.cozystack.io/v1alpha1
103+
kind: SecurityGroup
104+
metadata:
105+
name: plain-cnp-e2e
106+
107+
- name: delete-propagates
108+
try:
109+
# Deleting the SecurityGroup removes its backing policy (the delete op
110+
# waits for full disappearance of the view); the unmarked CNP survives.
111+
- delete:
112+
timeout: 1m
113+
ref:
114+
apiVersion: sdn.cozystack.io/v1alpha1
115+
kind: SecurityGroup
116+
name: sg-e2e
117+
- error:
118+
timeout: 1m
119+
resource:
120+
apiVersion: cilium.io/v2
121+
kind: CiliumNetworkPolicy
122+
metadata:
123+
name: sg-e2e
124+
- assert:
125+
timeout: 30s
126+
resource:
127+
apiVersion: cilium.io/v2
128+
kind: CiliumNetworkPolicy
129+
metadata:
130+
name: plain-cnp-e2e
131+
---
132+
apiVersion: chainsaw.kyverno.io/v1alpha1
133+
kind: Test
134+
metadata:
135+
name: securitygroup-membership
136+
spec:
137+
catch:
138+
- events: {}
139+
- describe:
140+
apiVersion: v1
141+
kind: Pod
142+
name: sg-member-pod-e2e
143+
- podLogs:
144+
namespace: cozy-securitygroup-controller
145+
selector: app=securitygroup-controller
146+
tail: 100
147+
steps:
148+
- name: create-member-pod-and-securitygroup
149+
try:
150+
- apply:
151+
file: member-pod.yaml
152+
- apply:
153+
file: securitygroup-member.yaml
154+
155+
- name: membership-label-stamped
156+
try:
157+
# The controller must stamp the membership label onto the member pod;
158+
# the value is the empty string (reconciler sets pod.Labels[key] = "").
159+
- assert:
160+
timeout: 2m
161+
resource:
162+
apiVersion: v1
163+
kind: Pod
164+
metadata:
165+
name: sg-member-pod-e2e
166+
labels:
167+
securitygroup.sdn.cozystack.io/sg-member-e2e: ""
168+
169+
- name: delete-clears-membership
170+
try:
171+
# Deleting the SecurityGroup while a pod is still a member must strip the
172+
# membership label off that pod (the controller's finalizer runs before
173+
# the backing policy is garbage-collected), leaving no orphaned label.
174+
# This is the delete-with-members path, distinct from detach.
175+
- delete:
176+
timeout: 2m
177+
ref:
178+
apiVersion: sdn.cozystack.io/v1alpha1
179+
kind: SecurityGroup
180+
name: sg-member-e2e
181+
- error:
182+
timeout: 1m
183+
resource:
184+
apiVersion: cilium.io/v2
185+
kind: CiliumNetworkPolicy
186+
metadata:
187+
name: sg-member-e2e
188+
- assert:
189+
timeout: 1m
190+
resource:
191+
apiVersion: v1
192+
kind: Pod
193+
metadata:
194+
name: sg-member-pod-e2e
195+
(metadata.labels."securitygroup.sdn.cozystack.io/sg-member-e2e" == null): true
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# A pod carrying the lineage labels of a managed application. The lineage
2+
# webhook skips it (managed-by is already set), so these labels are exactly
3+
# what the securitygroup-controller resolves the attachment to. The full
4+
# restricted-profile securityContext is required by tenant-test PSA.
5+
apiVersion: v1
6+
kind: Pod
7+
metadata:
8+
name: sg-member-pod-e2e
9+
labels:
10+
internal.cozystack.io/managed-by-cozystack: "true"
11+
apps.cozystack.io/application.group: apps.cozystack.io
12+
apps.cozystack.io/application.kind: Postgres
13+
apps.cozystack.io/application.name: sg-member-app
14+
spec:
15+
securityContext:
16+
runAsNonRoot: true
17+
runAsUser: 65532
18+
seccompProfile:
19+
type: RuntimeDefault
20+
containers:
21+
- name: pause
22+
image: registry.k8s.io/pause:3.9
23+
securityContext:
24+
allowPrivilegeEscalation: false
25+
readOnlyRootFilesystem: true
26+
capabilities:
27+
drop: ["ALL"]
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
apiVersion: cilium.io/v2
2+
kind: CiliumNetworkPolicy
3+
metadata:
4+
name: plain-cnp-e2e
5+
spec:
6+
endpointSelector:
7+
matchLabels:
8+
app: untouched
9+
# A rule is mandatory: the CiliumNetworkPolicy CRD requires at least one of
10+
# ingress/ingressDeny/egress/egressDeny (spec anyOf), so a selector-only
11+
# policy is rejected at apply time. This keeps the fixture a valid, applyable
12+
# CNP while leaving it unmarked (no securitygroup label) — so it must stay
13+
# invisible to the SecurityGroup API.
14+
ingress:
15+
- fromEndpoints:
16+
- matchLabels:
17+
app: peer
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
apiVersion: sdn.cozystack.io/v1alpha1
2+
kind: SecurityGroup
3+
metadata:
4+
name: sg-member-e2e
5+
spec:
6+
attachments:
7+
- kind: Postgres
8+
name: sg-member-app
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# SecurityGroup created through the aggregated API (storage-backed by the
2+
# CiliumNetworkPolicy itself). It attaches to a managed application; the
3+
# backing endpointSelector is the group's own membership label, and peers are
4+
# expressed as fromApp/fromSG.
5+
apiVersion: sdn.cozystack.io/v1alpha1
6+
kind: SecurityGroup
7+
metadata:
8+
name: sg-e2e
9+
spec:
10+
attachments:
11+
- kind: Postgres
12+
name: db
13+
ingress:
14+
- fromApp:
15+
- kind: Kubernetes
16+
name: web
17+
fromSG:
18+
- sg-frontend-e2e
19+
toPorts:
20+
- ports:
21+
- port: "5432"
22+
protocol: TCP

0 commit comments

Comments
 (0)