Skip to content

Commit 5a9a727

Browse files
test(e2e): assert the webhook deny is enforced and scoped to its port
The churn case added alongside it proves the webhooks stay AVAILABLE. Nothing proved they are still UNREACHABLE from pods, which is the property the policies exist for, and no test anywhere checked that the deny is confined to the webhook port -- which is why a rule that denied all TCP/443 egress cluster-wide passed every existing assertion. Probe from tenant-root, where the host cluster's ingress-nginx lives and where allow-internal-communication permits pods to reach it, so the deny is the only thing that can block the webhook port; from the isolated tenant-test the probe would pass for the wrong reason. Three assertions against the same pods and the same tenant egress allowances, differing only in port: the controller's own HTTPS port must stay reachable, the admission Service ClusterIP must not be, and the backend port must not be. The first is the one that separates a scoped deny from a wildcard. Uses busybox:musl, already staged on every node by hack/e2e-prepull-images.sh, with a securityContext that satisfies the restricted PodSecurity profile. Service names, ports and the backend port are discovered at runtime rather than hardcoded, since the release name differs between installs. Assisted-By: Claude <[email protected]> Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent 37bfcb6 commit 5a9a727

1 file changed

Lines changed: 153 additions & 0 deletions

File tree

‎hack/e2e-chainsaw/gateway/chainsaw-test.yaml‎

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1440,3 +1440,156 @@ spec:
14401440
apiVersion: v1
14411441
kind: Namespace
14421442
name: tenant-admission-churn-probe
1443+
---
1444+
# Companion to admission-webhooks-survive-namespace-churn above. That test
1445+
# proves the webhooks stay AVAILABLE; this one proves they are still
1446+
# UNREACHABLE from pods, which is the property the policies exist for, and that
1447+
# the deny is scoped to the webhook port alone.
1448+
#
1449+
# The third assertion is the one that earns this test. The policies previously
1450+
# expressed the Service-frontend deny with `toServices`, which Cilium resolves
1451+
# only inside ALLOW rules -- leaving the deny rule with an empty L3 selector
1452+
# that Cilium reads as a wildcard, so "deny this Service on 443" became "deny
1453+
# EVERY destination on 443" for every selected pod. Every rendered-YAML
1454+
# assertion in the package suites passed on that policy. Only a probe that
1455+
# checks an unrelated 443 destination still works can tell the two apart, and
1456+
# the ingress controller's own HTTPS port is exactly that: the same pods, the
1457+
# same tenant egress allowances, a different port.
1458+
#
1459+
# Runs in tenant-root because that is where the host cluster's ingress-nginx
1460+
# lives and where pods are permitted to reach it (allow-internal-communication),
1461+
# so the deny is the only thing that can block the webhook port. From the
1462+
# isolated tenant-test the probe would pass for the wrong reason.
1463+
apiVersion: chainsaw.kyverno.io/v1alpha1
1464+
kind: Test
1465+
metadata:
1466+
name: admission-webhook-deny-still-enforced
1467+
spec:
1468+
namespace: tenant-root
1469+
catch:
1470+
- events: {}
1471+
steps:
1472+
- name: pods-cannot-reach-the-webhook-but-keep-unrelated-443-egress
1473+
try:
1474+
- script:
1475+
timeout: 6m
1476+
content: |
1477+
set -eu
1478+
ns="$NAMESPACE"
1479+
probe=admission-deny-probe
1480+
sel=app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/component=controller
1481+
1482+
svcs=$(kubectl -n "$ns" get svc -l "$sel" \
1483+
-o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}')
1484+
adm=$(printf '%s\n' "$svcs" | grep -- '-admission$' | head -1 || true)
1485+
ctl=$(printf '%s\n' "$svcs" | grep -v -- '-admission$' | grep . | head -1 || true)
1486+
[ -n "$adm" ] || { echo "SETUP FAILURE: no ingress-nginx admission Service in $ns" >&2; exit 1; }
1487+
[ -n "$ctl" ] || { echo "SETUP FAILURE: no ingress-nginx controller Service in $ns" >&2; exit 1; }
1488+
1489+
adm_port=$(kubectl -n "$ns" get svc "$adm" -o jsonpath='{.spec.ports[0].port}')
1490+
ctl_port=$(kubectl -n "$ns" get svc "$ctl" -o jsonpath='{.spec.ports[?(@.name=="https")].port}')
1491+
[ -n "$ctl_port" ] || { echo "SETUP FAILURE: $ctl exposes no https port" >&2; exit 1; }
1492+
1493+
kubectl -n "$ns" wait --for=condition=Ready pod -l "$sel" --timeout=3m
1494+
pod_ip=$(kubectl -n "$ns" get pod -l "$sel" \
1495+
-o jsonpath='{.items[0].status.podIP}')
1496+
hook_port=$(kubectl -n "$ns" get pod -l "$sel" \
1497+
-o jsonpath='{.items[0].spec.containers[0].ports[?(@.name=="webhook")].containerPort}')
1498+
[ -n "$pod_ip" ] || { echo "SETUP FAILURE: no controller Pod IP in $ns" >&2; exit 1; }
1499+
[ -n "$hook_port" ] || { echo "SETUP FAILURE: controller declares no webhook port" >&2; exit 1; }
1500+
1501+
echo "probing from $ns: allow ${ctl}:${ctl_port}, deny ${adm}:${adm_port}, deny ${pod_ip}:${hook_port}"
1502+
1503+
kubectl -n "$ns" delete pod "$probe" --ignore-not-found --wait=true --timeout=60s
1504+
# busybox:musl is staged onto every node by hack/e2e-prepull-images.sh,
1505+
# so this adds no image pull. The securityContext satisfies the
1506+
# restricted PodSecurity profile, so the probe runs in a tenant
1507+
# namespace whatever level is enforced there.
1508+
kubectl -n "$ns" apply -f - <<EOF
1509+
apiVersion: v1
1510+
kind: Pod
1511+
metadata:
1512+
name: ${probe}
1513+
spec:
1514+
restartPolicy: Never
1515+
automountServiceAccountToken: false
1516+
securityContext:
1517+
runAsNonRoot: true
1518+
runAsUser: 65534
1519+
runAsGroup: 65534
1520+
seccompProfile:
1521+
type: RuntimeDefault
1522+
containers:
1523+
- name: probe
1524+
image: busybox:musl
1525+
securityContext:
1526+
allowPrivilegeEscalation: false
1527+
capabilities:
1528+
drop: ["ALL"]
1529+
env:
1530+
- name: CTL_HOST
1531+
value: "${ctl}.${ns}.svc"
1532+
- name: CTL_PORT
1533+
value: "${ctl_port}"
1534+
- name: ADM_HOST
1535+
value: "${adm}.${ns}.svc"
1536+
- name: ADM_PORT
1537+
value: "${adm_port}"
1538+
- name: POD_IP
1539+
value: "${pod_ip}"
1540+
- name: HOOK_PORT
1541+
value: "${hook_port}"
1542+
command: ["sh", "-c"]
1543+
args:
1544+
- |
1545+
set -u
1546+
rc=0
1547+
reach() { nc -w 5 "\$1" "\$2" </dev/null >/dev/null 2>&1; }
1548+
1549+
if reach "\$CTL_HOST" "\$CTL_PORT"; then
1550+
echo "OK unrelated 443 egress works (\$CTL_HOST:\$CTL_PORT)"
1551+
else
1552+
echo "BUG unrelated 443 egress DENIED (\$CTL_HOST:\$CTL_PORT) -- the webhook deny is not scoped to the webhook port"
1553+
rc=1
1554+
fi
1555+
1556+
if reach "\$ADM_HOST" "\$ADM_PORT"; then
1557+
echo "BUG webhook reachable through its Service ClusterIP (\$ADM_HOST:\$ADM_PORT)"
1558+
rc=1
1559+
else
1560+
echo "OK webhook denied through its Service ClusterIP"
1561+
fi
1562+
1563+
if reach "\$POD_IP" "\$HOOK_PORT"; then
1564+
echo "BUG webhook reachable directly on its backend port (\$POD_IP:\$HOOK_PORT)"
1565+
rc=1
1566+
else
1567+
echo "OK webhook denied on its backend port"
1568+
fi
1569+
exit \$rc
1570+
EOF
1571+
1572+
# Wait for a terminal phase rather than a condition: a Failed probe has
1573+
# to be read, not waited out.
1574+
deadline=$(( $(date +%s) + 180 ))
1575+
phase=""
1576+
while [ "$(date +%s)" -lt "$deadline" ]; do
1577+
phase=$(kubectl -n "$ns" get pod "$probe" -o jsonpath='{.status.phase}' 2>/dev/null || true)
1578+
case "$phase" in Succeeded|Failed) break ;; esac
1579+
sleep 3
1580+
done
1581+
1582+
kubectl -n "$ns" logs "$probe" 2>&1 || true
1583+
[ "$phase" = "Succeeded" ] || {
1584+
echo "probe did not pass (phase=${phase:-unknown})" >&2
1585+
kubectl -n "$ns" describe pod "$probe" >&2 2>&1 || true
1586+
exit 1
1587+
}
1588+
check:
1589+
($error == null): true
1590+
finally:
1591+
- delete:
1592+
ref:
1593+
apiVersion: v1
1594+
kind: Pod
1595+
name: admission-deny-probe

0 commit comments

Comments
 (0)