Skip to content

Commit 686a81a

Browse files
docs(networking): stop claiming the churn coupling is removed
Restoring the target-side pod deny put a cluster-spanning peer selector back on the webhook endpoint, which is the exact thing the source-side rules were added to take off it. The comments still said the coupling was "structural and removed", and the release note still said deleting a namespace no longer rewrites that endpoint's policy map. Both were true of the previous revision and are not true of this one. What ships now is two denies that fail in different directions: the source-side one keeps enforcing while the webhook endpoint's policy is being recomputed or has overflowed its map, the target-side one keeps enforcing where Cilium is not the service load balancer, and neither subsumes the other. So the coupling is not eliminated, it is made non-load-bearing -- whichever end is degraded the other still denies, and the map writes are writes rather than an outage. Eliminating it outright would mean dropping the target-side rule and accepting a silent fail-open, which is the worse trade. The chainsaw case's header claimed the deletion "briefly made fail-closed admission webhooks unreachable". That is the unconfirmed hypothesis, and the errno in the report contradicts it: EHOSTUNREACH cannot come from a policy denial, which drops after connect() has returned. It now says what the case actually asserts -- both webhooks still admit across a namespace deletion -- rather than claiming to reproduce a cause. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent 0335421 commit 686a81a

4 files changed

Lines changed: 69 additions & 31 deletions

File tree

‎hack/e2e-chainsaw/gateway/chainsaw-test.yaml‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1345,11 +1345,14 @@ spec:
13451345
- apply:
13461346
file: ingress-external-allow.yaml
13471347
---
1348-
# Regression for #3983. The old target-side Cilium policies selected callers
1349-
# from every namespace, so deleting an unrelated namespace regenerated the
1350-
# webhook endpoints' ingress policy and briefly made fail-closed admission
1351-
# webhooks unreachable from the API server. Exercise both affected webhook
1352-
# paths immediately after the deletion, with no retry to hide the outage.
1348+
# Regression for #3983, which reported fail-closed admission webhooks becoming
1349+
# unreachable from the API server while an unrelated namespace was deleted. The
1350+
# mechanism is NOT confirmed: the errno in that report cannot come from a policy
1351+
# denial, and namespace deletion churns service backends as well as Cilium
1352+
# identities. So this case asserts the property the issue is about -- both
1353+
# webhooks still admit across a namespace deletion -- rather than claiming to
1354+
# reproduce a cause. Both affected paths are exercised immediately after the
1355+
# deletion, with no retry to hide a window.
13531356
#
13541357
# Every apply in this test passes through both fail-closed webhooks, so a
13551358
# cluster whose webhooks were ALREADY down fails here in a way that reads

‎packages/system/cilium/tests/kube_proxy_replacement_test.yaml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -37,11 +37,11 @@ suite: cilium owns service load-balancing
3737
# happened, so it enforces whoever performs the translation. Losing
3838
# kubeProxyReplacement therefore degrades the source-side rules and leaves the
3939
# target-side ones standing. That is defence in depth, not a reason to relax
40-
# this pin: the source-side form is what stops a namespace delete from
41-
# recomputing a cluster-spanning selector attached to a fail-closed endpoint,
42-
# and the target-side form is what survives a different service-LB
43-
# implementation. Each carries its own cost, and both are argued in the
44-
# templates next to the rules rather than here.
40+
# this pin: the source-side form is what still denies while the webhook
41+
# endpoint's own policy is being recomputed or has overflowed its map, and the
42+
# target-side form is what survives a different service-LB implementation.
43+
# Neither subsumes the other, each carries its own cost, and both are argued in
44+
# the templates next to the rules rather than here.
4545
#
4646
# The other half of the guard is behavioural rather than static: the
4747
# admission-webhook-deny-still-enforced case in

‎packages/system/ingress-nginx/templates/admission-webhook-egress-policy.yaml‎

Lines changed: 32 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -15,20 +15,40 @@ flaw in the webhook then becomes exploitable by any workload in any
1515
namespace, which is what made CVE-2025-1974 (IngressNightmare) critical
1616
rather than merely serious.
1717

18-
This is deliberately source-side. A target-side ingress policy with a
19-
cluster-wide fromEndpoints selector makes every namespace identity part of
20-
the webhook endpoint's own policy, so every namespace create or delete
21-
recomputes a selector attached to the fail-closed endpoint. Selecting callers
22-
instead keeps the namespace-wide selector on source endpoints, where a
23-
recomputation cannot take admission down.
18+
The deny is expressed from BOTH ends, and the reason is that each end fails in
19+
a direction the other does not.
20+
21+
The source-side specs (the first two) are an egressDeny evaluated on the
22+
caller's own endpoint, so they hold while the webhook endpoint's policy is
23+
being recomputed or has overflowed its own policy map. What they cannot do is
24+
survive a datapath where Cilium is not the service load balancer, because they
25+
name the webhook's backend port and rely on the DNAT happening first.
26+
27+
The target-side spec (the third) is an ingressDeny on the webhook endpoint,
28+
evaluated after whatever DNAT already happened, so it is independent of the
29+
service-LB implementation. What it cannot do is stay off the fail-closed
30+
endpoint's own policy: denying pods needs a cluster-spanning peer, and that is
31+
enumerated per identity on this endpoint.
32+
33+
So be exact about what shipping both means, because the source-side rules were
34+
added to get a cluster-spanning selector OFF the webhook endpoint and the
35+
target-side rule puts one back. This change does not eliminate the
36+
namespace-churn map writes on that endpoint. It makes them non-load-bearing:
37+
whichever end is degraded, the other still denies, and the writes are writes
38+
rather than an outage (see the cost paragraph on the third spec). Removing the
39+
coupling outright would mean dropping the target-side rule and accepting a
40+
silent fail-open wherever Cilium does not own the DNAT, which is the worse
41+
trade.
2442

2543
Stated precisely, because #3983 was reported as a hard outage rather than a
26-
flap: that coupling is structural and is removed here, but it was not
27-
reproduced in isolation as the trigger, and the world deny below -- the other
28-
candidate named in the issue -- is retained on the evidence quoted at the end
29-
of this comment. The source-side form is the better shape either way, and
30-
hack/e2e-chainsaw/gateway/chainsaw-test.yaml now covers both the availability
31-
and the deny it is supposed to provide.
44+
flap: the coupling above is real, but it was not reproduced in isolation as the
45+
trigger, and the symptom the issue reports -- connect: no route to host, i.e.
46+
EHOSTUNREACH -- cannot come from a policy denial at all, since a deny drops
47+
after connect() has returned and the caller sees a timeout. See the PR
48+
description for where that errno does come from. The world deny below, the
49+
other candidate the issue named, is retained on the evidence quoted at the end
50+
of this comment. hack/e2e-chainsaw/gateway/chainsaw-test.yaml covers both the
51+
availability and the deny these rules are supposed to provide.
3252

3353
Only the BACKEND port is denied, and that is sufficient rather than an
3454
oversight. Cilium translates a service to its backends BEFORE egress policy

‎packages/system/kubeovn-webhook/templates/networkpolicy.yaml‎

Lines changed: 24 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -13,15 +13,30 @@ distinction is why it works: packages/system/kubeovn/values.yaml sets
1313
ENABLE_NP: false, so kube-ovn programs no ACLs and a core NetworkPolicy would
1414
be created and never enforced. Cilium enforces its own policies regardless.
1515

16-
The policy is source-side and deny-only. Selecting callers avoids coupling the
17-
webhook endpoint's ingress policy to every namespace identity, where every
18-
namespace create or delete recomputes a selector attached to a fail-closed
19-
endpoint. That coupling is structural and is removed here; it was not
20-
reproduced in isolation as the trigger of #3983, and the world deny below --
21-
the other candidate named in that issue -- is retained on the evidence quoted
22-
further down. enableDefaultDeny stays false, so the policy subtracts one
23-
destination and port from pod egress and leaves everything else exactly as it
24-
was.
16+
Deny-only, and expressed from both ends because each end fails in a direction
17+
the other does not. The source-side spec is an egressDeny on the caller's own
18+
endpoint, so it holds while this webhook's endpoint policy is being recomputed
19+
or has overflowed its map, but it relies on Cilium performing the service DNAT
20+
before egress policy runs. The target-side spec is an ingressDeny on this
21+
webhook's endpoint, evaluated after whatever DNAT already happened, so it is
22+
independent of the service-LB implementation, but denying pods there needs a
23+
cluster-spanning peer enumerated per identity on a fail-closed endpoint.
24+
25+
Be exact about what that means: the source-side spec was added to get a
26+
cluster-spanning selector OFF this endpoint, and the target-side spec puts one
27+
back. Shipping both does not eliminate the namespace-churn map writes here; it
28+
makes them non-load-bearing, because whichever end is degraded the other still
29+
denies, and the writes are writes rather than an outage (see the cost paragraph
30+
on the second spec). The alternative -- dropping the target-side rule -- buys
31+
the coupling back at the price of a silent fail-open wherever Cilium does not
32+
own the DNAT.
33+
34+
That coupling was not reproduced in isolation as the trigger of #3983, and the
35+
errno the issue reports cannot come from a policy denial; see the PR
36+
description. The world deny below, the other candidate that issue named, is
37+
retained on the evidence quoted further down. enableDefaultDeny stays false, so
38+
the policy subtracts one destination and port and leaves everything else
39+
exactly as it was.
2540

2641
Only the BACKEND port is denied, and that is sufficient. Cilium translates a
2742
service to its backends BEFORE egress policy is enforced, so a pod dialling

0 commit comments

Comments
 (0)