Repository navigation
bpf: Introduce wildcard service entries to drop traffic towards unknown protocol/port combinations - #40684
Merged
Conversation
ajmmm
force-pushed
the
pr/slb-drop-wildcard-vip
branch
12 times, most recently
from
July 29, 2025 14:01
bd5a7e7 to
f5a6f60
Compare
ajmmm
marked this pull request as ready for review
July 29, 2025 14:31
joamaki
requested changes
Jul 29, 2025
ajmmm
force-pushed
the
pr/slb-drop-wildcard-vip
branch
from
July 29, 2025 16:02
fb75646 to
adf633a
Compare
joamaki
reviewed
Jul 30, 2025
ajmmm
force-pushed
the
pr/slb-drop-wildcard-vip
branch
from
July 31, 2025 08:50
adf633a to
c391c35
Compare
joamaki
reviewed
Jul 31, 2025
ajmmm
force-pushed
the
pr/slb-drop-wildcard-vip
branch
4 times, most recently
from
August 1, 2025 08:42
8b5e388 to
f408d79
Compare
dylandreimerink
requested changes
Aug 1, 2025
dylandreimerink
left a comment
Member
There was a problem hiding this comment.
This looks great overall. One point of feedback though.
ajmmm
force-pushed
the
pr/slb-drop-wildcard-vip
branch
from
August 1, 2025 11:27
f408d79 to
6b2266c
Compare
…paths. Previously, lbX_lookup_service() would zeroise the key protocol field, such that callers would need to reinstate the key protocol field if their logic re-used the key. Following the previous commit, the lbX_lookup_service() routines will now only zeroise the key protocol (and dport) fields if an external wild-card entry is returned. Thus, the restoration of key values can be removed. This commit also removes lbX_key_set_protocol() helpers as they became broadly unnecessary. Signed-off-by: Alasdair McWilliam <[email protected]>
This set of tests verifies that if we receive a packet towards a VIP on either an unknown destination port, or unknown protocol, the data plane will correctly drop if a wildcard entry is present in the service map. Signed-off-by: Alasdair McWilliam <[email protected]>
This change extends the nodePortAddrKey structure to include the L4 Protocol number of an address, such that addresses with differing addresses will be better distributed in the nodePortAddrByPort map. This required the introduction of a helper routine to translate the L4Addr protocol field into the real IANA number. Signed-off-by: Alasdair McWilliam <[email protected]>
Introduce logic in bpf_reconciler to map frontend service entries to wildcard service entries, and program them in the data path, for LoadBalancer and ClusterIP frontends of external scope. This will ensure the data path drops traffic towards unknown destination ports and/or protocols. Wildcard state is tracked as a map (indexed by IP Address) that contains a slice of Frontend ServiceIDs. The first entry that requires a wildcard will trigger programming of the data path entry. The last entry that depends on a wildcard entry to be removed will trigger removal of the wildcard entry. Signed-off-by: Alasdair McWilliam <[email protected]> Signed-off-by: Mikael Knutsson <[email protected]>
Update the unit tests for loadbalancer and ciliumenvoyconfig to include wildcard entries. Signed-off-by: Alasdair McWilliam <[email protected]>
Remove lb_v4_add_mixed_proto_service_with_flags() as it no longer makes sense. Signed-off-by: Alasdair McWilliam <[email protected]>
ajmmm
force-pushed
the
pr/slb-drop-wildcard-vip
branch
from
August 27, 2025 09:28
fe966e5 to
abf0fec
Compare
Member
Author
|
/test |
Member
Author
|
ci-ginkgo failures is likely a flake - happens across different tests on main, eg: |
9 of 17 tasks
9 of 12 tasks
Closed
2 of 3 tasks
2 of 3 tasks
lllamnyp
added a commit
to cozystack/cozystack
that referenced
this pull request
Apr 8, 2026
) ## What this PR does Adds the `service.kubernetes.io/service-proxy-name: "cozy-proxy"` label to VM LoadBalancer services when `external: true`. This tells Cilium to completely ignore the service — no BPF entries, no DNAT, no wildcard drops. Routing continues to work through kube-ovn (`networking.cozystack.io/wholeIP` annotation) and MetalLB L2 advertisement is unaffected. **This fixes two issues:** 1. **Inter-tenant connectivity via public LB IPs**: With `kube-proxy-replacement: true`, Cilium DNATs traffic destined to LB IPs (on ports declared in the Service) to the backend pod IP *before* evaluating network policies. The `CiliumClusterwideNetworkPolicy` then sees a cross-tenant pod-to-pod flow and blocks it. Ports not in the Service pass as "world" traffic and work fine — so does ICMP. With this label, Cilium doesn't DNAT anything, all traffic is classified as "world", and the existing `allow-external-communication` policy allows it. Pod IP isolation between tenants is preserved. 2. **WholeIP broken on Cilium 1.19+** (#2327): Cilium 1.19 introduced wildcard service drop entries (`cilium/cilium#40684`) that drop all traffic to a LB IP on ports not declared in the Service, before it reaches netfilter/cozy-proxy. With this label, no BPF entries are created at all, so there's nothing to drop. **Tested on our lab cluster:** - Cilium 1.18.6: inter-tenant via LB IP ✅, pod IP blocked ✅, external SSH ✅, WholeIP all ports ✅ - Cilium 1.19.1: same results ✅, no wildcard drops ✅ - MetalLB L2 advertisement: unaffected ✅ - Other services without the label: unaffected ✅ **Note:** The `externalPorts` / PortList filtering is currently not enforced at the network level (the `wholeIP` annotation causes kube-ovn to route all ports regardless). This was the case before this change and is a separate concern — per-VM CiliumNetworkPolicies could be re-introduced for that (ref: `4cbf562a`, #1601). ```release-note [virtual-machine] Fix inter-tenant connectivity via public LB IPs and WholeIP on Cilium 1.19+ ```
This was referenced Apr 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR to resolve a forwarding bug in data path.
This work is derived from work undertaken by Mikael Knutsson [email protected] so has been credited in the relevant commit logs.
Summary of issue
Given a configured LoadBalancer or Cluster IP on a specific L4 destination port and protocol, ingress traffic towards the IP on an alternate destination port or protocol will not match eBPF service map lookup, so traffic be punted back to the network. Where the upstream network re-delivers the traffic back to Cilium, a packet loop is formed.
Summary of Changes
Release Notes: