Skip to content

Commit d2361f3

Browse files
authored
[virtual-machine] Exclude external VM services from Cilium BPF LB (#2357)
## What this PR does Adds the `service.kubernetes.io/service-proxy-name: "cozy-proxy"` label to VM LoadBalancer services when `external: true`. This tells Cilium to completely ignore the service — no BPF entries, no DNAT, no wildcard drops. Routing continues to work through kube-ovn (`networking.cozystack.io/wholeIP` annotation) and MetalLB L2 advertisement is unaffected. **This fixes two issues:** 1. **Inter-tenant connectivity via public LB IPs**: With `kube-proxy-replacement: true`, Cilium DNATs traffic destined to LB IPs (on ports declared in the Service) to the backend pod IP *before* evaluating network policies. The `CiliumClusterwideNetworkPolicy` then sees a cross-tenant pod-to-pod flow and blocks it. Ports not in the Service pass as "world" traffic and work fine — so does ICMP. With this label, Cilium doesn't DNAT anything, all traffic is classified as "world", and the existing `allow-external-communication` policy allows it. Pod IP isolation between tenants is preserved. 2. **WholeIP broken on Cilium 1.19+** (#2327): Cilium 1.19 introduced wildcard service drop entries (`cilium/cilium#40684`) that drop all traffic to a LB IP on ports not declared in the Service, before it reaches netfilter/cozy-proxy. With this label, no BPF entries are created at all, so there's nothing to drop. **Tested on our lab cluster:** - Cilium 1.18.6: inter-tenant via LB IP ✅, pod IP blocked ✅, external SSH ✅, WholeIP all ports ✅ - Cilium 1.19.1: same results ✅, no wildcard drops ✅ - MetalLB L2 advertisement: unaffected ✅ - Other services without the label: unaffected ✅ **Note:** The `externalPorts` / PortList filtering is currently not enforced at the network level (the `wholeIP` annotation causes kube-ovn to route all ports regardless). This was the case before this change and is a separate concern — per-VM CiliumNetworkPolicies could be re-introduced for that (ref: `4cbf562a`, #1601). ```release-note [virtual-machine] Fix inter-tenant connectivity via public LB IPs and WholeIP on Cilium 1.19+ ```
2 parents 38624b7 + 026b1c7 commit d2361f3

1 file changed

Lines changed: 1 addition & 0 deletions

File tree

‎packages/apps/vm-instance/templates/service.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ metadata:
77
apps.cozystack.io/user-service: "true"
88
{{- include "virtual-machine.labels" . | nindent 4 }}
99
{{- if .Values.external }}
10+
service.kubernetes.io/service-proxy-name: "cozy-proxy"
1011
annotations:
1112
networking.cozystack.io/wholeIP: "true"
1213
{{- end }}

0 commit comments

Comments
 (0)