v0.2.0 degraded #8

Activity

1. kvaps commented on Sep 9, 2025

   

   Andrei Kvapil (kvaps)

   on Sep 9, 2025

   Member

   More actions

   Hey, thanks for report. I think we can introduce an option to choice the method.
   Could you please provide your calico configuration for better understanding cause of the problem?
2. nikitos commented on Sep 12, 2025

   

   Andriyanov Nikita (nikitos)

   on Sep 12, 2025

   Last edited by nikitos

   Author

   More actions

   all just default on rancher rke2 cluster. only added metallb helm and this: (could not make it looks better :( )

   apiVersion: metallb.io/v1beta1 kind: IPAddressPool metadata: name: real-pool namespace: metallb-system spec: addresses: - <ip>

   apiVersion: metallb.io/v1beta2 kind: BGPPeer metadata: name: sample namespace: metallb-system spec: myASN: 64512 peerASN: 65000 peerAddress: 10.20.30.1

   apiVersion: metallb.io/v1beta1 kind: BGPAdvertisement metadata: name: example namespace: metallb-system spec: ipAddressPools: - real-pool
3. kvaps commented on Apr 21, 2026

   

   Andrei Kvapil (kvaps)

   on Apr 21, 2026

   Member

   More actions

   Hi Andriyanov Nikita (@nikitos), thanks for the details.

   This is very likely the same class of issue we just fixed on the cozystack side in cozystack/cozystack#2357.

   Since v0.2.0 cozy-proxy performs stateless NAT in the raw table (early_snat chain), which runs before conntrack. In a plain kube-proxy (iptables/IPVS) setup — which is exactly what RKE2 + Calico uses — kube-proxy also installs its own rules for LoadBalancer services, and those rules end up interfering with cozy-proxy's NAT (most visibly on the outbound path, which matches your "outbound traffic does no SNAT back" symptom). In v0.1.4 we worked in the nat table, so the ordering happened to line up differently and the conflict didn't surface.

   The standard Kubernetes way to tell kube-proxy "ignore this service, someone else handles it" is the service.kubernetes.io/service-proxy-name label. Could you try adding this label to your LoadBalancer service and retest?

   apiVersion: v1
   kind: Service
   metadata:
     name: example-service
     labels:
       service.kubernetes.io/service-proxy-name: "cozy-proxy"
     annotations:
       networking.cozystack.io/wholeIP: "true"
   spec:
     type: LoadBalancer
     allocateLoadBalancerNodePorts: false
     externalTrafficPolicy: Local
     ports:
     - port: 65535
     selector:
       app: nginx

   With this label:

   • kube-proxy creates no iptables/IPVS entries for the service at all
   • MetalLB BGP advertisement is unaffected
   • cozy-proxy's raw-table rules become the only thing doing NAT, no conflicts

   If that resolves it for you, I think we should document this as the required setup for the plain-kube-proxy case (and possibly default to it in the chart / examples). If it doesn't help, please share nft list ruleset and iptables-save from the node while traffic is flowing — we'll dig further.
4. Andrei Kvapil (kvaps)

   added a commit that references this issue on Apr 27, 2026

   feat: select services by service-proxy-name label

   ...

   073f650