Skip to content

bpf: host traffic to a local pod is classified as world under CNI chaining since the SNAT conflict generalization #49208

Open
cozystack/cozystack
#3343

Description

@lexfrei

Is there an existing issue for this?

  • I have searched the existing issues

Version

equal or higher than 1.20.2 and lower than v1.21.0

What happened?

Since v1.20.0, with kube-ovn in generic-veth chaining mode, a connection from a node's host network namespace to a pod on the same node reaches the pod with the world identity instead of host. On v1.19.x the same connection arrives as host. A connection from the host to a pod on another node is still classified correctly, because the remote node takes the identity from the ipcache.

What breaks is everything that depends on the source being host. The reserved:host allow that the agent synthesizes for every default-deny endpoint (allow-localhost resolves to always on Kubernetes) stops matching, so kubelet probes time out under any ingress policy that does not also admit world. An ingress deny on world drops kube-apiserver when it calls an admission webhook whose backend runs on its own node. Admission then times out in bursts, only on nodes that host a webhook replica.

I traced the mark of one SYN from the node IP to a local pod with pwru (skb->mark per kernel function):

ip_output              mark=0xc00  (MARK_MAGIC_HOST, set by iptables)
__dev_queue_xmit       ovn0  0xc00
dev_hard_start_xmit    ovn0  0x300 (MARK_MAGIC_SNAT_DONE)
ovs_vport_receive      ovn0  0x300
ovs_vport_send         ...   0x300 -> host side of the pod veth
sk_skb_reason_drop     cil_to_container: Policy denied, identity world

The only hook between the two ovn0 lines is tc egress, i.e. cil_to_netdev. In chaining mode host traffic to a local pod leaves through the CNI's host interface (ovn0 for kube-ovn), and bpf_host is attached there. Commit 990f79b (#43955, "bpf: Generalize SNAT conflict detection") changed nodeport_has_nat_conflict_ipv4()/_ipv6() in bpf/lib/nodeport_egress.h to return true for any packet carrying MARK_MAGIC_HOST on any bpf_host device. Before it, the check only fired on the direct-routing device with the direct-routing source address, and in this setup the direct-routing device is the physical NIC, not ovn0. The conflict path then runs snat_v4_nat(), which only reserves the port, and calls ctx_snat_done_set(). That replaces the MARK_MAGIC_HOST_MASK bits with MARK_MAGIC_SNAT_DONE, so inherit_identity_from_host() in to-container falls through to WORLD_ID. The ipcache fallback in tail_ipv4_to_endpoint() cannot recover it, since it deliberately ignores a HOST_ID result for a reserved source.

How can we reproduce the issue?

Cilium v1.20.x with cni.chainingMode=generic-veth, cni.chainingTarget=kube-ovn, kubeProxyReplacement=true, native routing, BPF masquerade off. Put a CiliumNetworkPolicy with only ingress: [{fromEntities: [cluster]}] on a pod that has an HTTP readiness probe. The pod never becomes Ready, and cilium-dbg monitor --type drop on its endpoint shows the probe SYNs dropped with identity world. The same setup on v1.19.x is Ready. Setting enable-identity-mark=true or bpf-lb-sock-hostns-only=false does not change it.

Cilium Version

v1.20.2 (also present on main as of 2026-10-06)

Kernel Version

6.18.38 (Talos Linux v1.13.6)

Kubernetes Version

v1.36.2

Regression

Yes. Works on v1.19.5, broken since v1.20.0. Introduced by 990f79b (#43955, "bpf: Generalize SNAT conflict detection").

Sysdump

No response

Relevant log output

Anything else?

No response

Cilium Users Document

  • Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

  • I agree to follow this project's Code of Conduct

Activity

  1. added theissue type on Oct 6, 2026
  2. added
    kind/bugThis is a bug in the Cilium logic.
    needs/triageThis issue requires triaging to establish severity and affected area.
    kind/community-reportThis was reported by a user in the Cilium community, eg via Slack.
    on Oct 6, 2026
  3. adityamukesh commented on Oct 7, 2026

    @adityamukesh

    Hi maintainers, I would like to work on this issue.

    🔍 Analysis & Root Cause

    Following the SNAT conflict generalization changes in recent releases, local host-to-pod traffic under CNI chaining mode is erroneously classified as world traffic in BPF datapath endpoint lookups. Because CNI chaining involves secondary interfaces, endpoint identification in bpf/bpf_lxc.c falls back to the world security identity rather than identifying the local host security identity.

    🛠️ Proposed Implementation Plan

    1. Datapath Classification Check: In bpf/bpf_lxc.c, refine the host source IP and tunnel verification when running under chained CNI modes.
    2. Identity Preservation: Ensure host-originated traffic retains HOST_ID security identity before applying ingress policies.

    🧪 Testing Plan

    • Run datapath connectivity tests in CI reproducing CNI chaining scenarios and verifying policy enforcement between host and local pods.

    Could you please assign this issue to me? Thanks!

  4. added
    area/datapathImpacts bpf/ or low-level forwarding details, including map management and monitor messages.
    affects/v1.20This issue affects v1.20 branch
    and removed
    needs/triageThis issue requires triaging to establish severity and affected area.
    on Oct 7, 2026
  5. added 3 commits that reference this issue on Oct 8, 2026
    1966b58
    e046227
    4ba3e28
  6. lexfrei commented on Oct 10, 2026

    @lexfrei
    ContributorAuthor

    Reopening. This was closed by mistake: a downstream PR mentioned it next to the word "fixes" and GitHub closed it on merge. The fix is still pending in #49232.

  7. added a commit that references this issue on Oct 13, 2026
    d376073
  8. added a commit that references this issue on Nov 11, 2026
    65a4036
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    affects/v1.20This issue affects v1.20 brancharea/datapathImpacts bpf/ or low-level forwarding details, including map management and monitor messages.kind/bugThis is a bug in the Cilium logic.kind/community-reportThis was reported by a user in the Cilium community, eg via Slack.

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions