hostFirewall drops Neighbor Discovery Protocol(NDP/ICMP v6) packets between host and leaf router using Link Local Addresses(LLA)

Is there an existing issue for this?

I have searched the existing issues

What happened?

My host uses Link Local Address(LLA) to do BGP peering with leaf routers, with hostFirewall disabled in cilium, everything works fine, but once I enabled hostFirewall, cilium starts to drop LLA based NDP traffics.

I suppose this is caused by Cilium's built-in protection, described in this blog ?

But my use case is a valid one, to achieve BGP routing on the host by BGP Unnumbered via LLA. Is it possible to configure Cilium in a way to support that ?

My cilium helm values(ipv6 is disabled):

hostFirewall:
  enabled: true
nodePort:
  directRoutingDevice: "eth+"
ipam:
  mode: kubernetes
bpf:
  masquerade: true
devices: "eth+"
tunnel: vxlan
securityContext:
  privileged: true
hostPort:
  enabled: true
hubble:
  enabled: true
  relay:
    enabled: true
  ui:
    enabled: true
kubeProxyReplacement: true
k8sServiceHost: "10.10.10.10"
k8sServicePort: "6443"
enableXTSocketFallback: false
cni:
  configMap: cni-configuration
extraConfig:
  derive-masquerade-ip-addr-from-device: dummy0
operator:
  replicas: 1
sctp:
  enabled: true

output of cilium monitor -t drop

# cilium monitor -t drop
Listening for events on 64 CPUs with 64x4096 of shared memory
Press Ctrl-C to quit
time="2024-06-14T12:32:08Z" level=info msg="Initializing dissection cache..." subsys=monitor
xx drop (Unsupported L3 protocol) flow 0x0 to endpoint 0, ifindex 9, file bpf_host.c:1406, , identity unknown->unknown: fe80::526b:4bff:feb6:d437 -> ff02::1:ff6d:122 NeighborSolicitation
xx drop (Unsupported L3 protocol) flow 0x99ad3e17 to endpoint 0, ifindex 9, file bpf_host.c:1189, , identity unknown->unknown: fe80::b24f:13ff:fe6d:122 -> ff02::1:ffb6:d437 NeighborSolicitation
xx drop (Unsupported L3 protocol) flow 0xe8c8fc7f to endpoint 0, ifindex 8, file bpf_host.c:1406, , identity unknown->unknown: fe80::526b:4bff:feb6:d436 -> ff02::1 RouterAdvertisement
xx drop (Unsupported L3 protocol) flow 0xbd40f179 to endpoint 0, ifindex 8, file bpf_host.c:1189, , identity unknown->unknown: fe80::b24f:13ff:fe6d:a9a2 -> ff02::1:ffb6:d436 NeighborSolicitation
xx drop (Unsupported L3 protocol) flow 0x0 to endpoint 0, ifindex 9, file bpf_host.c:1406, , identity unknown->unknown: fe80::526b:4bff:feb6:d437 -> ff02::1:ff6d:122 NeighborSolicitation
xx drop (Unsupported L3 protocol) flow 0x6379c6d0 to endpoint 0, ifindex 9, file bpf_host.c:1406, , identity unknown->unknown: fe80::526b:4bff:feb6:d437 -> ff02::1 RouterAdvertisement

Cilium Version

1.15.6

Kernel Version

6.6.32-talos

Kubernetes Version

v1.28.10

Regression

No response

Sysdump

No response

Relevant log output

No response

Anything else?

No response

Cilium Users Document

Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

I agree to follow this project's Code of Conduct

I know NodePort (kubeProxyReplacement=true) has this issue, but not aware of the hostFirewall. I guess in your use case (BGP Unnumbered), you want to accept RA, and I know that hostFirewall is intended to accept it.

Could you try to disable KPR and see if that solve the problem?

@YutaroHayakawa

thx @YutaroHayakawa , I tried with kubeProxyReplacement=false, it does not help, still those NDP/ICMP v6 and BGP(TCP/179) packets are dropped.

Hmm, ICMP is understandable, but I'm not sure why BGP packets are dropped. Are you allowing BGP traffic in HostFirewall? BGP speaker is in the host namespace (part of Cilium agent), so you need to allow it as well.

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

Hmm, ICMP is understandable, but I'm not sure why BGP packets are dropped. Are you allowing BGP traffic in HostFirewall? BGP speaker is in the host namespace (part of Cilium agent), so you need to allow it as well.

I have following policy, but it does not help:

apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: cilium-host-firewall
spec:
  nodeSelector:
    matchLabels:
      node-type: master
  ingress:
  - toPorts:
    - ports:
      - port: "179"
        protocol: TCP
      - port: "6443"
        protocol: TCP
      - port: "50000"
        protocol: TCP
  - icmps:
    - fields:
      - type: RouterSolicitation
        family: IPv6
      - type: RouterAdvertisement
        family: IPv6
      - type: NeighborSolicitation
        family: IPv6
      - type: NeighborAdvertisement
        family: IPv6

Actually to make BGP unnumered to work, NDP/ICMP v6 has to work first, if they don't, we can not get to the BGP session creation phase(TCP/179). So the problem is cilium drops e.g. RouterAdvertisement packet in both directions:

# RA sent from host to leaf router
xx drop (Unsupported L3 protocol) flow 0x496dd881 to endpoint 0, ifindex 8, file bpf_host.c:1604, , identity unknown->unknown: fe80::526b:4bff:feb6:d436 -> ff02::1 RouterAdvertisement
xx drop (Unsupported L3 protocol) flow 0x91ae139 to endpoint 0, ifindex 9, file bpf_host.c:1604, , identity unknown->unknown: fe80::526b:4bff:feb6:d437 -> ff02::1 RouterAdvertisement

# RA sent from leaf router to host
xx drop (Unsupported L3 protocol) flow 0xce7d94d5 to endpoint 0, ifindex 9, file bpf_host.c:1210, , identity unknown->unknown: fe80::b24f:13ff:fe6d:122 -> ff02::1 RouterAdvertisement
xx drop (Unsupported L3 protocol) flow 0xf7a4b2bd to endpoint 0, ifindex 8, file bpf_host.c:1210, , identity unknown->unknown: fe80::b24f:13ff:fe6d:a9a2 -> ff02::1 RouterAdvertisement

I found there is a config enable-ipv6-ndp, but it requires enable-ipv6: "true" to work, we are running the cluster with ipv4 only, can not enable ipv6 at this stage yet.

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

This issue has not seen any activity since it was marked stale.
Closing.

I have the same problem on Cilium 1.18.6 and I get following error on FRR:
Tx RA failed, socket 12 error 105 (No buffer space available)

bpf: host: pass IPv6 to the stack when the IPv6 datapath is disabled

fix(cilium): pass IPv6 to kernel stack when IPv6 datapath is disabled (…

bpf: host: pass IPv6 to the stack when the IPv6 datapath is disabled

Navigation Menu

Repository navigation

hostFirewall drops Neighbor Discovery Protocol(NDP/ICMP v6) packets between host and leaf router using Link Local Addresses(LLA) #33155

Is there an existing issue for this?

What happened?

Cilium Version

Kernel Version

Kubernetes Version

Regression

Sysdump

Relevant log output

Anything else?

Cilium Users Document

Code of Conduct

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

hostFirewall drops Neighbor Discovery Protocol(NDP/ICMP v6) packets between host and leaf router using Link Local Addresses(LLA) #33155

Description

Is there an existing issue for this?

What happened?

Cilium Version

Kernel Version

Kubernetes Version

Regression

Sysdump

Relevant log output

Anything else?

Cilium Users Document

Code of Conduct

Activity

YutaroHayakawa commented on Jun 24, 2024

abckey commented on Jun 24, 2024

YutaroHayakawa commented on Jul 1, 2024

github-actions commented on Aug 31, 2024

abckey commented on Sep 13, 2024

github-actions commented on Dec 8, 2024

github-actions commented on Dec 23, 2024

nfarhadian commented on Mar 26, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions