Trying the new nftables backend, it fails on EL 9.4
# nft --version
nftables v1.0.9 (Old Doc Yak #3)
# uname -r
5.14.0-427.40.1.el9_4.x86_64
conf
{
"type": "portmap",
"capabilities": {"portMappings": true},
"backend": "nftables",
"conditionsV4": ["ip", "daddr", "!=", "{ 127.0.0.0/8, 198.19.254.254 }"]
},
Error
Nov 05 17:15:34 atsc2 kubelet[4258]: E1105 17:15:34.352838 4258 remote_runtime.go:193] "RunPodSandbox from runtime service failed" err=<
Nov 05 17:15:34 atsc2 kubelet[4258]: rpc error: code = Unknown desc = failed to setup network for sandbox "9fa1b2975bf1dc51c273d7b5e53e7f4500faa1f931d8552e2761fd534b37d621": plugin type="multus-cni" name="multus-cni-network" failed (add): [traefik/appliance-ingress-traefik-698cd97568-5xg88/3908060c-0419-481c-865d-3230b95c3a84:mgmt]: error adding container to network "mgmt": plugin type="portmap" failed (add): unable to set up nftables rules for port mappings: /dev/stdin:13:1-182: Error: Could not process rule: Operation not supported
Nov 05 17:15:34 atsc2 kubelet[4258]: add rule ip cni_hostport hostports ip protocol tcp th dport 81 dnat ip addr . port to 198.18.1.194 . 10081 comment "9fa1b2975bf1dc51c273d7b5e53e7f4500faa1f931d8552e2761fd534b37d621"
Nov 05 17:15:34 atsc2 kubelet[4258]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 05 17:15:34 atsc2 kubelet[4258]: /dev/stdin:14:1-184: Error: Could not process rule: Operation not supported
Nov 05 17:15:34 atsc2 kubelet[4258]: add rule ip cni_hostport hostports ip protocol tcp th dport 8444 dnat ip addr . port to 198.18.1.194 . 18444 comment "9fa1b2975bf1dc51c273d7b5e53e7f4500faa1f931d8552e2761fd534b37d621"
Nov 05 17:15:34 atsc2 kubelet[4258]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 05 17:15:34 atsc2 kubelet[4258]: /dev/stdin:15:1-182: Error: Could not process rule: Operation not supported
Nov 05 17:15:34 atsc2 kubelet[4258]: add rule ip cni_hostport hostports ip protocol tcp th dport 82 dnat ip addr . port to 198.18.1.194 . 10082 comment "9fa1b2975bf1dc51c273d7b5e53e7f4500faa1f931d8552e2761fd534b37d621"
Nov 05 17:15:34 atsc2 kubelet[4258]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 05 17:15:34 atsc2 kubelet[4258]: /dev/stdin:16:1-184: Error: Could not process rule: Operation not supported
Nov 05 17:15:34 atsc2 kubelet[4258]: add rule ip cni_hostport hostports ip protocol tcp th dport 8445 dnat ip addr . port to 198.18.1.194 . 18445 comment "9fa1b2975bf1dc51c273d7b5e53e7f4500faa1f931d8552e2761fd534b37d621"
Nov 05 17:15:34 atsc2 kubelet[4258]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 05 17:15:34 atsc2 kubelet[4258]: /dev/stdin:17:1-182: Error: Could not process rule: Operation not supported
Nov 05 17:15:34 atsc2 kubelet[4258]: add rule ip cni_hostport hostports ip protocol tcp th dport 83 dnat ip addr . port to 198.18.1.194 . 10083 comment "9fa1b2975bf1dc51c273d7b5e53e7f4500faa1f931d8552e2761fd534b37d621"
Nov 05 17:15:34 atsc2 kubelet[4258]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 05 17:15:34 atsc2 kubelet[4258]: /dev/stdin:18:1-184: Error: Could not process rule: Operation not supported
Nov 05 17:15:34 atsc2 kubelet[4258]: add rule ip cni_hostport hostports ip protocol tcp th dport 8446 dnat ip addr . port to 198.18.1.194 . 18446 comment "9fa1b2975bf1dc51c273d7b5e53e7f4500faa1f931d8552e2761fd534b37d621"
Nov 05 17:15:34 atsc2 kubelet[4258]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 05 17:15:34 atsc2 kubelet[4258]: /dev/stdin:19:1-168: Error: Could not process rule: Operation not supported
Nov 05 17:15:34 atsc2 kubelet[4258]: add rule ip cni_hostport masquerading ip saddr 198.18.1.194 ip daddr 198.18.1.194 masquerade comment "9fa1b2975bf1dc51c273d7b5e53e7f4500faa1f931d8552e2761fd534b37d621"
Nov 05 17:15:34 atsc2 kubelet[4258]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 05 17:15:34 atsc2 kubelet[4258]: /dev/stdin:20:1-165: Error: Could not process rule: Operation not supported
Nov 05 17:15:34 atsc2 kubelet[4258]: add rule ip cni_hostport masquerading ip saddr 127.0.0.1 ip daddr 198.18.1.194 masquerade comment "9fa1b2975bf1dc51c273d7b5e53e7f4500faa1f931d8552e2761fd534b37d621"
Nov 05 17:15:34 atsc2 kubelet[4258]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nov 05 17:15:34 atsc2 kubelet[4258]: >
Using strace, the rules are:
add table ip cni_hostport { comment "CNI portmap plugin" ; }
add chain ip cni_hostport hostports
add chain ip cni_hostport hostip_hostports
add chain ip cni_hostport input { type nat hook input priority -100 ; }
flush chain ip cni_hostport input
add rule ip cni_hostport input ip daddr != { 127.0.0.0/8, 198.19.254.254 } jump hostip_hostports
add rule ip cni_hostport input ip daddr != { 127.0.0.0/8, 198.19.254.254 } jump hostports
add chain ip cni_hostport output { type nat hook output priority -100 ; }
flush chain ip cni_hostport output
add rule ip cni_hostport output ip daddr != { 127.0.0.0/8, 198.19.254.254 } jump hostip_hostports
add rule ip cni_hostport output ip daddr != { 127.0.0.0/8, 198.19.254.254 } fib daddr type local jump hostports
add chain ip cni_hostport masquerading { type nat hook postrouting priority 100 ; }
add rule ip cni_hostport hostports ip protocol tcp th dport 81 dnat ip addr . port to 198.18.0.56 . 10081 comment "4b624583ded419060960435aec2264eda03b1b0ac54c009a2ac98dc8492bf729"
add rule ip cni_hostport hostports ip protocol tcp th dport 8444 dnat ip addr . port to 198.18.0.56 . 18444 comment "4b624583ded419060960435aec2264eda03b1b0ac54c009a2ac98dc8492bf729"
add rule ip cni_hostport hostports ip protocol tcp th dport 82 dnat ip addr . port to 198.18.0.56 . 10082 comment "4b624583ded419060960435aec2264eda03b1b0ac54c009a2ac98dc8492bf729"
add rule ip cni_hostport hostports ip protocol tcp th dport 8445 dnat ip addr . port to 198.18.0.56 . 18445 comment "4b624583ded419060960435aec2264eda03b1b0ac54c009a2ac98dc8492bf729"
add rule ip cni_hostport hostports ip protocol tcp th dport 83 dnat ip addr . port to 198.18.0.56 . 10083 comment "4b624583ded419060960435aec2264eda03b1b0ac54c009a2ac98dc8492bf729"
add rule ip cni_hostport hostports ip protocol tcp th dport 8446 dnat ip addr . port to 198.18.0.56 . 18446 comment "4b624583ded419060960435aec2264eda03b1b0ac54c009a2ac98dc8492bf729"
add rule ip cni_hostport masquerading ip saddr 198.18.0.56 ip daddr 198.18.0.56 masquerade comment "4b624583ded419060960435aec2264eda03b1b0ac54c009a2ac98dc8492bf729"
add rule ip cni_hostport masquerading ip saddr 127.0.0.1 ip daddr 198.18.0.56 masquerade comment "4b624583ded419060960435aec2264eda03b1b0ac54c009a2ac98dc8492bf729"
I think instead of
add chain ip cni_hostport input { type nat hook input priority -100 ; }
we should use
add chain ip cni_hostport input { type nat hook prerouting priority -100 ; }
(and rename input to prerouting)
Trying the new nftables backend, it fails on EL 9.4
conf
Error
Using strace, the rules are:
I think instead of
we should use
(and rename input to prerouting)