I switched our cluster over from metallb to the l2 announcer in a single evening with 0-downtime. This is a very awesome feature!
However, it is issuing IP's outside of the IP pool...
Example pool:
apiVersion: cilium.io/v2alpha1
kind: CiliumLoadBalancerIPPool
metadata:
name: pool
spec:
cidrs:
- cidr: 123.123.123.123/32
- cidr: 123.123.123.125/32
disabled: falseI'm seeing it issue IP's to services (LB type) to 123.123.123.122, for example. Deleting the status doesn't cause it to reissue the IP address from the pool (in fact, I couldn't figure out how to invalidate the assignment), and had to force an actual IP using the io.cilium/lb-ipam-ips annotation.
However, when setting that, I got the following status:
No pool exists with a CIDR containing '123.123.123.123'
I had to set the /32 cidr on the annotation, which doesn't match the documentation here: https://docs.cilium.io/en/stable/network/lb-ipam/#requesting-ips
The only thing I can think of is that I'm using /32's because my IP block has some 'holes' in it (external services) that I don't want to be included in the pool, thus I had to enumerate the entire block using /32's.
# cilium version
cilium-cli: v0.15.8 compiled with go1.21.0 on linux/amd64
cilium image (default): v1.14.1
cilium image (stable): v1.14.2
cilium image (running): 1.14.2
# uname -a
Linux capital 5.15.0-84-generic #93-Ubuntu SMP Tue Sep 5 17:16:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
# kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.6+k3s1", GitCommit:"bd04941a294793ec92e8703d5e5da14107902e88", GitTreeState:"clean", BuildDate:"2023-09-20T23:05:58Z", GoVersion:"go1.20.8", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.6+k3s1", GitCommit:"bd04941a294793ec92e8703d5e5da14107902e88", GitTreeState:"clean", BuildDate:"2023-09-20T23:05:58Z", GoVersion:"go1.20.8", Compiler:"gc", Platform:"linux/amd64"}
cilium-sysdump-20230924-231254.zip
I switched our cluster over from metallb to the l2 announcer in a single evening with 0-downtime. This is a very awesome feature!
However, it is issuing IP's outside of the IP pool...
Example pool:
I'm seeing it issue IP's to services (LB type) to 123.123.123.122, for example. Deleting the status doesn't cause it to reissue the IP address from the pool (in fact, I couldn't figure out how to invalidate the assignment), and had to force an actual IP using the
io.cilium/lb-ipam-ipsannotation.However, when setting that, I got the following status:
I had to set the
/32cidr on the annotation, which doesn't match the documentation here: https://docs.cilium.io/en/stable/network/lb-ipam/#requesting-ipsThe only thing I can think of is that I'm using /32's because my IP block has some 'holes' in it (external services) that I don't want to be included in the pool, thus I had to enumerate the entire block using /32's.
cilium-sysdump-20230924-231254.zip