Use the pool name as a key into the per-node in-use pools map instead of
looping over each key to find a match.

Signed-off-by: Fabio Falzoi <[email protected]>

Rely on the same code path to both add a new pool and to update an
existing one, removing the old function that was taking care of the
insertion separately.

This commit is not meant to change anything in the pools handling logic,
but it allows to apply additional checks while handling a pool upsertion
in a single place, being a new pool or an update of an existing one.

Signed-off-by: Fabio Falzoi <[email protected]>

Whenever a CIDR is removed from a pool or a pool is removed altogether,
there might be nodes that are still using those CIDRs. Previously, the
operator would emit a warning, but would not track those CIDRs, leading
to issues in case of a subsequent upsertion of the same CIDRs.

Specifically, consider the following sequence of events:

- CiliumPodIPPool is added with a CIDR "10.10.0.0/16" and a mask size
  equal to 24
- A CiliumNode requests IPs from that pool, and got assigned the CIDR
  "10.10.1.0/24"
- The CiliumPodIPPool is deleted. The operator emits a warning, but the
  allocated CIDR is released from its internal allocation maps.
  Nonetheless, the CIDR is still in use by the node and there might be
  pods scheduled on it with IPs belonging to that CIDR.
- The previously deleted CiliumPodIPPool is added again. No more actions
  are taken by the operator, that is, the operator thinks "10.10.1.0/24"
  is free to use.
- A new node joins the cluster and requests a CIDR allocation from the
  same pool. The CIDR "10.10.1.0/24" is assigned to it, ultimately
  leading to multiple pods on different nodes having the same IPs.

To avoid the above scenario we add the concept of "orphan CIDRs": an
orphan CIDR is a CIDR still allocated to the node but not listed in any
pools anymore, either because the parent CiliumPodIPPool has been
removed or shrunk.

When the operator detects that a CIDR has been orphaned, it stores it in
a separate map. The map will be checked at each subsequent
CiliumPodIPPool upsert event: if the original pool containing the orphan
CIDR is restored, the CIDR is marked as allocated to reconcile the
allocations status in the operator internal maps with the effective one.

Fixes: cilium#41986

Signed-off-by: Fabio Falzoi <[email protected]>

Since orphan CIDRs are stored in a map, we lose track of them after an
operator restart. Therefore, at startup, after the allocator received
all the CiliumPodIPPool resources from k8s, it will mark as orphan all
the CIDRs that are "allocated" in the CiliumNodes.

This allows to reconstruct the status of the orphan CIDRs after the
restart, even if a CiliumPodIPPool was deleted but its CIDRs were still
in use by nodes.

Signed-off-by: Fabio Falzoi <[email protected]>

Add a test to verify that a newly added pool with a name different than
the one originating the orphan CIDRs cannot "steal" those CIDRs.

Since we always keep orphan CIDRs untouched in the
spec.IPAM.Pools.Allocated section of the CiliumNode, we cannot reassign
those CIDRs to other pools, otherwise the allocations reported in the
CiliumNode would be out of date.  Therefore, in order to keep the
CiliumNode always consistent, we return an error when upserting a new
pool. To unorphan a CIDR, the original pool must be restored first.

Signed-off-by: Fabio Falzoi <[email protected]>

When a pool is updated, we have to loop over all the old CIDRs to remove
the ones that are not part of the latest version of the pool.

However, we should not immediately delete the removed CIDRs while
looping on them, because doing that interfere with the loop itself,
leading to incorrect results.  Instead we should mark each stale CIDR
and finally remove them from the slice later, once the loop is finished.

This can be proven by running the new unit test, that was failing before
this change:

=== RUN   TestUpdatePoolKeepOldCIDRs
    pool_allocator_test.go:1082:
        	Error Trace:	github.com/cilium/cilium/pkg/ipam/allocator/multipool/pool_allocator_test.go:1082
        	Error:      	Should be false
        	Test:       	TestUpdatePoolKeepOldCIDRs
--- FAIL: TestUpdatePoolKeepOldCIDRs (0.00s)

Signed-off-by: Fabio Falzoi <[email protected]>

Whenever a pool is updated, we have to reconcile the status of the pool
allocator accordingly, for both v4 and v6 allocators. Simplify and
improve the readability of the update logic.

Signed-off-by: Fabio Falzoi <[email protected]>