Skip to content

pkg/aws/eni: Adding support for prefix delegation in AWS bare metal#39678

Merged
tklauser merged 1 commit intocilium:mainfrom
DataDog:alex.melhem/prefix-delegation-on-metal
Jun 6, 2025
Merged

pkg/aws/eni: Adding support for prefix delegation in AWS bare metal#39678
tklauser merged 1 commit intocilium:mainfrom
DataDog:alex.melhem/prefix-delegation-on-metal

Conversation

@41ks
Copy link
Copy Markdown
Contributor

@41ks 41ks commented May 22, 2025

What does this do

This PR aims to add the possibility of prefix delegation on AWS instances of type bare metal. The original idea was to allow this feature only on instances of Hypervisor type nitro (PR #18463). However, this also works on nitro based instances of type bare metal. The only instances that don't support this feature are those of Hypervisor type xen.

This feature is also available on AWS's own CNI ENABLE_PREFIX_DELEGATION.

How was it tested

This change was successfully tested on our clusters. Having noticed a problem with AWS nitro based bare metal instances not being assigned IP prefixes, this fix solved our issue.

AWS ENI IPAM Mode: Added support for prefix delegation on AWS bare metal instances

@maintainer-s-little-helper maintainer-s-little-helper bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label May 22, 2025
@github-actions github-actions bot added the kind/community-contribution This was a contribution made by a community member. label May 22, 2025
@41ks 41ks marked this pull request as ready for review May 22, 2025 12:13
@41ks 41ks requested a review from a team as a code owner May 22, 2025 12:13
@41ks 41ks requested a review from liyihuang May 22, 2025 12:13
@antonipp
Copy link
Copy Markdown
Contributor

/test

1 similar comment
@devodev
Copy link
Copy Markdown
Contributor

devodev commented May 26, 2025

/test

@aanm aanm added the release-note/minor This PR changes functionality that users may find relevant to operating Cilium. label May 26, 2025
@maintainer-s-little-helper maintainer-s-little-helper bot removed the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label May 26, 2025
@aanm aanm enabled auto-merge May 26, 2025 14:36
@aanm
Copy link
Copy Markdown
Member

aanm commented May 26, 2025

/test

@liyihuang
Copy link
Copy Markdown
Contributor

@41ks thanks for the contribution. I personally think we should follow AWS logic as much as possible in case there are new hypervisor from AWS. I know we probably need to introduce a new field for baremetal but I think it's worthy in the long run

https://github.com/aws/amazon-vpc-cni-k8s/blob/a1d8f065ae04272b1d3a552543b4637546484e00/pkg/awsutils/awsutils.go#L1723-L1724

auto-merge was automatically disabled May 30, 2025 10:03

Head branch was pushed to by a user without write access

@41ks 41ks force-pushed the alex.melhem/prefix-delegation-on-metal branch from 3b46213 to fe6bbfb Compare May 30, 2025 10:03
@41ks 41ks requested a review from a team as a code owner May 30, 2025 10:03
@41ks 41ks requested a review from doniacld May 30, 2025 10:03
@antonipp
Copy link
Copy Markdown
Contributor

/test

@41ks 41ks force-pushed the alex.melhem/prefix-delegation-on-metal branch from fe6bbfb to d5e9c6e Compare May 30, 2025 11:19
Copy link
Copy Markdown
Contributor

@liyihuang liyihuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution. Overall it looks right to me. Hope you can add the unit test for this logic if possible

@tklauser
Copy link
Copy Markdown
Member

tklauser commented Jun 5, 2025

/test

@tklauser tklauser enabled auto-merge June 5, 2025 11:29
Copy link
Copy Markdown
Member

@tklauser tklauser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@41ks please run make generate-k8s-api && make manifests to update the generated deepequal method (see https://github.com/cilium/cilium/actions/runs/15391313276/job/43537611720?pr=39678) and please also squash all the commits into a single one. Thanks!

auto-merge was automatically disabled June 5, 2025 16:04

Head branch was pushed to by a user without write access

@41ks 41ks force-pushed the alex.melhem/prefix-delegation-on-metal branch from e6319be to 4ea4a12 Compare June 5, 2025 16:04
@41ks 41ks requested a review from tklauser June 5, 2025 16:06
@tklauser
Copy link
Copy Markdown
Member

tklauser commented Jun 5, 2025

/test

Copy link
Copy Markdown
Member

@tklauser tklauser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@tklauser tklauser enabled auto-merge June 5, 2025 16:13
@tklauser
Copy link
Copy Markdown
Member

tklauser commented Jun 6, 2025

@41ks Looks like this PR needs a rebase on top of latest main for the EKS related tests to passe.

This adds support for prefix delegation on AWS bare metal instances by checking against the BareMetal flag provided by the AWS API.

Signed-off-by: Alex Melhem <[email protected]>
auto-merge was automatically disabled June 6, 2025 11:51

Head branch was pushed to by a user without write access

@41ks 41ks force-pushed the alex.melhem/prefix-delegation-on-metal branch from 4ea4a12 to 3aa2600 Compare June 6, 2025 11:51
@HadrienPatte
Copy link
Copy Markdown
Member

/test

@tklauser tklauser enabled auto-merge June 6, 2025 11:59
@tklauser tklauser added this pull request to the merge queue Jun 6, 2025
@maintainer-s-little-helper maintainer-s-little-helper bot added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label Jun 6, 2025
Merged via the queue into cilium:main with commit a1a8ed3 Jun 6, 2025
68 checks passed
@HadrienPatte HadrienPatte deleted the alex.melhem/prefix-delegation-on-metal branch June 6, 2025 16:22
github-merge-queue bot pushed a commit to chezmoidotsh/arcane that referenced this pull request Jul 29, 2025
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [cilium](https://cilium.io/)
([source](https://redirect.github.com/cilium/cilium)) | HelmChart |
minor | `1.17.6` -> `1.18.0` |

---

### Release Notes

<details>
<summary>cilium/cilium (cilium)</summary>

###
[`v1.18.0`](https://redirect.github.com/cilium/cilium/releases/tag/v1.18.0):
1.18.0

[Compare
Source](https://redirect.github.com/cilium/cilium/compare/1.17.6...1.18.0)

We are excited to announce the **[Cilium
1.18.0](https://redirect.github.com/cilium/cilium/releases/tag/v1.18.0)**
release!

A total of **3298 new commits** have been contributed to this release by
a growing community of over **955 developers** and over **22,000 GitHub
stars**! ⭐

To keep up to date with all the latest Cilium releases, see
[Announcements](https://redirect.github.com/cilium/cilium/discussions/categories/announcements)

Here's what's new in
[v1.18.0](https://redirect.github.com/cilium/cilium/releases/tag/v1.18.0):

#### 🚠 Networking

- **⚖️ Load Balancing Redesign**: The service load-balancing
control-plane in the Cilium agent has been redesigned to reduce memory
usage and improve future extensibility of load-balancing features
([cilium/cilium#38469](https://redirect.github.com/cilium/cilium/pull/38469),
[@&#8203;joamaki](https://redirect.github.com/joamaki))
- **🔌 Virtual Network Devices**: Added support for new virtual network
device configurations such as VXLAN in IPsec (VinE) and IPIP tunnels
([cilium/cilium#37723](https://redirect.github.com/cilium/cilium/pull/37723),
[@&#8203;ldelossa](https://redirect.github.com/ldelossa);
[cilium/cilium#37346](https://redirect.github.com/cilium/cilium/pull/37346),
[@&#8203;gyutaeb](https://redirect.github.com/gyutaeb))
- **Ⓜ️ Multiple Egress Gateways**: Egress Gateways policies can now
direct traffic towards multiple gateway nodes
([cilium/cilium#39304](https://redirect.github.com/cilium/cilium/pull/39304),
[@&#8203;carlos-abad](https://redirect.github.com/carlos-abad))
- **🚦 Ingress Rate Limiting**: The bandwidth manager now supports
ingress rate limiting
([cilium/cilium#36351](https://redirect.github.com/cilium/cilium/pull/36351),
[@&#8203;l1b0k](https://redirect.github.com/l1b0k))
- **📢 Multi-Device L2 Announcements**: The L2 pod announcement feature
now supports multiple devices
([cilium/cilium#38198](https://redirect.github.com/cilium/cilium/pull/38198),
[@&#8203;dylandreimerink](https://redirect.github.com/dylandreimerink))
- **🏢 Neighbor Subsystem Rework**: The neighbor subsystem was made more
resilient through a new system that reconciles desired neighbor entries
with the kernel state
([cilium/cilium#39987](https://redirect.github.com/cilium/cilium/pull/39987),
[@&#8203;dylandreimerink](https://redirect.github.com/dylandreimerink))

#### 🌐 IPv6

- **🚇 Tunneling Underlay**: The tunneling datapath mode now supports
using an IPv6 network underlay, including when configured with IPsec
transparent encryption
([cilium/cilium#38296](https://redirect.github.com/cilium/cilium/pull/38296),
[cilium/cilium#39497](https://redirect.github.com/cilium/cilium/pull/39497),
[@&#8203;pchaigno](https://redirect.github.com/pchaigno))
- **💬 Kube Proxy Replacement**: Cilium now implements service
translation when running on an IPv6 underlay
([cilium/cilium#39074](https://redirect.github.com/cilium/cilium/pull/39074),
[@&#8203;pchaigno](https://redirect.github.com/pchaigno))
- **📋 Delegated IPAM**: When delegating IP address management to a third
party plugin, Cilium now configures IPv6 routes for connectivity if the
plugin supports IPv6
([cilium/cilium#38249](https://redirect.github.com/cilium/cilium/pull/38249),
[@&#8203;caorui-io](https://redirect.github.com/caorui-io),
[@&#8203;kadevu](https://redirect.github.com/kadevu))
- **📦 IP Fragment Support**: Cilium now processes ordered IPv6 fragments
to apply policy and routing functionality
([cilium/cilium#38110](https://redirect.github.com/cilium/cilium/pull/38110),
[@&#8203;gentoo-root](https://redirect.github.com/gentoo-root))
- **🚪 Egress gateway policies** can now match IPv6 address ranges
([cilium/cilium#38452](https://redirect.github.com/cilium/cilium/pull/38452),
[@&#8203;rgo3](https://redirect.github.com/rgo3))

#### 🛡️ Policy & Observability

- **🏷️ Policy Names in Hubble-CLI**: Show the names of (C)CNPs that
allowed or denied traffic when monitoring flows in Hubble
([cilium/cilium#39453](https://redirect.github.com/cilium/cilium/pull/39453),
[@&#8203;antonipp](https://redirect.github.com/antonipp))
- **📝 Policy Log Fields**: A new free-text log field is added to
policies, which is exposed in Hubble flows for easy correlation and
searching
([cilium/cilium#39902](https://redirect.github.com/cilium/cilium/pull/39902),
[@&#8203;squeed](https://redirect.github.com/squeed))
- **🛰️ Encapsulated Traffic Decoding**: Hubble decodes encapsulated
traffic for deeper introspection into traffic flows
([cilium/cilium#37634](https://redirect.github.com/cilium/cilium/pull/37634),
[@&#8203;kaworu](https://redirect.github.com/kaworu))
- **🏰 ClusterMesh Policy Restriction**: A new option allows the
**cluster** entity to apply only to the local cluster in ClusterMesh
environment
([cilium/cilium#39338](https://redirect.github.com/cilium/cilium/pull/39338),
[@&#8203;MrFreezeex](https://redirect.github.com/MrFreezeex))
- **✨ Enhanced Policy Dashboard**: The Policy section of the Cilium
Grafana dashboard has been improved to show more relevant graphs,
including policy drops in both directions
([cilium/cilium#36492](https://redirect.github.com/cilium/cilium/pull/36492),
[cilium/cilium#37445](https://redirect.github.com/cilium/cilium/pull/37445),
[@&#8203;squeed](https://redirect.github.com/squeed))

#### 🌅 Performance

- **📊 Scale Test Results**: Cilium implements policies and services up
to 45% faster in higher scale environments (Various;
[@&#8203;marseel](https://redirect.github.com/marseel),
[cilium/cilium#40227](https://redirect.github.com/cilium/cilium/pull/40227))
- **📦 Image Size Reduction**: Docker image sizes are reduced by 32% on
arm64 architecture images
([cilium/cilium#40005](https://redirect.github.com/cilium/cilium/pull/40005),
[@&#8203;marseel](https://redirect.github.com/marseel))
- **⚡ Improved Policy Performance**: The DNS proxy can process large
numbers of IPs faster, and the EndpointSelector match implementation has
been optimized
([cilium/cilium#39340](https://redirect.github.com/cilium/cilium/pull/39340),
[@&#8203;squeed](https://redirect.github.com/squeed);
[cilium/cilium#40414](https://redirect.github.com/cilium/cilium/pull/40414),
[@&#8203;marseel](https://redirect.github.com/marseel))
- **🪞 EndpointSlice Mirroring for Multi-Cluster Services**: Clustermesh
mirrors EndpointSlice from the local cluster instead of copying the
Service selectors when using the MCS-API controller
([cilium/cilium#38596](https://redirect.github.com/cilium/cilium/pull/38596),
[@&#8203;MrFreezeex](https://redirect.github.com/MrFreezeex))
- **🌐 KVStoreMesh Optimization**: Cross-cluster state distribution is
optimized by only synchronizing identities keyed by ID, not by value
([cilium/cilium#36471](https://redirect.github.com/cilium/cilium/pull/36471),
[@&#8203;HadrienPatte](https://redirect.github.com/HadrienPatte))
- **🧠 Egress Gateway Processing**: Egress gateway policy processing is
significantly improved when matching a large number of pods
([cilium/cilium#37714](https://redirect.github.com/cilium/cilium/pull/37714),
[@&#8203;giorio94](https://redirect.github.com/giorio94))
- **🗑️ Optimized Garbage Collection for Connection Tracking**: Cilium
leverages batched iterators for CTMap GC
([cilium/cilium#36288](https://redirect.github.com/cilium/cilium/pull/36288),
[@&#8203;tommyp1ckles](https://redirect.github.com/tommyp1ckles))

#### ⚙️ Operations

- **📈 API Server Connections at Scale**: Improve kube-apiserver
connections behavior at scale through failover and setting better jitter
and backoff configurations
([cilium/cilium#37601](https://redirect.github.com/cilium/cilium/pull/37601),
[@&#8203;aditighag](https://redirect.github.com/aditighag);
[cilium/cilium#38031](https://redirect.github.com/cilium/cilium/pull/38031),
[@&#8203;orange30](https://redirect.github.com/orange30);
[cilium/cilium#36648](https://redirect.github.com/cilium/cilium/pull/36648),
[@&#8203;wedaly](https://redirect.github.com/wedaly))
- **🔄 ConfigMap Synchronization**: New option to automatically
synchronize ConfigMap changes into the agent and report metrics for when
the effective configuration is different from the desired configuration
([cilium/cilium#36510](https://redirect.github.com/cilium/cilium/pull/36510),
[@&#8203;ovidiutirla](https://redirect.github.com/ovidiutirla))
- **🎓 CRD Promotion to Stable**: Promote **CiliumCIDRGroup**,
**CiliumLoadBalancerIPPool** and all **BGP** CRDs to stable API
([cilium/cilium#38940](https://redirect.github.com/cilium/cilium/pull/38940),
[@&#8203;christarazi](https://redirect.github.com/christarazi);
[cilium/cilium#39090](https://redirect.github.com/cilium/cilium/pull/39090),
[@&#8203;pippolo84](https://redirect.github.com/pippolo84);
[cilium/cilium#37765](https://redirect.github.com/cilium/cilium/pull/37765),
[@&#8203;rastislavs](https://redirect.github.com/rastislavs))
- **⛔ Node Taints Handling**: The cilium-operator Deployment uses a new
default set of taints which avoids deploying to a drained node
([cilium/cilium#40137](https://redirect.github.com/cilium/cilium/pull/40137),
[@&#8203;Murat](https://redirect.github.com/Murat) Parlakisik)
- **:wood: Migrate to Slog**: Cilium now uses slog as log library for
all components
([cilium/cilium#39664](https://redirect.github.com/cilium/cilium/pull/39664),
[@&#8203;aanm](https://redirect.github.com/aanm))
- **🔧 Cilium dependencies** were updated to Kubernetes v1.33, Envoy
v1.34, LLVM 19.1, and CNI v1.1
([cilium/cilium#39124](https://redirect.github.com/cilium/cilium/pull/39124),
[