IPsec VinE Implementation by ldelossa · Pull Request #37723 · cilium/cilium

ldelossa · 2025-02-18T18:45:15Z

Implement IPsec VinE

See commits for details.

Add support for VXLAN in IPsec (VinE)

maintainer-s-little-helper · 2025-02-18T18:45:18Z

Commit 21605a5 does not match "(?m)^Signed-off-by:".

Please follow instructions provided in https://docs.cilium.io/en/stable/contributing/development/contributing_guide/#developer-s-certificate-of-origin

julianwiedmann

Some comments in-line :)

bpf/bpf_host.c

bpf/lib/encrypt.h

bpf/bpf_host.c

bpf/lib/encrypt.h

bpf/bpf_host.c

bpf/lib/encrypt.h

daemon/cmd/state.go

jschwinger233 · 2025-04-16T07:37:03Z

I still want to draw attention to the facts that ipsec encryption logic has changes:

Overlay encryption. 118 encrypts all overlay packets without inspecting the inner layer, while 117 makes encryption decision based on inner tuple. Egress gateway traffic will be affected.
Spi lookup. 118 always uses spi from node_map_v2[node_ip]->spi, while 117 sets spi from ipcache_map[dst_ip]->key. Ideally pod2pod shouldn't be affected.
Src_sec_id checks. 118 checks src_sec_id (ipsec_redirect_sec_id_ok()) for native routing, while 117 doesn't. Ideally pod2pod shouldn't be affected.

Above changes have been discussed multiple times and agreed upon by the team. The main goal is to consolidate the encryption datapath flow between IPsec and Wireguard for better consistency and maintainability. Let's keep in mind these important changes in case we unfortunately hit any issues.

ldelossa · 2025-04-16T12:54:32Z

@jschwinger233 thanks for the feedback Gray! ✅ to your comments.

tommyp1ckles

cli looks good

This commit was originally applied in ca9c056 removed in 152867f and is now being reapplied to support moving IPsec hooks to the egress device. To support upgrades properly we ensure that `bpf_host` is installed first when IPsec is enabled. This ensures the transition to `bpf_lxc` programs which no longer perform IPsec encryption do not result in leaked traffic. The IPsec enabled `bpf_host` programs will be in place prior to the IPsec disabled `bpf_lxc` transition. Signed-off-by: Paul Chaignon <[email protected]>

To fully support Encrypted Overlay and moving IPsec encrypt hooks to the egress device we must ensure the `to-netdev` program exists on native devices. Update the `DaemonConfig.AreDevicesRequired` function to return `true` when IPsec is enabled. This is a more general option which encompasses the `EnableIPSecEncryptedOverlay` flag. Moving forward Encrypted Overlay will be the default and this flag will be removed. Signed-off-by: Louis DeLosSantos <[email protected]>

Detect if traffic has been encrypted before traversing the overlay driver via a mark. This is useful for VinE, where encryption occurs at the ultimate egress device. If a packet was previously encrypted by the pre-VinE bpf_lxc programs, we can detect this and avoid performing double encryption. Likewise, if a packet arrives at the egress device and is unencrypted tunnel traffic, we can encrypt it to ensure the traffic does not leak. Implementation of the previous statement will be done in a subsequent commit. Signed-off-by: Louis DeLosSantos <[email protected]>

Move IPsec encryption hooks to the native egress device. Prior to this commit IPsec hooks would occur at pod egress (or at proxy egress if l7 policy is enforced). This move accomplish two goals: 1. Consolidate the encryption datapath flow between IPsec and Wireguard 2. Encrypt security IDs when tunnel mode is enabled (VinE/Encrypted Overlay) Thus, when this commit lands IPsec hooks will begin to be applied at bpf_host and only after the vlxan driver has encapsulated a packet, if tunnel mode is enabled. To cleanly migrate to the new IPsec datapath the Cilium cluster MUST be patched to the latest v1.17 patch release and then upgraded to v1.18. Signed-off-by: Louis DeLosSantos <[email protected]>

Remove the tests which are no longer valid for IPsec. This includes the group of tests which focused on bpf_lxc and encrypted overlay. IPsec encryption hooks no longer run at bpf_lxc and encrypted overlay is deprecated. Add two new tests: 1. Integration test with bpf_host which ensures the IPsec hooks are called for traffic about to leave the host 2. Unit tests for the IPsec encryption hook and its hairpin functionality. The latter ensures the `ctx` is prepared correctly for the hairpin and a call to ctx_redirect is made to recirculate the packet into the stack for XFRM processing Signed-off-by: Louis DeLosSantos <[email protected]>

Enable the v2 pod-to-pod encryption tests for IPsec and not just wireguard, now that VinE is implemented. Additionally, we can now disable the v1 pod-to-pod encryption tests for all encryption modes, since IPsec no longer requires it to run. Signed-off-by: Louis DeLosSantos <[email protected]>

This commit is added to work around an issue with the cluster mesh testing. This issue is being tracked and worked on saparately to this PR in issue: 38113 See issue for full details. Signed-off-by: gray <[email protected]>

Add documentation outlining the move to VinE in v1.18 forward. Signed-off-by: Louis DeLosSantos <[email protected]>

ldelossa · 2025-04-21T21:35:57Z

/test

ldelossa · 2025-04-21T22:20:25Z

IPsec flake is now cleared with merge of #38843 🎉

julianwiedmann · 2025-04-26T10:32:37Z

@ldelossa I feel this broke conformance-gke (which is no longer part of the default /test set).

In particular https://github.com/cilium/cilium/actions/runs/14585424143 was a scheduled run with this PR as the top-commit, and all the failed configs have tunnel-ipsec. These configs are failing consistently since then.

ldelossa · 2025-05-05T14:37:48Z

@julianwiedmann Thanks for the heads up.

Created: #39337

Will be debugging this now.

bpf/tests/ipsec_from_lxc_tunnel.c

pchaigno · 2025-06-02T08:37:08Z

bpf/tests/ipsec_redirect_native.c

+	 * The tunnel_endpoint is used in the IPsec hook under test below to
+	 * find the associated NodeID for an egress packet
+	 */
+	ipcache_v4_add_entry(DST_IP, 0, 0xAC, DST_IP, TARGET_SPI);


Hm, shouldn't all of this be in a SETUP instead of CHECK?

Oh, I see the comment above 😬 Yeah... we should debug.

I often run into strange anomalies when writing BPF unit tests. I tend to follow the path of least resistance once I find something that works.

Can't hurt to debug further tho.

I think part of the issue is that we keep calling them BPF unit tests when they're actually integration tests. The SETUP, CHECK, PKTGEN things are for integration tests, but you tried to use them to test just one function (so unit test). It took me a bit to understand that's why you had issues with SETUP.

Oh, I see. Cuz the first example in the BPF testing docs is a unit test which tests one function :-D

Hm, I see. And the docs don't say when to use which.

bpf/tests/ipsec_redirect_native.c

maintainer-s-little-helper bot added dont-merge/needs-sign-off The author needs to add signoff to their commits before merge. dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. labels Feb 18, 2025

github-actions bot added the cilium-cli This PR contains changes related with cilium-cli label Feb 18, 2025

ldelossa added release-note/major This PR introduces major new functionality to Cilium. and removed cilium-cli This PR contains changes related with cilium-cli labels Feb 18, 2025

maintainer-s-little-helper bot removed the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Feb 18, 2025

ldelossa force-pushed the ldelossa/ipsec-vxlan-in-esp branch from 6e08fac to 97e66cf Compare February 18, 2025 18:49

maintainer-s-little-helper bot removed the dont-merge/needs-sign-off The author needs to add signoff to their commits before merge. label Feb 18, 2025

ldelossa force-pushed the ldelossa/ipsec-vxlan-in-esp branch from 97e66cf to cf4f1d3 Compare February 18, 2025 20:19

julianwiedmann reviewed Feb 19, 2025

View reviewed changes

jschwinger233 added the feature/ipsec Relates to Cilium's IPsec feature label Feb 20, 2025

jschwinger233 reviewed Feb 20, 2025

View reviewed changes

bpf/bpf_host.c Show resolved Hide resolved

ldelossa added the pinned These issues are not marked stale by our issue bot. label Feb 20, 2025

ldelossa force-pushed the ldelossa/ipsec-vxlan-in-esp branch 3 times, most recently from 4f36d57 to b075675 Compare February 25, 2025 18:16

jschwinger233 reviewed Feb 26, 2025

View reviewed changes

bpf/lib/encrypt.h Outdated Show resolved Hide resolved

ldelossa force-pushed the ldelossa/ipsec-vxlan-in-esp branch 6 times, most recently from 4cad2ea to 7c1c9c8 Compare March 4, 2025 00:00

julianwiedmann reviewed Mar 4, 2025

View reviewed changes

daemon/cmd/state.go Show resolved Hide resolved

ldelossa force-pushed the ldelossa/ipsec-vxlan-in-esp branch 4 times, most recently from 8493c56 to 9b19920 Compare March 8, 2025 00:08

jschwinger233 mentioned this pull request Mar 11, 2025

ipsec: NodeID map sync delay causing no_nodeid_drop increaments during clustermesh init #38113

Closed

3 tasks

tommyp1ckles approved these changes Apr 16, 2025

View reviewed changes

pchaigno and others added 8 commits April 21, 2025 14:46

ipsec: add VinE related documentation

2cb2d66

Add documentation outlining the move to VinE in v1.18 forward. Signed-off-by: Louis DeLosSantos <[email protected]>

julianwiedmann mentioned this pull request Apr 22, 2025

Follow-up cleanups for IPSec Encrypted-Overlay #39071

Merged

ldelossa mentioned this pull request May 5, 2025

CI: Conformance GKE (ci-gke) - IPsec related failure #39337

Closed

julianwiedmann mentioned this pull request May 6, 2025

[v1.19] ipsec: remove legacy datapath code #39354

Closed

7 tasks

pippolo84 mentioned this pull request May 13, 2025

Multi-pool IPAM with tunneling and IPSec support #39442

Merged

pchaigno reviewed Jun 2, 2025

View reviewed changes

bpf/tests/ipsec_from_lxc_tunnel.c Show resolved Hide resolved

pchaigno reviewed Jun 2, 2025

View reviewed changes

bpf/tests/ipsec_redirect_native.c Show resolved Hide resolved

pchaigno mentioned this pull request Jun 3, 2025

bpf: Improve ipsec_redirect test #39873

Merged

pippolo84 mentioned this pull request Jul 22, 2025

Multi-pool IPAM: Add support for ipsec encryption in direct routing mode #40460

Merged

renovate bot mentioned this pull request Jul 31, 2025

fix(deps): update all non-major dependencies castai/kvisor#512

Open

1 task

This was referenced Sep 1, 2025

bpf: host: remove stale do_netdev_encrypt_encap() path #41452

Merged

bpf: lxc: don't special-case the RevDNAT path for IPsec configs #41487

Merged

datapath: remove IPsec logic in bpf_overlay #41540

Merged

julianwiedmann mentioned this pull request Sep 22, 2025

datapath: s/MARK_MAGIC_PROXY_TO_WORLD/MARK_MAGIC_SKIP_TPROXY #41813

Merged

mhofstetter mentioned this pull request Jan 14, 2026

endpoint/restore: remove special handling for host endpoint in case of ipsec #43757

Merged

Conversation

ldelossa commented Feb 18, 2025 • edited by joestringer Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maintainer-s-little-helper bot commented Feb 18, 2025

Uh oh!

julianwiedmann left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jschwinger233 commented Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ldelossa commented Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tommyp1ckles left a comment

Choose a reason for hiding this comment

Uh oh!

ldelossa commented Apr 21, 2025

Uh oh!

ldelossa commented Apr 21, 2025

Uh oh!

julianwiedmann commented Apr 26, 2025

Uh oh!

ldelossa commented May 5, 2025

Uh oh!

Uh oh!

pchaigno Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

pchaigno Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

ldelossa Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

pchaigno Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

ldelossa Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

pchaigno Jun 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

ldelossa commented Feb 18, 2025 •

edited by joestringer

Loading

jschwinger233 commented Apr 16, 2025 •

edited

Loading

ldelossa commented Apr 16, 2025 •

edited

Loading