I have tried this on a local build, and it functionally works for our use case.
Probably need more tests cover and documentation but I'd like some feedback and thoughts on high level approach, and what is there so far.

Thanks for the review so far @bimmlerd

I've made an attempt at moving the configuration into the iptables cell as requested, and I've updated the iptables implementation such that it adds a rule at the top of the CILIUM_POST_nat chain. This also opens the possibility of support an array (i.e. comma separated list of CIDR) as input, which would be useful for this feature, but not a pattern I remember having seen in cilium yet. Would appreciate thoughts on this.

For the bpf implementation I'm unsure how to pass this. Would really appreciate some suggestions.

Hi @bimmlerd

I added support for multiple CIDRs to the flag (comma-separated) and changed the BPF implementation to use LPM trie maps.

I'd appreciate a review on the way the config is done now, and where the map update logic is put in loader.go.

The ip-masq-agent BPF map lookup was the same as ours so I made a common function for that as well.

I've also found that by default the iptables implementation doesn't masquerade the additional IP pools, but leaving in our iptables implementation anyway to be more explicit about it.

Thanks

Administrative note: we're in feature freeze at the moment - since this is new it'll not be merged until 1.18 is branched. Freeze usually also means that review of bugfixes is prioritized at the cost of feature PRs such as this one, might take a bit of patience 🙏

that said, had a brief look and it looks better - but the CODEOWNERs will give detailed reviews

We're encountering a similar issue where a subnet advertised via BGP (used as a secondary Cilium IP pool) requires internet access through an external firewall that performs sNAT. While this PR is currently on hold (or is there a chance it could be merged soon?), we're wondering if it's possible to specify a wildcard range in ipMasqAgent.config.nonMasqueradeCIDRs to represent all internet ranges except a subnet such as 10.0.0.0/8?

Specifying 0.0.0.0/0 in nonMasqueradeCIDRs disables NAT for all pods, which causes issues - specifically, cluster workloads can't reach external services anymore since node-level masquerading is no longer applied.

@hassenius Are there any known workarounds to achieve this behavior in the meantime? Also, is there an estimated timeline for when this PR might be merged or released?

Are there any known workarounds to achieve this behavior in the meantime? Also, is there an estimated timeline for when this PR might be merged or released?

The workaround we will use until this is merged is to revert back to iptables masq - so no BPF-based masquerade. Since this is a new feature I imagine it will go into 1.19.

Are there any known workarounds to achieve this behavior in the meantime? Also, is there an estimated timeline for when this PR might be merged or released?

The workaround we will use until this is merged is to revert back to iptables masq - so no BPF-based masquerade. Since this is a new feature I imagine it will go into 1.19.

Thank you.

We have come up with this nonMasqueradeCIDR list to exclude our internal subnet (10.0.0.0/8). It's been working fine so far 😊

nonMasqueradeCIDRs:
      - "0.0.0.0/5"
      - "8.0.0.0/7"
      - "11.0.0.0/8"
      - "12.0.0.0/6"
      - "16.0.0.0/4"
      - "32.0.0.0/3"
      - "64.0.0.0/2"
      - "128.0.0.0/1"

@julianwiedmann & @brb I've re-done the PR to implement both configuration knobs that I mentioned, --only-masquerade-default-pool as a global configuration to skip masquerade for all non-default pools, and ipam.cilium.io/skip-masquerade as a per-pool annotation. I've not run it on a cluster yet to test (waiting for the Push dev charts CI to run) but I'd love to hear some feedback on the general direction.

edit: I did a cursory look, fixed a couple silly bugs, but overall behaviour looks like its working. I only tested the basic happy path as it's quite late so I'll keep testing over the next few days.

k8s-node-2> db/show podippools
Name      v4CIDRs                                                                               v4MaskSize   v6CIDRs   v6MaskSize   Flags
default   10.233.64.0/18                                                                        24                     -
other     10.235.200.0/24, 10.235.201.0/24, 10.235.202.0/24, 10.235.203.0/24, 10.235.204.0/24   32                     -            SkipMasquerade=true

This is the stateDB table - the CIDRs field might get quite unwieldy on larger pools so I might remove that or limit how many are shown here.

• i can confirm that this does indeed work as expected via the annotation method for tcp and udp traffic (as far as i have tested it) using quay.io/cilium/cilium-ci:915be6b1ac6ce2f467fae807f75841549353f9fe on top of rke2
• we have clusters and workloads this can be tested against further if y'all need people to test things irl
• thank you for implementing this, it'll allow me to eliminate the very last of our docker-compose based infra <3

Looked into the failing test "K8s Network E2E tests (dual)" - looks to be timing out on endpoint creation. Found some other pipelines from unrelated MRs with the same issue (I downloaded the logs and checked):
https://github.com/cilium/cilium/actions/runs/19664117723/job/56316862487
https://github.com/cilium/cilium/actions/runs/19672493789/job/56344795734
https://github.com/cilium/cilium/actions/runs/19461327067/job/55686038082

Could we give it a retry?

Also thanks @thehonker - appreciate you testing it out

/test

@julianwiedmann if you are happy with the latest changes, could you remove the dont-merge/discussion label?

/test

Thanks a lot @julianwiedmann, appreciate the review (and the idea for this approach). I've removed the unnecessary add_world_entry and set_identity_mark calls from the tests and added two additional tests to verify behaviour without the SKIP_MASQ flag

/test

/test

Looks like we have CI fallout in the multipool workflow from this change, although it's not clear to me why CI on the PR itself didn't catch it:

example

time=2025-12-16T13:12:45.808654792Z level=error source=/go/src/github.com/cilium/cilium/vendor/github.com/cilium/hive/cell/lifecycle.go:129 msg="Start hook failed" function="cmd.daemonLegacyInitialization.func1 (cmd/daemon_main.go:1294) (agent.controlplane.daemon)" error="daemon configuration failed: Unable to allocate router IP for family ipv4: IP pool 'default' not found in stateDB table" (1 occurrences)

Seems like it's a race. But yeah, indeed strange that we didn't catch it in this PR. We need to check in multiPoolManager.waitForAllPools if the StateDB table also contains the pool, before we return from NewIPAM

For posterity: The CI issue has been fixed: #43380