Skip to content

Commit aced011

Browse files
lexfreilexfrei
committed
feat(platform): add opt-in in-cluster east-west encryption toggle
Add networking.encryption.enabled (off by default), a platform-level opt-in that turns on in-cluster transport encryption for the active datapath. On the default kubeovn-cilium datapath (and its generic variant) KubeOVN owns the overlay, so the toggle maps to KubeOVN's Geneve-overlay IPsec (kube-ovn ENABLE_OVN_IPSEC). KubeOVN's controller provisions the IPsec CA and per-node certificates through the Kubernetes CSR API, so no cert-manager wiring is required. This is defense-in-depth for the whole mesh without per-application changes; it complements, and does not replace, application-level / end-to-end TLS. Values comments document the per-datapath scope (including Kilo's cross-location-only mesh and the unreached Cilium-native path on cilium/cilium-generic variants) and the MTU/throughput overhead. Assisted-By: Claude <[email protected]> Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent 77d42ab commit aced011

3 files changed

Lines changed: 190 additions & 0 deletions

File tree

‎packages/core/platform/templates/bundles/system.yaml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,15 @@
1313
{{- if .Values.networking.kubeovn.MASTER_NODES -}}
1414
{{- $_ := set $kubeovnDict "MASTER_NODES" .Values.networking.kubeovn.MASTER_NODES -}}
1515
{{- end -}}
16+
{{- /* Opt-in east-west encryption (networking.encryption.enabled): on the
17+
default kubeovn-cilium datapath the overlay is KubeOVN's Geneve
18+
tunnel, so encrypting it means turning on KubeOVN IPsec. KubeOVN's
19+
own controller provisions the IPsec CA and per-node certs through
20+
the Kubernetes CSR API — no cert-manager wiring needed here. Off by
21+
default; complements (does not replace) application-level TLS. */ -}}
22+
{{- if dig "encryption" "enabled" false .Values.networking -}}
23+
{{- $_ := set $kubeovnDict "func" (dict "ENABLE_OVN_IPSEC" true) -}}
24+
{{- end -}}
1625
{{- $kubeovnValues := dict "kube-ovn" $kubeovnDict -}}
1726
{{- $_ := set $networkingComponents "kubeovn" (dict "values" $kubeovnValues) -}}
1827
{{- /* For Talos (isp-full): use KubePrism endpoint and disable cgroup autoMount */ -}}
@@ -79,6 +88,12 @@
7988
{{- end -}}
8089
{{- /* For generic k8s (k3s), control-plane label has value "true" */ -}}
8190
{{- $_ := set $kubeovnDict "MASTER_NODES_LABEL" "node-role.kubernetes.io/control-plane=true" -}}
91+
{{- /* Opt-in east-west encryption (networking.encryption.enabled): same
92+
KubeOVN Geneve-overlay IPsec path as isp-full. See the isp-full
93+
branch above for the mechanism notes. */ -}}
94+
{{- if dig "encryption" "enabled" false .Values.networking -}}
95+
{{- $_ := set $kubeovnDict "func" (dict "ENABLE_OVN_IPSEC" true) -}}
96+
{{- end -}}
8297
{{- $kubeovnValues := dict "kube-ovn" $kubeovnDict -}}
8398
{{- $_ := set $networkingComponents "kubeovn" (dict "values" $kubeovnValues) -}}
8499
{{- /* Cilium configuration - for generic k8s, always enable cgroup autoMount */ -}}

‎packages/core/platform/tests/bundles_networking_encryption_test.yaml‎

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
suite: networking.encryption bundle wiring
2+
templates:
3+
- templates/bundles/system.yaml
4+
release:
5+
name: cozystack
6+
namespace: cozy-system
7+
tests:
8+
- it: leaves KubeOVN overlay IPsec unset on the default isp-full datapath when encryption is off
9+
set:
10+
bundles:
11+
system:
12+
enabled: true
13+
variant: isp-full
14+
networking:
15+
encryption:
16+
enabled: false
17+
documentSelector:
18+
path: metadata.name
19+
value: cozystack.networking
20+
asserts:
21+
- equal:
22+
path: spec.variant
23+
value: kubeovn-cilium
24+
- notExists:
25+
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
26+
27+
- it: leaves KubeOVN overlay IPsec unset on isp-full when networking.encryption is absent (backward compatible, no nil-pointer)
28+
set:
29+
bundles:
30+
system:
31+
enabled: true
32+
variant: isp-full
33+
documentSelector:
34+
path: metadata.name
35+
value: cozystack.networking
36+
asserts:
37+
- equal:
38+
path: spec.variant
39+
value: kubeovn-cilium
40+
- notExists:
41+
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
42+
43+
- it: enables KubeOVN Geneve-overlay IPsec on the default isp-full datapath when encryption is on
44+
set:
45+
bundles:
46+
system:
47+
enabled: true
48+
variant: isp-full
49+
networking:
50+
encryption:
51+
enabled: true
52+
documentSelector:
53+
path: metadata.name
54+
value: cozystack.networking
55+
asserts:
56+
- equal:
57+
path: spec.variant
58+
value: kubeovn-cilium
59+
- equal:
60+
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
61+
value: true
62+
63+
- it: enables KubeOVN overlay IPsec on the generic kubeovn-cilium-generic datapath when encryption is on
64+
set:
65+
bundles:
66+
system:
67+
enabled: true
68+
variant: isp-full-generic
69+
networking:
70+
encryption:
71+
enabled: true
72+
documentSelector:
73+
path: metadata.name
74+
value: cozystack.networking
75+
asserts:
76+
- equal:
77+
path: spec.variant
78+
value: kubeovn-cilium-generic
79+
- equal:
80+
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
81+
value: true
82+
83+
- it: leaves the generic datapath IPsec unset when encryption is off
84+
set:
85+
bundles:
86+
system:
87+
enabled: true
88+
variant: isp-full-generic
89+
networking:
90+
encryption:
91+
enabled: false
92+
documentSelector:
93+
path: metadata.name
94+
value: cozystack.networking
95+
asserts:
96+
- equal:
97+
path: spec.variant
98+
value: kubeovn-cilium-generic
99+
- notExists:
100+
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
101+
102+
- it: keeps the existing kube-ovn ipv4 component overrides alongside the IPsec toggle (deep merge, not replace)
103+
set:
104+
bundles:
105+
system:
106+
enabled: true
107+
variant: isp-full
108+
networking:
109+
encryption:
110+
enabled: true
111+
documentSelector:
112+
path: metadata.name
113+
value: cozystack.networking
114+
asserts:
115+
- equal:
116+
path: spec.components.kubeovn.values.kube-ovn.ipv4.POD_CIDR
117+
value: 10.244.0.0/16
118+
- equal:
119+
path: spec.components.kubeovn.values.kube-ovn.func.ENABLE_OVN_IPSEC
120+
value: true
121+
122+
- it: renders without a nil-pointer error when networking is set to null and encryption is therefore unreachable
123+
set:
124+
bundles:
125+
system:
126+
enabled: true
127+
variant: isp-full
128+
networking: null
129+
documentSelector:
130+
path: metadata.name
131+
value: cozystack.networking
132+
asserts:
133+
- equal:
134+
path: spec.variant
135+
value: kubeovn-cilium
136+
- notExists:
137+
path: spec.components

‎packages/core/platform/values.yaml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,44 @@ networking:
4545
# Set this to comma-separated list of master node IPs to override.
4646
kubeovn:
4747
MASTER_NODES: ""
48+
# In-cluster (east-west) transport encryption. Opt-in, OFF by default.
49+
#
50+
# When enabled, pod-to-pod traffic that crosses nodes is encrypted at the
51+
# network layer by the ACTIVE datapath, regardless of whether the workload
52+
# speaks TLS — defense-in-depth for the whole mesh without per-application
53+
# changes. This COMPLEMENTS, and does NOT replace, application-level /
54+
# end-to-end TLS: it never reaches an external client and gives a tenant no
55+
# verifiable endpoint certificate (see the per-app certificate series). Use
56+
# it alongside, not instead of, TLS on exposed endpoints.
57+
#
58+
# The mechanism is datapath-specific (selected by bundles.system.variant):
59+
#
60+
# * kubeovn-cilium (isp-full, the default) and kubeovn-cilium-generic
61+
# (isp-full-generic): KubeOVN owns the datapath; its Geneve overlay is
62+
# encrypted with IPsec (kube-ovn ENABLE_OVN_IPSEC). KubeOVN's own
63+
# controller provisions the IPsec CA and per-node certificates through
64+
# the Kubernetes CSR API — no cert-manager wiring is required. This is
65+
# the path this toggle drives today.
66+
# * cilium / cilium-generic (Cilium owns the datapath): Cilium-native
67+
# WireGuard/IPsec is the right mechanism there, but the platform system
68+
# bundle does not select those variants, so this toggle does not reach
69+
# them yet.
70+
# * cilium-kilo: Kilo already establishes a WireGuard mesh, but only at
71+
# meshGranularity=location — it encrypts CROSS-LOCATION traffic and
72+
# leaves intra-location pod-to-pod in cleartext by design. This toggle
73+
# targets exactly that intra-location gap and composes with Kilo rather
74+
# than conflicting with it.
75+
#
76+
# CAVEATS: encrypting the overlay adds per-packet ESP overhead and lowers
77+
# the effective MTU, so east-west throughput drops and latency rises; the
78+
# cost is highest for high-bandwidth flows. IPsec here protects the
79+
# inter-node overlay tunnels — same-node pod-to-pod traffic never leaves the
80+
# host and is not tunneled or encrypted. Flipping this on an existing
81+
# cluster triggers a rollout of the CNI components. On variants without a
82+
# cozystack-managed encryptable datapath (e.g. isp-hosted / noop) the toggle
83+
# has no effect.
84+
encryption:
85+
enabled: false
4886
# Service publishing and ingress configuration
4987
publishing:
5088
host: "example.org"

0 commit comments

Comments
 (0)