Skip to content

Commit 39bdc9c

Browse files
mattia-eleuterilexfrei
authored andcommitted
fix(vm-instance): make the netplan network-data opt-in
Keying the network-data on instanceProfile reached every VM that never set the field, because the profile defaults to ubuntu whatever image the disk holds. The sysconfig, NetworkManager and eni renderers turn each match stanza into an interface literally named after the netplan ID, so a non-netplan guest left on the default profile would lose the DHCP it gets today. A VM with neither cloudInit nor sshKeys would also gain a cloud-init disk on upgrade, and KubeVirt's NoCloud instance-id is new to that guest, so its next boot would regenerate host keys as a first boot. The configuration also shadows a static NIC set up through netplan: netplan emits one networkd file per ID, networkd applies only the first file in lexical order that matches a link, and 10-netplan-cozy-en sorts before the file for enp2s0. The external-VLAN recipe claimed otherwise. An explicit netplanDhcp value, off by default, leaves every existing VM rendering exactly as before and lets the documentation say when not to turn it on. Assisted-by: LLM Signed-off-by: Mattia Eleuteri <[email protected]>
1 parent b5181b0 commit 39bdc9c

10 files changed

Lines changed: 79 additions & 95 deletions

File tree

‎api/apps/v1alpha1/vminstance/types.go‎

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/vm-external-vlan.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,7 @@ spec:
112112

113113
## Step 3 — Attach the VM and assign a static address
114114

115-
Reference the NAD by name in the `vm-instance` values. The chart's own `networkData` only asks every ethernet for DHCP, and this VLAN has no DHCP server, so the static address goes into cloud-init `userData` (`cloudInit`), written by the guest at first boot:
115+
Reference the NAD by name in the `vm-instance` values. This VLAN has no DHCP server, so the static address goes into cloud-init `userData` (`cloudInit`), written by the guest at first boot, and `netplanDhcp` stays off (see [Gotchas](#gotchas)):
116116

117117
```yaml
118118
# vm-instance values
@@ -145,7 +145,7 @@ The VM ends up with two interfaces: the always-present **pod-network** NIC (`def
145145
## Gotchas
146146

147147
- **VMs are dual-homed.** The `vm-instance` chart always adds the pod-network NIC in addition to any `networks` you declare; there is no single-homed (VLAN-only) option today. Address the VLAN NIC inside the guest and leave the pod NIC to the cluster.
148-
- **`networkData` is the chart's, not yours.** For a guest whose `instanceProfile` is `ubuntu` or `debian` the chart supplies its own cloud-init network-data, asking every `en*` and `eth*` interface for DHCP, and there is no value that replaces it. It does not conflict with the recipe above — netplan merges `/etc/netplan/*.yaml`, and a higher-numbered file wins — but a static address still has to come from `userData`, as above.
148+
- **Leave `netplanDhcp` off on a VM with a static NIC.** `netplanDhcp: true` makes the chart supply cloud-init network-data that asks every `en*` and `eth*` interface for DHCP, under the netplan IDs `cozy-en` and `cozy-eth`. Netplan writes one networkd file per ID, and systemd-networkd applies only the first `.network` file in lexical order that matches a link, so `10-netplan-cozy-en.network` shadows the `10-netplan-enp2s0.network` the recipe above produces and the static address is never applied, whatever number the recipe's netplan file carries. The key is off by default and is meant for guests whose every interface takes its address over DHCP, such as a VM on a VPC subnet.
149149
- **MAC changes on VM re-creation.** KubeVirt generates a fresh guest MAC each time the VM object is re-created, and `vm-instance` exposes no way to pin it, so re-creating a VM changes its MAC. The upstream gateway then holds a stale ARP entry for the old MAC for a few minutes, so "gateway unreachable" immediately after re-creating a VM is expected — wait for the ARP entry to age out (~5 min) rather than treating it as a fault.
150150
- **Host ↔ VM traffic.** If the host must talk to the VMs (a proxy, a health check), give the bridge a host address on the VLAN (step 1) — traffic through a bare VLAN sub-interface to bridge-attached guests will not work the way `macvlan` users expect.
151151

‎packages/apps/vm-instance/README.md‎

Lines changed: 31 additions & 30 deletions
Large diffs are not rendered by default.

‎packages/apps/vm-instance/templates/_helpers.tpl‎

Lines changed: 0 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -230,24 +230,3 @@ affinity:
230230
{{- end -}}
231231
{{- end -}}
232232
{{- end -}}
233-
234-
{{/*
235-
Guests whose interfaces are rendered by netplan, which is the population the
236-
cloud-init network-data below exists for. Non-empty means netplan.
237-
238-
instanceProfile is a KubeVirt preference rather than a declaration of the guest
239-
network stack, so this is a proxy; it is the same proxy the Windows node
240-
affinity above already relies on, and the field that replaced the
241-
systemDisk.image the network-data template was gated on before the
242-
virtual-machine chart was split into vm-disk and vm-instance. Narrow by design:
243-
cloud-init passes a v2 config through verbatim only for the netplan renderer,
244-
so widening this to a guest that renders through sysconfig or eni would hand it
245-
a match stanza and a dhcp4-overrides key that renderer drops, leaving the
246-
interface with no DHCP at all.
247-
*/}}
248-
{{- define "virtual-machine.netplanGuest" -}}
249-
{{- $profile := toString .Values.instanceProfile -}}
250-
{{- if or (hasPrefix "ubuntu" $profile) (hasPrefix "debian" $profile) -}}
251-
true
252-
{{- end -}}
253-
{{- end -}}

‎packages/apps/vm-instance/templates/secret-network-config.yaml‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,13 @@ Matching en* and eth* rather than naming interfaces keeps this correct whatever
1515
the guest calls its NICs, and is the shape Debian's own cloud images ship.
1616
networkDataSecretRef replaces the fallback configuration instead of adding to
1717
it, so every interface has to be described here, not just the secondary ones.
18+
19+
Opt-in rather than keyed on instanceProfile, whose default is ubuntu whatever
20+
the disk holds: the sysconfig, NetworkManager and eni renderers turn each match
21+
stanza into an interface literally named cozy-en or cozy-eth, so a non-netplan
22+
guest would lose DHCP on its real NIC.
1823
*/}}
19-
{{- if include "virtual-machine.netplanGuest" . }}
24+
{{- if .Values.netplanDhcp }}
2025
---
2126
apiVersion: v1
2227
kind: Secret

‎packages/apps/vm-instance/templates/vm.yaml‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,6 @@
2020
{{- fail "Either instanceType or resources (cpu, sockets, memory) must be specified" }}
2121
{{- end }}
2222
{{- $networks := .Values.networks | default .Values.subnets }}
23-
{{- $netplanGuest := include "virtual-machine.netplanGuest" . }}
2423

2524
apiVersion: kubevirt.io/v1
2625
kind: VirtualMachine
@@ -110,7 +109,7 @@ spec:
110109
{{- fail (printf "Specified disk not exists in cluster: %s" .name) }}
111110
{{- end }}
112111
{{- end }}
113-
{{- if or .Values.cloudInit .Values.sshKeys $netplanGuest }}
112+
{{- if or .Values.cloudInit .Values.sshKeys .Values.netplanDhcp }}
114113
- name: cloudinitdisk
115114
disk: {}
116115
{{- end }}
@@ -143,14 +142,14 @@ spec:
143142
dataVolume:
144143
name: vm-disk-{{ .name }}
145144
{{- end }}
146-
{{- if or .Values.cloudInit .Values.sshKeys $netplanGuest }}
145+
{{- if or .Values.cloudInit .Values.sshKeys .Values.netplanDhcp }}
147146
- name: cloudinitdisk
148147
cloudInitNoCloud:
149148
{{- if or .Values.cloudInit .Values.sshKeys }}
150149
secretRef:
151150
name: {{ include "virtual-machine.fullname" . }}-cloud-init
152151
{{- end }}
153-
{{- if $netplanGuest }}
152+
{{- if .Values.netplanDhcp }}
154153
networkDataSecretRef:
155154
name: {{ include "virtual-machine.fullname" . }}-network-config
156155
{{- end }}

‎packages/apps/vm-instance/tests/network_data_test.yaml‎

Lines changed: 24 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
suite: vm-instance cloud-init network-data for netplan guests
1+
suite: vm-instance cloud-init network-data for netplan guests (netplanDhcp)
22

33
# A netplan guest configures only the interface cloud-init's fallback covers and
44
# keeps none of the DHCP-supplied search domains, so the assertions below are on
@@ -36,22 +36,14 @@ kubernetesProvider:
3636
kind: VirtualMachineClusterPreference
3737
metadata:
3838
name: ubuntu
39-
- apiVersion: instancetype.kubevirt.io/v1beta1
40-
kind: VirtualMachineClusterPreference
41-
metadata:
42-
name: debian
43-
- apiVersion: instancetype.kubevirt.io/v1beta1
44-
kind: VirtualMachineClusterPreference
45-
metadata:
46-
name: windows.2k22
4739
- apiVersion: instancetype.kubevirt.io/v1beta1
4840
kind: VirtualMachineClusterPreference
4941
metadata:
5042
name: rhel.9
5143
tests:
52-
- it: gives an ubuntu guest DHCP on every ethernet and keeps the offered search domains
44+
- it: gives every ethernet DHCP and keeps the offered search domains
5345
set:
54-
instanceProfile: ubuntu
46+
netplanDhcp: true
5547
asserts:
5648
- template: templates/secret-network-config.yaml
5749
equal:
@@ -75,7 +67,7 @@ tests:
7567
7668
- it: attaches the network-data to the VM even when no user-data is requested
7769
set:
78-
instanceProfile: ubuntu
70+
netplanDhcp: true
7971
asserts:
8072
- template: templates/vm.yaml
8173
equal:
@@ -91,7 +83,7 @@ tests:
9183

9284
- it: carries user-data and network-data side by side
9385
set:
94-
instanceProfile: ubuntu
86+
netplanDhcp: true
9587
cloudInit: |
9688
#cloud-config
9789
packages:
@@ -106,43 +98,40 @@ tests:
10698
path: spec.template.spec.volumes[?(@.name=="cloudinitdisk")].cloudInitNoCloud.networkDataSecretRef.name
10799
value: test-vm-network-config
108100

109-
- it: covers debian, which netplan renders the same way
110-
set:
111-
instanceProfile: debian
112-
asserts:
113-
- template: templates/secret-network-config.yaml
114-
hasDocuments:
115-
count: 1
116-
117-
- it: leaves a windows guest untouched
118-
set:
119-
instanceProfile: windows.2k22
101+
# The default instanceProfile is ubuntu whatever image the disk holds, and a
102+
# cloud-init disk appearing on a VM that never had one makes the guest's next
103+
# boot a first boot, so a VM that leaves the key alone must render exactly as
104+
# before: no network-data and no cloud-init disk.
105+
- it: leaves a VM on the default profile without network-data or a cloud-init disk
120106
asserts:
121107
- template: templates/secret-network-config.yaml
122108
hasDocuments:
123109
count: 0
124110
- template: templates/vm.yaml
125111
notExists:
126112
path: spec.template.spec.volumes[?(@.name=="cloudinitdisk")]
127-
128-
- it: leaves a guest rendered by sysconfig untouched
129-
set:
130-
instanceProfile: rhel.9
131-
asserts:
132-
- template: templates/secret-network-config.yaml
133-
hasDocuments:
134-
count: 0
135113
- template: templates/vm.yaml
136114
notExists:
137-
path: spec.template.spec.volumes[?(@.name=="cloudinitdisk")].cloudInitNoCloud.networkDataSecretRef
115+
path: spec.template.spec.domain.devices.disks[?(@.name=="cloudinitdisk")]
138116

139-
- it: keeps a sysconfig guest's user-data disk
117+
- it: keeps user-data alone when the key is off
140118
set:
141-
instanceProfile: rhel.9
142119
cloudInit: |
143120
#cloud-config
144121
asserts:
145122
- template: templates/vm.yaml
146123
equal:
147124
path: spec.template.spec.volumes[?(@.name=="cloudinitdisk")].cloudInitNoCloud.secretRef.name
148125
value: test-vm-cloud-init
126+
- template: templates/vm.yaml
127+
notExists:
128+
path: spec.template.spec.volumes[?(@.name=="cloudinitdisk")].cloudInitNoCloud.networkDataSecretRef
129+
130+
- it: follows the key rather than the instance profile
131+
set:
132+
instanceProfile: rhel.9
133+
netplanDhcp: true
134+
asserts:
135+
- template: templates/secret-network-config.yaml
136+
hasDocuments:
137+
count: 1

‎packages/apps/vm-instance/values.schema.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -220,6 +220,11 @@
220220
"type": "string",
221221
"default": ""
222222
},
223+
"netplanDhcp": {
224+
"description": "Supply cloud-init network-data that requests DHCP on every `en*` and `eth*` interface and keeps the DNS search domains DHCP offers. Needed for a netplan guest (Ubuntu, Debian) to address a secondary network such as a VPC subnet. Leave it off for a guest rendered by anything other than netplan, which would lose DHCP on every interface, and for a guest that configures an interface statically through netplan, which this configuration would shadow. Turning it on for a VM that has neither `cloudInit` nor `sshKeys` adds a cloud-init disk, and the guest's next boot is then treated as a first boot.",
225+
"type": "boolean",
226+
"default": false
227+
},
223228
"cloudInitSeed": {
224229
"description": "Seed string to generate SMBIOS UUID for the VM.",
225230
"type": "string",

‎packages/apps/vm-instance/values.yaml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,9 @@ cloudInit: ""
115115
## chpasswd: { expire: False }
116116
##
117117

118+
## @param {bool} netplanDhcp - Supply cloud-init network-data that requests DHCP on every `en*` and `eth*` interface and keeps the DNS search domains DHCP offers. Needed for a netplan guest (Ubuntu, Debian) to address a secondary network such as a VPC subnet. Leave it off for a guest rendered by anything other than netplan, which would lose DHCP on every interface, and for a guest that configures an interface statically through netplan, which this configuration would shadow. Turning it on for a VM that has neither `cloudInit` nor `sshKeys` adds a cloud-init disk, and the guest's next boot is then treated as a first boot.
119+
netplanDhcp: false
120+
118121
## @param {string} cloudInitSeed - Seed string to generate SMBIOS UUID for the VM.
119122
cloudInitSeed: ""
120123
## Example:

0 commit comments

Comments
 (0)