Skip to content

Commit 54621bc

Browse files
committed
feat(migration): tenant self-service VM import from VMware vSphere
Hidora's customers are leaving VMware now, and a tenant has to be able to import their own machines without a platform administrator. The chart approach could not express that: a tenant can neither create the Secret holding vCenter credentials nor name the VDDK image, so both had to come from somewhere they cannot reach. Two CRDs in a new `forklift.cozystack.io` group, split by lifecycle. A VMImportSource is a long-lived connection -- type, endpoint, credentials, readiness -- and a VMImportTask is the one-shot operation that names machines and produces VMDisks and VMInstances. The outputs carry no owner reference back to the task, so deleting a finished import leaves the machines alone. Konveyor Forklift does the transfer underneath; the controller renders its objects, mirrors its verdicts, and hands each finished volume into a VMDisk without copying it. What a live vCenter migration forced out, and no amount of source reading would have: - A guest copied verbatim keeps the drivers it had under VMware and no virtio, so on a virtio disk it does not fail the import -- it succeeds and then does not boot. Imported disks go on SATA. - VDDK connects straight to the ESXi host at the address vCenter advertises, which is routinely unreachable, and on the first cluster collided with the Service CIDR. `spec.hosts` redirects per host. - Firmware must be carried across or a UEFI guest imports as BIOS and never boots (stacked on #3002). - Deleting a task mid-transfer orphaned its DataVolume; a finalizer now removes the engine's volumes and leaves the outputs. The console gains a Migration section: connections are editable, and the VM field is a picker built from a machine list the controller publishes, since the aggregated API cannot reach the inventory and should not hold a source's credentials to try. Two general mechanisms fall out of that -- an option source may take an argument, and one named in a CRD annotation may reference a sibling field. Design: cozystack/community#62. Docs: cozystack/website#673. Signed-off-by: Andrei Kvapil <[email protected]>
1 parent d000d02 commit 54621bc

78 files changed

Lines changed: 16647 additions & 52 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/pr-labeler.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,7 @@ jobs:
149149
'cdi': 'area/virtualization',
150150
'vmi': 'area/virtualization',
151151
'vm-import': 'area/virtualization',
152+
'migration': 'area/virtualization',
152153
'virtual-machine': 'area/virtualization',
153154
'hami': 'area/virtualization',
154155
'gpu-operator': 'area/virtualization',

‎api/apps/v1alpha1/vminstance/types.go‎

Lines changed: 12 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎api/apps/v1alpha1/vminstance/zz_generated.deepcopy.go‎

Lines changed: 16 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
/*
2+
Copyright 2026.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
// Package v1alpha1 contains API Schema definitions for the v1alpha1 API group.
18+
// +kubebuilder:object:generate=true
19+
// +groupName=forklift.cozystack.io
20+
package v1alpha1
21+
22+
import (
23+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
24+
"k8s.io/apimachinery/pkg/runtime"
25+
"k8s.io/apimachinery/pkg/runtime/schema"
26+
)
27+
28+
const (
29+
thisGroup = "forklift.cozystack.io"
30+
thisVersion = "v1alpha1"
31+
)
32+
33+
var (
34+
GroupVersion = schema.GroupVersion{Group: thisGroup, Version: thisVersion}
35+
SchemeBuilder = runtime.NewSchemeBuilder(addGroupVersion)
36+
AddToScheme = SchemeBuilder.AddToScheme
37+
)
38+
39+
func addGroupVersion(scheme *runtime.Scheme) error {
40+
metav1.AddToGroupVersion(scheme, GroupVersion)
41+
return nil
42+
}
Lines changed: 224 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,224 @@
1+
// SPDX-License-Identifier: Apache-2.0
2+
// Package v1alpha1 defines forklift.cozystack.io API types.
3+
//
4+
// Group: forklift.cozystack.io
5+
// Version: v1alpha1
6+
package v1alpha1
7+
8+
import (
9+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
10+
"k8s.io/apimachinery/pkg/runtime"
11+
)
12+
13+
func init() {
14+
SchemeBuilder.Register(func(s *runtime.Scheme) error {
15+
s.AddKnownTypes(GroupVersion,
16+
&VMImportSource{},
17+
&VMImportSourceList{},
18+
)
19+
return nil
20+
})
21+
}
22+
23+
const (
24+
// ManagedByLabel marks the objects the migration controller owns, so a
25+
// projected Secret can never overwrite one the controller did not create.
26+
ManagedByLabel = "apps.cozystack.io/managed-by"
27+
// ManagedByValue is the value ManagedByLabel carries on our objects.
28+
ManagedByValue = "cozystack-migration"
29+
30+
// SourceReadyCondition reports whether the provider endpoint answered and
31+
// everything the platform must supply for this provider type is in place.
32+
SourceReadyCondition = "Ready"
33+
)
34+
35+
// Condition reasons carried on a VMImportSource.
36+
const (
37+
// ReasonConnected means the provider endpoint answered an authenticated call.
38+
ReasonConnected = "Connected"
39+
// ReasonConnectionFailed means the endpoint was unreachable or rejected the credentials.
40+
ReasonConnectionFailed = "ConnectionFailed"
41+
// ReasonCredentialsMissing means the credentials Secret could not be projected.
42+
ReasonCredentialsMissing = "CredentialsMissing"
43+
// ReasonVDDKNotConfigured means a vSphere source was created on a cluster
44+
// whose administrator has not supplied a VDDK image. The VMware path is
45+
// unavailable rather than broken: see the platform's migration settings.
46+
ReasonVDDKNotConfigured = "VDDKNotConfigured"
47+
// ReasonForkliftNotInstalled means the migration engine's CRDs are absent.
48+
// The controller ships independently of Forklift, so this is a normal state
49+
// on a cluster that has enabled one package and not the other.
50+
ReasonForkliftNotInstalled = "ForkliftNotInstalled"
51+
)
52+
53+
// InventoryTruncatedAnnotation marks a published VM list that was cut short
54+
// because the source holds more machines than one object should carry. A picker
55+
// missing an entry then has a visible reason rather than looking broken.
56+
const InventoryTruncatedAnnotation = thisGroup + "/inventory-truncated"
57+
58+
// The contract between the controller that publishes a source's machine list
59+
// and the aggregated API that serves it to the console's picker. It lives here
60+
// because both sides must agree on it and neither owns the other.
61+
const (
62+
// inventoryConfigMapPrefix makes the object's name derivable from the
63+
// source alone, so a reader addresses it without a lookup.
64+
inventoryConfigMapPrefix = "vmimport-inventory-"
65+
66+
// InventoryVMsKey holds the JSON array of PublishedVM.
67+
InventoryVMsKey = "vms"
68+
)
69+
70+
// InventoryConfigMapName is where a source's published machine list lives.
71+
func InventoryConfigMapName(source string) string {
72+
return inventoryConfigMapPrefix + source
73+
}
74+
75+
// PublishedVM is one entry of that list: deliberately the smallest thing a
76+
// picker needs, a value to write and a name to show.
77+
type PublishedVM struct {
78+
// ID is the managed-object reference, which is what a task records.
79+
ID string `json:"id"`
80+
// Name is the machine's name as the source knows it.
81+
Name string `json:"name"`
82+
// Path distinguishes two machines sharing a name in different folders.
83+
Path string `json:"path,omitempty"`
84+
}
85+
86+
// ProviderType names a source virtualization platform.
87+
// +kubebuilder:validation:Enum=vsphere
88+
type ProviderType string
89+
90+
const (
91+
// ProviderVSphere is VMware vSphere, reached through its vCenter SDK endpoint.
92+
ProviderVSphere ProviderType = "vsphere"
93+
)
94+
95+
// ProviderCredentials holds the account the platform uses to read the source
96+
// inventory and the disk data. The controller materializes these into a Secret;
97+
// a tenant never creates a Kubernetes Secret itself.
98+
type ProviderCredentials struct {
99+
// Username is the account used to authenticate against the provider
100+
// (e.g. `[email protected]`).
101+
// +kubebuilder:validation:MinLength=1
102+
Username string `json:"username"`
103+
104+
// Password is the account's password.
105+
// +kubebuilder:validation:MinLength=1
106+
Password string `json:"password"`
107+
108+
// CACert is the PEM-encoded certificate authority that signed the
109+
// endpoint's TLS certificate. Required unless insecureSkipVerify is set:
110+
// the migration engine refuses a vSphere connection that can neither
111+
// verify the endpoint nor was told not to.
112+
//
113+
// This is a CA certificate, not a fingerprint. A SHA-1 thumbprint is what
114+
// the engine wants for a direct ESXi host connection, which v1 does not
115+
// expose, and supplying one here does not satisfy the check.
116+
// +optional
117+
CACert string `json:"caCert,omitempty"`
118+
119+
// InsecureSkipVerify disables verification of the endpoint's TLS
120+
// certificate. Convenient in a lab and a bad idea anywhere else: the
121+
// credentials above travel over that connection. Prefer caCert.
122+
// +optional
123+
InsecureSkipVerify bool `json:"insecureSkipVerify,omitempty"`
124+
}
125+
126+
// VMImportSourceSpec describes a connection to a source virtualization platform.
127+
type VMImportSourceSpec struct {
128+
// Type is the source platform this connection speaks to.
129+
// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="type is immutable"
130+
Type ProviderType `json:"type"`
131+
132+
// URL is the provider's API endpoint (e.g. `https://vcenter.example.com/sdk`).
133+
// +kubebuilder:validation:MinLength=1
134+
URL string `json:"url"`
135+
136+
// Credentials is the account used to reach the provider.
137+
// +kubebuilder:validation:XValidation:rule="has(self.caCert) || (has(self.insecureSkipVerify) && self.insecureSkipVerify)",message="either caCert must be set or insecureSkipVerify must be true: the migration engine will not open a vSphere connection it can neither verify nor was told to trust"
138+
Credentials ProviderCredentials `json:"credentials"`
139+
140+
// Hosts redirects the disk transfer for individual ESXi hosts.
141+
//
142+
// Disk data does not travel through vCenter. VDDK opens its connection
143+
// straight to the ESXi host holding the VM, at whatever address vCenter
144+
// advertises for it — and that address is frequently one the cluster
145+
// cannot reach: a management network the workers are not on, or, worse,
146+
// one that collides with the cluster's own Service CIDR, where the packet
147+
// is swallowed by service routing and never leaves. The transfer then
148+
// fails late, after validation has passed, with an NBD error naming
149+
// neither the address nor the reason.
150+
//
151+
// An entry here names the host and the address to use instead. Each
152+
// carries its own credentials because the ESXi host authenticates the
153+
// connection itself rather than honouring the vCenter session, and the
154+
// engine connection-tests them before the transfer starts.
155+
//
156+
// +optional
157+
// +listType=map
158+
// +listMapKey=id
159+
Hosts []HostOverride `json:"hosts,omitempty"`
160+
}
161+
162+
// HostOverride redirects the disk transfer for one ESXi host.
163+
type HostOverride struct {
164+
// ID is the host's managed-object reference in the source inventory
165+
// (e.g. `host-10`). It is what the VM's inventory record names as its
166+
// host, not the hostname.
167+
// +kubebuilder:validation:MinLength=1
168+
ID string `json:"id"`
169+
170+
// Address is the address the cluster should use to reach this host for the
171+
// disk transfer, replacing the one vCenter advertises.
172+
// +kubebuilder:validation:MinLength=1
173+
Address string `json:"address"`
174+
175+
// Credentials is the account on the ESXi host itself. The host
176+
// authenticates the transfer connection directly, so the provider's
177+
// vCenter account does not carry over.
178+
// +kubebuilder:validation:XValidation:rule="has(self.caCert) || (has(self.insecureSkipVerify) && self.insecureSkipVerify)",message="either caCert must be set or insecureSkipVerify must be true: the migration engine will not open a connection to the host it can neither verify nor was told to trust"
179+
Credentials ProviderCredentials `json:"credentials"`
180+
}
181+
182+
// VMImportSourceStatus reports whether the connection is usable.
183+
type VMImportSourceStatus struct {
184+
// ObservedGeneration is the .metadata.generation the conditions were computed from.
185+
// +optional
186+
ObservedGeneration int64 `json:"observedGeneration,omitempty"`
187+
188+
// Conditions represents the latest available observations of the connection's state.
189+
// The Ready condition is the connection's readiness: it is true only once the
190+
// endpoint has answered and every platform prerequisite for this provider
191+
// type is satisfied.
192+
// +optional
193+
Conditions []metav1.Condition `json:"conditions,omitempty"`
194+
}
195+
196+
// +kubebuilder:object:root=true
197+
// +kubebuilder:subresource:status
198+
// +kubebuilder:resource:shortName=vmis;vmimportsrc
199+
// +kubebuilder:printcolumn:name="Type",type="string",JSONPath=".spec.type",priority=0
200+
// +kubebuilder:printcolumn:name="URL",type="string",JSONPath=".spec.url",priority=0
201+
// +kubebuilder:printcolumn:name="Ready",type="string",JSONPath=".status.conditions[?(@.type=='Ready')].status",priority=0
202+
// +kubebuilder:printcolumn:name="Reason",type="string",JSONPath=".status.conditions[?(@.type=='Ready')].reason",priority=1
203+
// +kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp",priority=0
204+
// +kubebuilder:selectablefield:JSONPath=`.spec.type`
205+
206+
// VMImportSource registers a reusable connection to a source virtualization
207+
// platform. It is long-lived: VMImportTasks reference it, and deleting it
208+
// deregisters the connection without touching anything already imported.
209+
type VMImportSource struct {
210+
metav1.TypeMeta `json:",inline"`
211+
metav1.ObjectMeta `json:"metadata,omitempty"`
212+
213+
Spec VMImportSourceSpec `json:"spec,omitempty"`
214+
Status VMImportSourceStatus `json:"status,omitempty"`
215+
}
216+
217+
// +kubebuilder:object:root=true
218+
219+
// VMImportSourceList contains a list of VMImportSources.
220+
type VMImportSourceList struct {
221+
metav1.TypeMeta `json:",inline"`
222+
metav1.ListMeta `json:"metadata,omitempty"`
223+
Items []VMImportSource `json:"items"`
224+
}

0 commit comments

Comments
 (0)