Skip to content

Volume detach falls back to the VMI subresource, which needs the deprecated HotplugVolumes gate #381

Closed

Description

@lexfrei

removeVolumeFromVm takes the volume out of the VM object, and when the volume is not in VM.spec it falls back to RemoveVolumeFromVMI. That fallback is the path a control-plane teardown takes, and it only works while KubeVirt's HotplugVolumes feature gate is on.

KubeVirt marked HotplugVolumes deprecated in v1.9.0 (pkg/virt-config/featuregate/inactive.go), replaced by DeclarativeHotplugVolumes. The two are mutually exclusive at runtime: handleDeclarativeVolumeHotplug skips the declarative path entirely while HotplugVolumes is enabled, so a cluster cannot move to the replacement without turning the old gate off, and turning it off is what breaks this fallback. virt-api refuses a VMI-level removevolume on a VM-owned VMI unless ephemeralHotplugSupported() is true, and that function returns exactly HotplugVolumesEnabled().

The result for a downstream shipping both: it cannot enable the replacement gate until this driver stops needing the VMI subresource, and it cannot drop the deprecated one either. Cozystack is in that position today (cozystack/cozystack#4032).

I have read #112 and the VM-level path that came with #184, which is what we pin. This is about what is left underneath it: the VMI fallback and the gate it still implies.

What I am after is a direction rather than a specific patch. Whether the fallback can be dropped, or made conditional on the gate, or replaced with something that works when only DeclarativeHotplugVolumes is on. I can test a change against a tenant-cluster teardown, which is where the fallback actually fires for us.

Versions: KubeVirt v1.9.0, csi-driver pinned at v0.0.0-20260624122948-27b52aa22da7.

Activity

  1. awels commented on Sep 8, 2026

    @awels
    Member

    With the DeclarativeHotplug being the direction KubeVirt is going in, we should align the logic to work with that. The hotplug volumes featuregate is incompatible with the DeclarativeHotplug, and we want the VM to contain the desired state, and the original hotplug mechanism used a sub resource.

    If DeclarativeHotplug is enabled (it is a feature gate right now, once it graduates it will be the default) we should just skip the VMI fallback since it is on KubeVirt to properly detach the volume from the VMI if it still exists when the VM is gone.

  2. lexfrei commented on Sep 8, 2026

    @lexfrei
    ContributorAuthor

    @awels how should the driver know the gate is on? There is nothing on ControllerService that can read the KubeVirt CR, so the check has no source today.

    Three options I see:

    1. A driver flag, same shape as run-node-service. No new permissions, and it becomes dead weight once the gate graduates.
    2. Read the KubeVirt CR. Needs a new client method plus RBAC on kubevirts in the infra cluster.
    3. Keep the call and treat the ephemeralHotplugSupported rejection as a no-op. That means matching on an error text.

    I'd take the flag. Tell me which one you want and I can implement it for you. I'll test against a tenant cluster teardown, that's where the fallback fires for us.

  3. awels commented on Sep 8, 2026

    @awels
    Member

    Yeah I think the simplest would be a flag to the controller service. And based on that do one or the other. Should be simple enough to manage that, and we can remove it after a while if we are confident noone is using it anymore, or switch the default if not provided. So from a management perspective the flag is the simplest.

  4. lexfrei commented on Sep 22, 2026

    @lexfrei
    ContributorAuthor

    @awels flag it is. I'll keep the default as it is today, so the fallback stays on unless the flag turns it off and nobody picks up a silent change on upgrade. Clusters already on DeclarativeHotplugVolumes set it and lose a fallback they can't use anyway. Flipping the default can wait until the old gate is gone. I'll test against a tenant cluster teardown.

  5. awels commented on Sep 22, 2026

    @awels
    Member

    Sounds good to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions