feat(attach): force Protocol C on dual-attach for Protocol-A/B resources - #435

…h fix

Bump linstor-csi to v1.10.6 and add an out-of-tree patch that overrides
DrbdOptions/Net/protocol to C on the resource-definition whenever a
volume is attached to a second node with allow-two-primaries (the
live-migration flow), and reverts it on detach.

Without the patch, any KubeVirt live migration of a VM whose volume
sits on a Protocol-A/B resource (for example a "replicated-async"
StorageClass) ends up in a permanent evacuation loop: drbdadm adjust
on the satellite rejects allow-two-primaries with "Protocol C
required" (errno 139), the migration fails, and KubeVirt retries
until manual intervention.

The override is set at the resource-definition level so it covers
every connection, including diskless TieBreaker peers, and is tagged
with an Aux marker so Detach reverts only the override we installed.
The 001 relocate-after-clone patch was regenerated against v1.10.6
(context shift only, no logic change).

Verified end-to-end on dev5: live migration of a KubeVirt VM whose
PVC sits on a Protocol-A resource-group now succeeds in a single
Attach.

Patch is upstreamed as draft PR piraeusdatastore/linstor-csi#435.

Co-Authored-By: Claude <[email protected]>
Signed-off-by: Andrei Kvapil <[email protected]>

…h fix

Bump linstor-csi to v1.10.6 and add an out-of-tree patch that overrides
DrbdOptions/Net/protocol to C on the resource-definition whenever a
volume is attached to a second node with allow-two-primaries (the
live-migration flow), and reverts it on detach.

Without the patch, any KubeVirt live migration of a VM whose volume
sits on a Protocol-A/B resource (for example a "replicated-async"
StorageClass) ends up in a permanent evacuation loop: drbdadm adjust
on the satellite rejects allow-two-primaries with "Protocol C
required" (errno 139), the migration fails, and KubeVirt retries
until manual intervention.

The override is set at the resource-definition level so it covers
every connection, including diskless TieBreaker peers, and is tagged
with an Aux marker so Detach reverts only the override we installed.
The 001 relocate-after-clone patch was regenerated against v1.10.6
(context shift only, no logic change).

Verified end-to-end on dev5: live migration of a KubeVirt VM whose
PVC sits on a Protocol-A resource-group now succeeds in a single
Attach.

Patch is upstreamed as draft PR piraeusdatastore/linstor-csi#435.

Co-Authored-By: Claude <[email protected]>
Signed-off-by: Andrei Kvapil <[email protected]>

…h fix (#2496)

## What this PR does

Bumps `linstor-csi` from v1.10.5 to v1.10.6 and ships an out-of-tree
patch that fixes live migration of KubeVirt VMs whose volumes sit on a
DRBD Protocol-A/B resource (e.g. a `replicated-async` StorageClass).

### Problem

DRBD requires Protocol C whenever `allow-two-primaries=yes` is enabled.
Operators commonly opt into Protocol A on a per-resource-group basis for
async / WAN replication, which silently breaks every subsequent live
migration of consumers of those volumes: `drbdadm adjust` rejects the
second-attach with `Protocol C required` (errno 139), KubeVirt's
evacuation loop retries indefinitely, and the VM stays pinned to the
source node.

### Change

- `LINSTOR_CSI_VERSION` 1.10.5 → 1.10.6 (Makefile + Dockerfile default).
- New patch `002-protocol-c-override-for-dual-attach.diff`: when
`Attach` installs `allow-two-primaries=yes` on the resource-definition
during a second attach, it also installs `DrbdOptions/Net/protocol=C` as
an override on the resource-definition. The override applies to every
connection (including diskless TieBreaker peers, where a per-pair
override would still leave one connection broken). It is tagged with
`Aux/csi-protocol-override=yes` so `Detach` removes only the override
this driver installed, leaving any operator-set `Protocol` property on
the resource-definition untouched.
- Existing patch `001-relocate-after-clone-restore.diff` regenerated
against v1.10.6 (context shift only, no logic change — the old patch
hunks no longer aligned cleanly).

### Verification

- `make image-linstor-csi` builds successfully on linux/amd64 with both
patches applied.
- End-to-end test on dev5 cluster (KubeVirt v1.6.3, 3-node Talos):
created a Protocol-A resource-group + StorageClass, provisioned a VM on
top, and triggered live migration. Migration succeeds in a single Attach
with the override installed during dual-attach and removed by Detach.
Reproducer (without the patch) is the well-known evacuation loop with
`(node) Failed to adjust DRBD resource ... Protocol C required`.

### Upstream

Upstreamed as draft PR piraeusdatastore/linstor-csi#435.

### Compatibility

- No behaviour change for resources already using Protocol C (the common
case).
- No behaviour change for resources never attached with
allow-two-primaries.
- Idempotent: re-running `Attach` is a no-op once the override is
installed.
- Operator-set Protocol overrides on the resource-definition are
preserved (gated by the Aux marker).

### Release note

```release-note
fix(linstor): live migration of KubeVirt VMs on Protocol-A/B (async) DRBD volumes no longer fails with "Protocol C required" — linstor-csi now installs a Protocol=C override on the resource-definition during dual-attach and reverts it on detach.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Automatic replica relocation after volume clone and snapshot restore
to improve placement and load distribution.
* Conditional DRBD protocol override to enable/clean up dual-attach
(two-primaries) scenarios more reliably.

* **Chores**
  * Updated LINSTOR CSI default to v1.10.6.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

piraeus-operator v2.12.1 is released against LINSTOR CSI v1.13.2, and
the release line carries IO QoS limits through the qos.linbit.com/*
StorageClass parameters.

The operator gives the CSI pods their client certificate as mounted
files through LS_USER_CERTIFICATE_FILE and LS_USER_KEY_FILE, instead
of inline values in LS_USER_CERTIFICATE and LS_USER_KEY. The
linstor-wait-until v0.3.1 in this image is built on a golinstor
release that reads only the inline variables, so its init containers
present no client certificate, the LINSTOR API closes the TLS
connection, and they retry on EOF forever. Move to
linstor-wait-until v1.0.1, the version upstream v1.13.2 pins.

The Protocol C override for dual attach is dropped. Since v1.12 the
driver no longer sets allow-two-primaries itself: it asks LINSTOR to
manage dual-primary through make-available, and LINSTOR 1.35 sets
protocol C on the connections that need it. Upstream closed the
equivalent change as superseded for that reason
(piraeusdatastore/linstor-csi#435).

The relocate-after-clone patch is carried forward. Volume IDs became
a struct for consistency groups; Clone and VolFromSnap handle
standalone resources, so relocation keys on the resource name. The
enumer output is regenerated rather than hand-patched.

Upstream now pins grpc v1.84.0, which builds on Go 1.27 and includes
the fix for GO-2026-6443, so the grpc override is no longer needed.

The golden-image clone guard in vm-disk was re-checked: parameter
parsing for allowRemoteVolumeAccess and layerList, and the default
derived from the layer list, are unchanged in this range.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <[email protected]>

piraeus-operator v2.12.1 is released against LINSTOR CSI v1.13.2, and
the release line carries IO QoS limits through the qos.linbit.com/*
StorageClass parameters.

The operator gives the CSI pods their client certificate as mounted
files through LS_USER_CERTIFICATE_FILE and LS_USER_KEY_FILE, instead
of inline values in LS_USER_CERTIFICATE and LS_USER_KEY. The
linstor-wait-until v0.3.1 in this image is built on a golinstor
release that reads only the inline variables, so its init containers
present no client certificate, the LINSTOR API closes the TLS
connection, and they retry on EOF forever. Move to
linstor-wait-until v1.0.1, the version upstream v1.13.2 pins.

The Protocol C override for dual attach is dropped. Since v1.12 the
driver no longer sets allow-two-primaries itself: it asks LINSTOR to
manage dual-primary through make-available, and LINSTOR 1.35 sets
protocol C on the connections that need it. Upstream closed the
equivalent change as superseded for that reason
(piraeusdatastore/linstor-csi#435).

The relocate-after-clone patch is carried forward. Volume IDs became
a struct for consistency groups; Clone and VolFromSnap handle
standalone resources, so relocation keys on the resource name. The
enumer output is regenerated rather than hand-patched.

Upstream now pins grpc v1.84.0, which builds on Go 1.27 and includes
the fix for GO-2026-6443, so the grpc override is no longer needed.

The golden-image clone guard in vm-disk was re-checked: parameter
parsing for allowRemoteVolumeAccess and layerList, and the default
derived from the layer list, are unchanged in this range.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <[email protected]>

## What this PR does

Updates the LINSTOR stack to the current stable piraeus-operator
release: operator v2.12.1, LINSTOR server 1.35.3 and LINSTOR CSI
v1.13.2, the last two built here with our patches. This started as a fix
for the NFS server image hanging up the host console on bare-metal
storage nodes (Ubuntu 24.04, Cozystack 1.6.5 and 1.7.0). That needed
nfs-server v1.10.6 or later. Operators also asked for per-volume IO
limits, which need linstor-csi v1.11.3 or later and piraeus-operator
v2.10.8 or later, so the update goes to the current release rather than
the minimum.

The first commit fixes a problem the update exposed. The operator runs
the LINSTOR database migration in a `run-migration` init container, and
this package pinned only the main controller container to our build.
With the operator update, that init container ran the operator's default
LINSTOR 1.35.3 and migrated the database before the controller image
changed. The 1.33.3 controller then crash-looped until the linstor
release caught up. Both init containers that run the LINSTOR server
image (`run-migration` and the satellite's `setup-lvm-configuration`)
now use the platform image, with a test pinning it. This protects later
upgrades where the operator default is newer than our build. It does not
help this upgrade: the pin ships in the linstor release, and Flux
reconciles piraeus-operator first.

Patches:

- linstor-csi: the Protocol C override for dual attach is dropped. Since
v1.12 the driver asks LINSTOR to manage dual-primary, and LINSTOR 1.35
sets protocol C on the connections that need it. Upstream closed the
same change as superseded (piraeusdatastore/linstor-csi#435). The
relocate-after-clone patch is carried forward.
- LINSTOR server: six of the nine patches are in upstream by 1.35.3 and
are removed. `fix-k8s-create-already-exists`,
`retry-adjust-after-stale-bitmap` and `native-protoc` are carried
forward. LINSTOR 1.34 requires Java 21, so the image moves from bookworm
to trixie.

Behaviour that changes for users:

- No downgrade. LINSTOR 1.35 migrates the database and drops the `SEC_*`
tables, so a cluster that has run this version cannot go back to 1.33.
- The LINSTOR controller is down for a short time during this upgrade.
piraeus-operator is reconciled before the linstor release, so the
database is migrated to 1.35 while the old controller still runs. That
controller crash-loops until the linstor release rolls out, and if the
rollout fails there is no way back.
- A volume on a Protocol A or B resource group that was attached to two
nodes while the CSI driver was updated (a live migration in progress)
keeps the Protocol C override the old driver set on its resource
definition. The new driver does not remove it. Data is not affected, the
resource just stays on protocol C. Such resource definitions have the
`Aux/csi-protocol-override` property. To remove the override, run
`linstor resource-definition set-property <rd> DrbdOptions/Net/protocol`
and `linstor resource-definition set-property --aux <rd>
csi-protocol-override`.
- The CSIDriver moves to `fsGroupPolicy: File`, so `fsGroup` is also
applied to ReadWriteMany filesystem volumes.
- The operator manages Secrets only in its own namespace, and the CSI
controller loses cluster-wide read access to Secrets. Nothing in this
repository references snapshotter Secrets.
- The csi-health-monitor sidecar is removed.
- LINSTOR's default evict timeout goes from 1h to 12h; this package does
not set it.
- Volume group snapshots turn on only when
`groupsnapshot.storage.k8s.io/v1` is served. The snapshot controller
here still serves v1alpha1, so they stay off; #5239 changes that.
- IO limits can be set through `qos.linbit.com/{rbps,wbps,riops,wiops}`
StorageClass parameters. Enforcing them needs NRI enabled on the nodes
and the nri-volume-qos plugin, neither of which this PR adds.

Tested on a three-node Talos cluster, upgrading from the current main
packages to this branch:

- LINSTOR database migrated to 1.35.3, no faulty resources, no error
reports.
- Data written before the upgrade is intact, checked by checksum.
- A snapshot taken before the upgrade restores, and all three replicas
come up `UpToDate`; cloning and new snapshots work.
- KubeVirt live migration works for a ReplicatedBlock volume, including
one on a Protocol A resource group: LINSTOR switches the connection to
protocol C for the migration and restores A afterwards.
- A StorageClass with QoS parameters provisions, and the limits reach
the resource definition and the VolumeAttachment metadata.
- ZFS userspace in the satellite is now 2.4.4, matching the kernel
module.

Not tested: an arm64 build.

### Screenshots

No UI change.

### Downstream repositories

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(linstor): update piraeus-operator to v2.12.1, LINSTOR to 1.35.3 and LINSTOR CSI to v1.13.2. Fixes the NFS server breaking console login on storage nodes and adds IO QoS StorageClass parameters. LINSTOR 1.35 migrates its database, and a downgrade to an earlier version is not possible after the upgrade.
```