You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(linstor): update piraeus-operator to v2.12.1 - #5166
Updates the LINSTOR stack to the current stable piraeus-operator release: operator v2.12.1, LINSTOR server 1.35.3 and LINSTOR CSI v1.13.2, the last two built here with our patches. This started as a fix for the NFS server image hanging up the host console on bare-metal storage nodes (Ubuntu 24.04, Cozystack 1.6.5 and 1.7.0). That needed nfs-server v1.10.6 or later. Operators also asked for per-volume IO limits, which need linstor-csi v1.11.3 or later and piraeus-operator v2.10.8 or later, so the update goes to the current release rather than the minimum.
The first commit fixes a problem the update exposed. The operator runs the LINSTOR database migration in a run-migration init container, and this package pinned only the main controller container to our build. With the operator update, that init container ran the operator's default LINSTOR 1.35.3 and migrated the database before the controller image changed. The 1.33.3 controller then crash-looped until the linstor release caught up. Both init containers that run the LINSTOR server image (run-migration and the satellite's setup-lvm-configuration) now use the platform image, with a test pinning it. This protects later upgrades where the operator default is newer than our build. It does not help this upgrade: the pin ships in the linstor release, and Flux reconciles piraeus-operator first.
LINSTOR server: six of the nine patches are in upstream by 1.35.3 and are removed. fix-k8s-create-already-exists, retry-adjust-after-stale-bitmap and native-protoc are carried forward. LINSTOR 1.34 requires Java 21, so the image moves from bookworm to trixie.
Behaviour that changes for users:
No downgrade. LINSTOR 1.35 migrates the database and drops the SEC_* tables, so a cluster that has run this version cannot go back to 1.33.
The LINSTOR controller is down for a short time during this upgrade. piraeus-operator is reconciled before the linstor release, so the database is migrated to 1.35 while the old controller still runs. That controller crash-loops until the linstor release rolls out, and if the rollout fails there is no way back.
A volume on a Protocol A or B resource group that was attached to two nodes while the CSI driver was updated (a live migration in progress) keeps the Protocol C override the old driver set on its resource definition. The new driver does not remove it. Data is not affected, the resource just stays on protocol C. Such resource definitions have the Aux/csi-protocol-override property. To remove the override, run linstor resource-definition set-property <rd> DrbdOptions/Net/protocol and linstor resource-definition set-property --aux <rd> csi-protocol-override.
The CSIDriver moves to fsGroupPolicy: File, so fsGroup is also applied to ReadWriteMany filesystem volumes.
The operator manages Secrets only in its own namespace, and the CSI controller loses cluster-wide read access to Secrets. Nothing in this repository references snapshotter Secrets.
The csi-health-monitor sidecar is removed.
LINSTOR's default evict timeout goes from 1h to 12h; this package does not set it.
IO limits can be set through qos.linbit.com/{rbps,wbps,riops,wiops} StorageClass parameters. Enforcing them needs NRI enabled on the nodes and the nri-volume-qos plugin, neither of which this PR adds.
Tested on a three-node Talos cluster, upgrading from the current main packages to this branch:
LINSTOR database migrated to 1.35.3, no faulty resources, no error reports.
Data written before the upgrade is intact, checked by checksum.
A snapshot taken before the upgrade restores, and all three replicas come up UpToDate; cloning and new snapshots work.
KubeVirt live migration works for a ReplicatedBlock volume, including one on a Protocol A resource group: LINSTOR switches the connection to protocol C for the migration and restores A afterwards.
A StorageClass with QoS parameters provisions, and the limits reach the resource definition and the VolumeAttachment metadata.
ZFS userspace in the satellite is now 2.4.4, matching the kernel module.
Not tested: an arm64 build.
Screenshots
No UI change.
Downstream repositories
No downstream repository is affected by this change
fix(linstor): update piraeus-operator to v2.12.1, LINSTOR to 1.35.3 and LINSTOR CSI to v1.13.2. Fixes the NFS server breaking console login on storage nodes and adds IO QoS StorageClass parameters. LINSTOR 1.35 migrates its database, and a downgrade to an earlier version is not possible after the upgrade.
It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.
No actionable comments were generated in the recent review.
Walkthrough
The Piraeus chart and component image versions are updated. Chart templates set release namespaces and support additional volumes, mounts, and manifests. CRD descriptions clarify issuer references and toleration operators. The LINSTOR CSI image build uses a newer linstor-wait-until default.
Changes: CRD description updates
Layer / File(s)
Summary
Issuer references and toleration operators packages/system/piraeus-operator-crds/templates/crds.yaml
Updated cert-manager reference descriptions to identify issuers and document defaults. The toleration operator description now includes Lt and Gt, their numeric comparisons, and the required feature gate.
Piraeus Helm chart updates
Layer / File(s)
Summary
Chart release and component images packages/system/piraeus-operator/Chart.yaml, packages/system/piraeus-operator/charts/piraeus/Chart.yaml, packages/system/piraeus-operator/charts/piraeus/templates/config.yaml, packages/system/linstor/images/linstor-csi/Dockerfile, packages/system/linstor/tests/csi-topology_test.yaml
Updated chart and application versions, Piraeus and CSI image tags, module-loader fallback, and OS-image mappings. The parent chart now declares the local Piraeus chart as a dependency. The LINSTOR CSI image build default for linstor-wait-until is v1.0.0. The test comment records v2.10.7 as the last verification version.
Chart resources and the leader-election Role now specify the Helm release namespace. The controller-manager ClusterRole can read volumeattributesclasses.
Additional volumes and manifests packages/system/piraeus-operator/charts/piraeus/values.yaml, packages/system/piraeus-operator/charts/piraeus/templates/deployment.yaml, packages/system/piraeus-operator/charts/piraeus/templates/extra-objects.yaml
Added values and Deployment rendering for extra volumes and mounts. Added rendering for extraManifests supplied as a map or list.
cozystack/cozystack#5293 — The change updates the vendored operator version and matching NFS and CSI image versions, consistent with the issue’s version-alignment objective.
Merge Risk: Minimal · up to 03dd7
This updates the Piraeus operator chart and related component images to v2.10.7. No specific merge-blocking problem was identified in the supplied changes. As the PR notes, the upgrade can roll satellite, CSI node and NFS server workloads, so a normal rollout check is advisable.
Pre-merge checks: 3 · 1
Failed checks: 1 warning
Check name
Status
Explanation
Resolution
Title check
The title says the change updates piraeus-operator to v2.12.1. The changeset updates the bundled Piraeus chart to v2.10.7 (Chart.yaml version and appVersion), not v2.12.1. The title names a version th…
Change the title to match the actual version change, for example: "fix(linstor): update piraeus-operator chart to v2.10.7". If v2.12.1 is the intended target, update the chart files and the PR so the version matches the title.
Passed checks: 3 passed
Check name
Status
Explanation
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Description Check
Check skipped - CodeRabbit’s high-level summary is enabled.
Full details: Title check
Explanation
The title says the change updates piraeus-operator to v2.12.1. The changeset updates the bundled Piraeus chart to v2.10.7 (Chart.yaml version and appVersion), not v2.12.1. The title names a version that does not appear in the changes, so it misstates the main change. The change also includes related image bumps, a DRBD module loader change, RBAC and namespace updates, and a linstor-wait-until bump in the Dockerfile, which the title does not mention.
Resolution
Change the title to match the actual version change, for example: "fix(linstor): update piraeus-operator chart to v2.10.7". If v2.12.1 is the intended target, update the chart files and the PR so the version matches the title.
Finishing Touches
Generate unit tests (beta)
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 3 remain after this review.
github-actionsBot
added
size/XXL
This PR changes 1000+ lines, ignoring generated files
and removed
size/M
This PR changes 30-99 lines, ignoring generated files
labels
Nov 13, 2026
Aleksei Sviridkin (lexfrei)
changed the title
fix(linstor): update piraeus-operator to v2.10.7
fix(linstor): update piraeus-operator to v2.12.1
Nov 13, 2026
The reason will be displayed to describe this comment to others. Learn more.
Verdict
LGTM with non-blocking notes.
No introduced regression on any supported path. The migration init-container fix, the RBAC reduction, the CRD relocation and the carried patches all hold under direct checks. The items below are offline-review limits plus a couple of consistency notes, not defects.
Checked and holds:
Init-container pin (the core fix). Operator v2.12.1 names its init containers run-migration (controller) and setup-lvm-configuration (satellite), and the podTemplate merges by name, so pinning those to piraeusServer.image replaces the operator's default image rather than adding a second container. Reverting either pin turns server-image_test.yaml red, so the test guards the fix.
RBAC reduction.charts/piraeus/rbac.yaml is a faithful re-vendor of upstream v2.12.1: cluster-wide secrets read drops to a namespaced Role, pods/exec is added; net is a privilege reduction on a system-tier SA that is not tenant-reachable. No snapshotter Secret reference exists in the repo, so losing cluster-wide Secret read is safe for the shipped config.
CRD relocation. The deleted charts/piraeus/templates/crds.yaml was already inert (installCRDs: false); piraeus-operator-crds still serves the same kinds and its schema change is additive (no new required, no version/storage flip). No shrink or downgrade.
charts-direct-edit (flagged MINOR). False positive: every charts/piraeus/* file is byte-identical to upstream v2.12.1 except the scripted crds.yaml removal make update reproduces.
Carried patches. The three server patches and the csi relocate-after-clone patch git apply --check clean against their pinned upstreams, and Build packages/system/linstor is green, so the bookworm-to-trixie move builds.
Notes (non-blocking):
Image digests are not re-pinned.packages/system/linstor/values.yaml still pins piraeus-server and linstor-csi at the v1.6.0 build (LINSTOR 1.33.3 / CSI 1.11.2); the bump lives only in the Makefile, Dockerfiles and patches. This reaches no tested path, since the PR e2e rebuilds each image and patches the digest for the run and the release tag bake runs make image per unit. But the prior analogous bump (6ac1340) re-pinned in the same commit, so main is left momentarily inconsistent (operator chart v2.12.1 against images 1.33.3 / 1.11.2) for anyone installing raw main by committed digest. Confirm the release bake is the intended re-pin path, or re-pin here as before.
User-defined VolumeSnapshotClass with snapshotter secrets. Safe for everything this repo ships, but a cluster operator who added their own class with csi.storage.k8s.io/snapshotter-secret-* would get forbidden on CreateSnapshot/DeleteSnapshot once cluster-wide Secret read is dropped; upstream's upgrade doc notes this. One line in the upgrade notes covers it if custom snapshot classes are supported.
fsGroupPolicy: File is mutable on k8s >= 1.30, so the switch updates in place. The disclosed behaviour change stands: a pod with fsGroup and no fsGroupChangePolicy: OnRootMismatch can be slow to start on a large RWX volume.
Live-upgrade convergence was not exercised offline. SSA field ownership on the live controller/CSI Deployments can't be proven from a render; a cozystack-pr-test upgrade replay (current main to this branch) through Flux would confirm it.
CI at review time.Build packages/system/linstor green; Unit & controller tests still in progress (helm half green locally, 24/24); no E2E on this head (needs full-e2e). Confirm the unit job green and run full e2e before merge.
Disclosed and understood: downgrade is one-way after the 1.35 migration drops SEC_*, and the arm64 build is untested.
piraeus-operator fills in its own default LINSTOR server image for every
container this package leaves unset, and two of those are init
containers: run-migration in the controller pod and
setup-lvm-configuration in the satellite pods. Only the main containers
were pinned to the platform build.
run-migration upgrades the LINSTOR database in place. When the operator
default is newer than the platform build, it migrates the schema ahead
of the controller, the controller then fails to read the migrated
database and crash-loops, and the migration cannot be rolled back. This
stayed hidden while the operator default was older than the version
built here; an operator update that moves the default past it exposes
it during the window between the operator and linstor releases
reconciling.
Pin both init containers to the same image as the main containers, so
the database moves only together with the controller.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <[email protected]>
The nfs-server DaemonSet runs piraeus-csi-nfs-server v1.10.3, which
boots systemd as PID 1 in a privileged container with the host /dev
mounted. Its getty units hang up the host's getty@tty1 about a second
after the pod starts; systemd restarts the host getty until it hits
the start limit and gives up, so login on the local or out-of-band
console stops working on nodes that run the NFS server. Seen on
bare-metal Ubuntu 24.04 nodes with Cozystack 1.6.5 and 1.7.0.
Upstream fixed the image by disabling the services the NFS server does
not need (piraeusdatastore/linstor-csi#417, reported as
piraeusdatastore/piraeus-operator#961); v1.10.6 includes the fix.
v2.12.1 is the current stable release. Its image set is LINSTOR
1.35.3 and LINSTOR CSI v1.13.2, which this repository builds itself;
both builds follow in the next commits. The CSI release line is what
carries IO QoS limits (qos.linbit.com/* StorageClass parameters,
added in v1.11.3), which operators have asked for.
Behaviour that changes with this range:
- The CSIDriver moves to fsGroupPolicy: File, so fsGroup is applied
to ReadWriteMany filesystem volumes too.
- The operator manages Secrets only in its own namespace, and the CSI
controller loses its cluster-wide read on Secrets; nothing in this
repository references snapshotter Secrets.
- The csi-health-monitor sidecar is removed.
- Volume group snapshots are enabled only when the
groupsnapshot.storage.k8s.io/v1 API is served; the snapshot
controller here still ships v1alpha1, so they stay off.
The csi-provisioner arguments are unchanged in this range, so the
restated strictTopology list stays complete.
The chart is vendored the way `make update` does it, which also drops
the stale CRD copy from the vendored chart; piraeus-operator-crds
installs the CRDs. Declare the local chart dependency so Helm can
validate the umbrella chart without changing its rendered resources.
This commit is not deployable on its own. The operator now hands the
CSI pods their client certificate through LS_USER_CERTIFICATE_FILE and
LS_USER_KEY_FILE, which the linstor-wait-until in the current
linstor-csi image does not read, so the CSI init containers retry
forever until the linstor-csi update in the next commit.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <[email protected]>
piraeus-operator v2.12.1 is released against LINSTOR CSI v1.13.2, and
the release line carries IO QoS limits through the qos.linbit.com/*
StorageClass parameters.
The operator gives the CSI pods their client certificate as mounted
files through LS_USER_CERTIFICATE_FILE and LS_USER_KEY_FILE, instead
of inline values in LS_USER_CERTIFICATE and LS_USER_KEY. The
linstor-wait-until v0.3.1 in this image is built on a golinstor
release that reads only the inline variables, so its init containers
present no client certificate, the LINSTOR API closes the TLS
connection, and they retry on EOF forever. Move to
linstor-wait-until v1.0.1, the version upstream v1.13.2 pins.
The Protocol C override for dual attach is dropped. Since v1.12 the
driver no longer sets allow-two-primaries itself: it asks LINSTOR to
manage dual-primary through make-available, and LINSTOR 1.35 sets
protocol C on the connections that need it. Upstream closed the
equivalent change as superseded for that reason
(piraeusdatastore/linstor-csi#435).
The relocate-after-clone patch is carried forward. Volume IDs became
a struct for consistency groups; Clone and VolFromSnap handle
standalone resources, so relocation keys on the resource name. The
enumer output is regenerated rather than hand-patched.
Upstream now pins grpc v1.84.0, which builds on Go 1.27 and includes
the fix for GO-2026-6443, so the grpc override is no longer needed.
The golden-image clone guard in vm-disk was re-checked: parameter
parsing for allowRemoteVolumeAccess and layerList, and the default
derived from the layer list, are unchanged in this range.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <[email protected]>
piraeus-operator v2.12.1 ships LINSTOR 1.35.3 as its controller and
satellite version, and the in-repo piraeus-server build has to match it.
LINSTOR 1.34 raised the minimum Java to 21, so the build and runtime
stages move from bookworm to trixie, and the system Gradle used by
debian/rules follows the upstream wrapper to 8.14.2.
Six carried patches are part of upstream by v1.35.3 and are dropped;
the stale-bitmap adjust retry is rebased onto the new DrbdAdm.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
area/storageIssues or PRs related to storage (linstor, seaweedfs, bucket, velero, harbor)kind/backportCategorizes issue or PR as requiring a backport to the current release linekind/backport-previousCategorizes issue or PR as requiring a backport to the previous release linekind/bugCategorizes issue or PR as related to a bugsize/XXLThis PR changes 1000+ lines, ignoring generated files
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Aleksei Sviridkin (lexfrei)
added
kind/bug
github-actions
Bot
added
the
size/M
IvanHunters
left a comment
What this PR does
Updates the LINSTOR stack to the current stable piraeus-operator release: operator v2.12.1, LINSTOR server 1.35.3 and LINSTOR CSI v1.13.2, the last two built here with our patches. This started as a fix for the NFS server image hanging up the host console on bare-metal storage nodes (Ubuntu 24.04, Cozystack 1.6.5 and 1.7.0). That needed nfs-server v1.10.6 or later. Operators also asked for per-volume IO limits, which need linstor-csi v1.11.3 or later and piraeus-operator v2.10.8 or later, so the update goes to the current release rather than the minimum.
The first commit fixes a problem the update exposed. The operator runs the LINSTOR database migration in a
run-migrationinit container, and this package pinned only the main controller container to our build. With the operator update, that init container ran the operator's default LINSTOR 1.35.3 and migrated the database before the controller image changed. The 1.33.3 controller then crash-looped until the linstor release caught up. Both init containers that run the LINSTOR server image (run-migrationand the satellite'ssetup-lvm-configuration) now use the platform image, with a test pinning it. This protects later upgrades where the operator default is newer than our build. It does not help this upgrade: the pin ships in the linstor release, and Flux reconciles piraeus-operator first.Patches:
fix-k8s-create-already-exists,retry-adjust-after-stale-bitmapandnative-protocare carried forward. LINSTOR 1.34 requires Java 21, so the image moves from bookworm to trixie.Behaviour that changes for users:
SEC_*tables, so a cluster that has run this version cannot go back to 1.33.Aux/csi-protocol-overrideproperty. To remove the override, runlinstor resource-definition set-property <rd> DrbdOptions/Net/protocolandlinstor resource-definition set-property --aux <rd> csi-protocol-override.fsGroupPolicy: File, sofsGroupis also applied to ReadWriteMany filesystem volumes.groupsnapshot.storage.k8s.io/v1is served. The snapshot controller here still serves v1alpha1, so they stay off; feat(linstor, platform): upgrade snapshot-controller to v8.7.0 and enable VolumeGroupSnapshots #5239 changes that.qos.linbit.com/{rbps,wbps,riops,wiops}StorageClass parameters. Enforcing them needs NRI enabled on the nodes and the nri-volume-qos plugin, neither of which this PR adds.Tested on a three-node Talos cluster, upgrading from the current main packages to this branch:
UpToDate; cloning and new snapshots work.Not tested: an arm64 build.
Screenshots
No UI change.
Downstream repositories
Release note