### Walkthrough: Setup and Usage Commands Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/examples/consistency-groups/README.md Step-by-step commands to set up RBAC, StorageClass, golden image, VMs, snapshots, and restore. Note the caveat about applying VM disks back-to-back causing contention on the RWO golden; wait for the first clone to succeed or give the golden multi-node read access. ```sh # 1. RBAC + StorageClass (all disks of one VM must use the same StorageClass — see caveats). The StorageClass # carries the cdi.kubevirt.io/clone-strategy: copy annotation, so CDI host-assisted-clones the golden and the # cloned root disks can join a consistency group. kubectl apply -f rbac.yaml kubectl apply -f storageclass.yaml # 2. Import the golden image ONCE. Every VM's root disk is a clone of this PVC. kubectl apply -f golden-image.yaml kubectl get datavolume fedora-golden -w # wait for phase: Succeeded # 3. VM 1's disks: root (a host-assisted clone of the golden) + a blank data disk. Both carry the # consistency-group label 'fedora-vm-1', so they land in one LINSTOR resource. kubectl apply -f vm1-disks.yaml # Confirm both disks resolved to the same cg- resource at different volume numbers: # linstor volume-definition list # You should see one cg- resource with two volume definitions (0 and 1). # 4. VM 1 itself. Log in on the console as fedora / password: kubectl apply -f vm1.yaml # virtctl console fedora-vm-1 # 5. VM 2 — an independent second VM from the SAME golden (its own group 'fedora-vm-2'), for comparison. kubectl apply -f vm2-disks.yaml kubectl apply -f vm2.yaml # 6. Snapshot VM 1's whole group, crash-consistent. kubectl apply -f volume-group-snapshot-class.yaml kubectl apply -f volume-group-snapshot.yaml # 7. Restore VM 1's group into a fresh set of disks (see restore-vm.yaml header for the snapshot-name lookup). kubectl apply -f restore-vm.yaml ``` -------------------------------- ### NewHighLevelClient usage example Source: https://github.com/piraeusdatastore/linstor-csi Example of creating a HighLevelClient with a base URL, bearer token, and user agent. ```go import lc "github.com/piraeusdatastore/linstor-csi/pkg/linstor/highlevelclient" hlc, err := lc.NewHighLevelClient( lapi.BaseURL(u), lapi.BearerToken(token), lapi.UserAgent("my-app/1.0"), ) ``` -------------------------------- ### CapacityBytes usage example Source: https://github.com/piraeusdatastore/linstor-csi Shows how to call CapacityBytes with a nil storagePools slice, an overProvision ratio pointer, and a topology segment map. The example uses a zone topology key. ```go cap, err := c.CapacityBytes(ctx, nil, &ratio, map[string]string{"topology.kubernetes.io/zone": "us-1"}) ``` -------------------------------- ### GetNodeTopologies usage example Source: https://github.com/piraeusdatastore/linstor-csi Example call to GetNodeTopologies showing the resulting topology segments for a node. ```go topo, err := c.GetNodeTopologies(ctx, "worker-3") // topo.Segments["linbit.com/hostname"] == "worker-3" // topo.Segments["linbit.com/sp-ssd"] == "true" ``` -------------------------------- ### CreateVolume example call with AlreadyExists handling Source: https://github.com/piraeusdatastore/linstor-csi Example invocation of CreateVolume with placement count and storage pool parameters, including handling of AlreadyExists status. ```go resp, err := d.CreateVolume(ctx, &csi.CreateVolumeRequest{ Name: "pvc-1234", VolumeCapabilities: []*csi.VolumeCapability{{ AccessMode: &csi.VolumeCapability_AccessMode{Mode: csi.VolumeCapability_AccessMode_SINGLE_NODE_WRITER}, }}, CapacityRange: &csi.CapacityRange{RequiredBytes: 10 * 1024 * 1024 * 1024}, Parameters: map[string]string{ "linstor.csi.linbit.com/placementCount": "2", "linstor.csi.linbit.com/storagePool": "ssd", }, }) if status.Code(err) == codes.AlreadyExists { // reconcile with existing volume or surface to caller } ``` -------------------------------- ### SnapshotType usage example Source: https://github.com/piraeusdatastore/linstor-csi Shows how to convert a string to a SnapshotType using SnapshotTypeString and get the string name of a SnapshotType constant. ```go t, err := volume.SnapshotTypeString("S3") // SnapshotTypeS3 s := volume.SnapshotTypeLinstor.String() // "Linstor" ``` -------------------------------- ### Listen: Example usage with unix socket path Source: https://github.com/piraeusdatastore/linstor-csi Example call to Listen with a unix domain socket URL. On error, the program exits with log.Fatal. ```go l, err := driver.Listen("unix:///var/lib/kubelet/plugins/linstor.csi.linbit.com/csi.sock") if err != nil { log.Fatal(err) } ``` -------------------------------- ### NodePublishVolume example call Source: https://github.com/piraeusdatastore/linstor-csi Example invocation of NodePublishVolume with a typical request. Handles NotFound error when assignment is missing on the node. ```go resp, err := d.NodePublishVolume(ctx, &csi.NodePublishVolumeRequest{ VolumeId: volID, TargetPath: "/var/lib/kubelet/pods/xyz/volumes/linstor", VolumeCapability: &csi.VolumeCapability{ AccessMode: &csi.VolumeCapability_AccessMode{ Mode: csi.VolumeCapability_AccessMode_SINGLE_NODE_WRITER, }, }, VolumeContext: volCtx, PublishContext: pubCtx, }) if status.Code(err) == codes.NotFound { // assignment not found on this node } ``` -------------------------------- ### NewParameters usage example Source: https://github.com/piraeusdatastore/linstor-csi Shows how to call NewParameters with a map of storage-class parameters and a topology prefix. The comment indicates expected parsed values for PlacementCount and Properties. ```go params, err := volume.NewParameters(map[string]string{ "linstor.csi.linbit.com/placementCount": "2", "linstor.csi.linbit.com/storagePool": "ssd", "property.linstor.csi.linbit.com/DrbdOptions/auto-quorum": "suspend-io", }, "Aux") // params.PlacementCount == 2; params.Properties["DrbdOptions/auto-quorum"] == "suspend-io" ``` -------------------------------- ### Fsck usage example Source: https://github.com/piraeusdatastore/linstor-csi Shows how to call Fsck on a device and wrap errors. ```go if err := utils.Fsck(ctx, "/dev/drbd1000"); err != nil { return fmt.Errorf("fsck failed: %w", err) } ``` -------------------------------- ### PublishContext ToMap usage example Source: https://github.com/piraeusdatastore/linstor-csi Shows how to create a PublishContext and serialize it to a map, which is then consumed by ControllerNodePublish. ```go pub := (&PublishContext{ DevicePath: "/dev/drbd1000", FsType: "ext4", }).ToMap() // ControllerNodePublish consumes it via PublishContextFromMap. ``` -------------------------------- ### VolumeContext ToMap usage example Source: https://github.com/piraeusdatastore/linstor-csi Shows how to create a VolumeContext from parameters and serialize it to a map, with the marker key set to 'true'. ```go vc := VolumeContextFromParameters(¶ms) m, err := vc.ToMap() if err != nil { return nil, err } // m["linstor.csi.linbit.com/uses-volume-context"] == "true" ``` -------------------------------- ### AggregateGroupSnapshot usage example Source: https://github.com/piraeusdatastore/linstor-csi Example call to AggregateGroupSnapshot with two snapshots, showing the resulting GroupSnapshotId and per-snapshot SourceVolumeId. ```go gs := AggregateGroupSnapshot("gs-1", snap1, snap2) // gs.GroupSnapshotId == "gs-1"; gs.Snapshots[i].SourceVolumeId is each source ``` -------------------------------- ### Iterating consistency group volumes Source: https://github.com/piraeusdatastore/linstor-csi Example loop that prints volume numbers and CSI volume names from volume definitions using ConsistencyGroupVolumes. ```go for vnr, csiName := range util.ConsistencyGroupVolumes(rd.VolumeDefinitions...) { fmt.Println(vnr, csiName) } ``` -------------------------------- ### NewDriver usage example Source: https://github.com/piraeusdatastore/linstor-csi Shows how to create a driver, set options, listen on a Unix socket, and serve. Requires a LINSTOR client and context. ```go import ( linstorClient "github.com/piraeusdatastore/linstor-csi/pkg/client" "github.com/piraeusdatastore/linstor-csi/pkg/driver" ) client, err := linstorClient.NewLinstor() if err != nil { return err } drv, err := driver.NewDriver( client, driver.NodeID("k8s-worker-1"), driver.TopologyPrefix("Aux"), driver.LogLevel("debug"), ) if err != nil { return fmt.Errorf("failed to create driver: %w", err) } listener, err := driver.Listen("unix:///tmp/csi.sock") if err != nil { return err } if err := drv.Serve(ctx, listener); err != nil { return err } ``` -------------------------------- ### Attach usage example Source: https://github.com/piraeusdatastore/linstor-csi Shows how to call Attach with a volume ID, node name, and rwxBlock flag. The error handling comment notes possible failures like node offline or no diskless storage pool. ```go devPath, err := c.Attach(ctx, vol.ID, "worker-3", false) if err != nil { // e.g. node offline, no diskless storage pool } ``` -------------------------------- ### Pass PVC metadata to the driver Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/examples/consistency-groups/README.md Run the external-provisioner with this flag so the driver learns the PVC name/namespace. This is the default in recent piraeus-operator deployments. ```text --extra-create-metadata=true ``` -------------------------------- ### KubernetesClient usage example Source: https://github.com/piraeusdatastore/linstor-csi Shows how to call KubernetesClient and handle the error case when not running in a cluster. ```go kubeTyped, kubeSnap, err := utils.KubernetesClient() if err != nil { // not running in a cluster } ``` -------------------------------- ### CreateSnapshot request example Source: https://github.com/piraeusdatastore/linstor-csi Example request for CreateSnapshot with S3 parameters and secrets. The resulting SnapshotId is expected to be "s3://my-s3-remote/vol-123/snap-1". ```go req := &csi.CreateSnapshotRequest{ Name: "snap-1", SourceVolumeId: "vol-123", Parameters: map[string]string{"snap.linstor.csi.linbit.com/type": "S3", "snap.linstor.csi.linbit.com/remote-name": "my-s3-remote"}, Secrets: map[string]string{"access-key": "AK", "secret-key": "SK"}, } resp, err := d.CreateSnapshot(ctx, req) // resp.Snapshot.SnapshotId == "s3://my-s3-remote/vol-123/snap-1" ``` -------------------------------- ### EnsureMinimumApiVersion usage example Source: https://github.com/piraeusdatastore/linstor-csi Example of calling EnsureMinimumApiVersion and handling the error, which includes a message like 'LINSTOR 1.30 (REST API 1.27) is not supported: ...'. ```go if err := hlc.EnsureMinimumApiVersion(ctx); err != nil { log.Fatal(err) // e.g. "LINSTOR 1.30 (REST API 1.27) is not supported: ..." } ``` -------------------------------- ### Unexport usage example Source: https://github.com/piraeusdatastore/linstor-csi Calls Unexport and logs a warning with the volume ID if it fails. ```go if err := exporter.Unexport(ctx, volId); err != nil { log.Warnf("failed to unexport %s: %v", volId, err) } ``` -------------------------------- ### AccessibleSegments usage example Source: https://github.com/piraeusdatastore/linstor-csi Shows how to use AccessibleSegments with a zone key; the empty rule yields the empty segment, meaning access from anywhere. ```go pol := RemoteAccessPolicy{{FromSame: []string{"topology.kubernetes.io/zone"}}} segs := pol.AccessibleSegments(map[string]string{"topology.kubernetes.io/zone": "us-1"}) // segs == [map[string]string{"topology.kubernetes.io/zone": "us-1"}] ``` -------------------------------- ### ControllerPublishVolume usage example with NFS export Source: https://github.com/piraeusdatastore/linstor-csi Shows how to call ControllerPublishVolume with an NFS export volume context. The publish context contains the device path and file system type as NFS values. ```go pub, err := d.ControllerPublishVolume(ctx, &csi.ControllerPublishVolumeRequest{ VolumeId: volId, NodeId: node, VolumeCapability: cap_, VolumeContext: map[string]string{"linstor.csi.linbit.com/nfs-export": "nfs://svc.ns.svc:2049/vol"}, }) // pub.PublishContext["linstor.csi.linbit.com/device-path"] == "nfs://svc.ns.svc:2049/vol" // pub.PublishContext["linstor.csi.linbit.com/fs-type"] == "nfs" ``` -------------------------------- ### RWXBlockValidator usage example Source: https://github.com/piraeusdatastore/linstor-csi Shows how to create a validator and call ValidateAttachment, handling the error case. ```go validator, err := utils.NewRWXBlockValidator(ctx, kube, 5*time.Minute, logger) if err != nil { return err } vm, err := validator.ValidateAttachment(ctx, volumeID) if err != nil { // e.g. "RWX block volume ns/pvc is being used by pods from different VMs ..." } ``` -------------------------------- ### Export usage example Source: https://github.com/piraeusdatastore/linstor-csi Calls Export and returns an error wrapping 'unable to export' on failure. On success, u.String() equals 'nfs://linstor-csi-nfs.storage.svc:2049/myvol'. ```go u, err := exporter.Export(ctx, info, ¶ms) if err != nil { return nil, fmt.Errorf("unable to export: %w", err) } // u.String() == "nfs://linstor-csi-nfs.storage.svc:2049/myvol" ``` -------------------------------- ### NewSnapshotParameters usage example Source: https://github.com/piraeusdatastore/linstor-csi Creates a SnapshotParameters with type S3 and remote-name 'bucket', using secrets for access-key and secret-key. ```go sp, err := volume.NewSnapshotParameters( map[string]string{"snap.linstor.csi.linbit.com/type": "S3", "snap.linstor.csi.linbit.com/remote-name": "bucket"}, map[string]string{"access-key": "AK", "secret-key": "SK"}, ) ``` -------------------------------- ### Enable consistency groups on CSI controller Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/examples/consistency-groups/README.md Add this flag to the linstor-csi container of the CSI controller Deployment to enable the consistency group feature. ```text --enable-consistency-groups ``` -------------------------------- ### Basic StorageClass for LINSTOR CSI Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/README.md A basic Kubernetes StorageClass that uses the linstor.csi.linbit.com provisioner. Includes parameters for placement count, storage pool, resource group, filesystem type, and an example of overriding LINSTOR properties with the property.linstor.csi.linbit.com prefix. ```yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: linstor-basic-storage provisioner: linstor.csi.linbit.com allowVolumeExpansion: true parameters: linstor.csi.linbit.com/placementCount: "2" linstor.csi.linbit.com/storagePool: "my-storage-pool" linstor.csi.linbit.com/resourceGroup: "linstor-basic-storage" csi.storage.k8s.io/fstype: xfs # You can override LINSTOR properties by adding the property.linstor.csi.linbit.com prefix: property.linstor.csi.linbit.com/DrbdOptions/auto-quorum: suspend-io ``` -------------------------------- ### LogLevel option failure example Source: https://github.com/piraeusdatastore/linstor-csi Demonstrates that an invalid log level causes NewDriver to return an error, aborting construction. ```go // Options run in order; LogLevel failure aborts construction. _, err := driver.NewDriver(client, driver.LogLevel("not-a-level")) // err: unable to use not-a-level as a logging level: ... ``` -------------------------------- ### DRBD Reactor Promoter Configuration for NFS Volume Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/docs/design/read-write-many.md TOML configuration for a DRBD Reactor Promoter that manages an NFS volume. It defines services to prepare device links, clean stale endpoints, mount recovery and export volumes, grow filesystems, set permissions, start NFS Ganesha, and advertise the endpoint. The configuration uses systemd escaping not applied to template parameters for readability. ```toml [[promoters]] [promoter.resources.vol1] runner = "systemd" target-as = "BindsTo" stop-services-on-exit = true start = [ "prepare-device-links@pvc-12345.service", "clean-nfs-endpoint@pvc-12345.service", "mount-recovery@pvc-12345.service", "mount-export@pvc-12345.service", "growfs@/srv/exports/pvc-12345.timer", "chmod@/srv/exports/pvc-12345.service", "nfs-ganesha@pvc-12345.service", "advertise-nfs-endpoint@pvc-12345.service", ] [promoter.resources.vol1.metadata] service-name = "linstor-csi-nfs" port = 2345 export-id = 2345 config-template-path = "/etc/nfs-helper/default-config.tmpl" squash = "no_root_squash" ``` -------------------------------- ### ParseVolumeId function and usage examples Source: https://github.com/piraeusdatastore/linstor-csi Parses a CSI volume ID. Rejects empty strings, missing resource names, and volume numbers that are not integers or fall outside [0, MaxVolumesPerResource). The examples show valid parses. ```go func ParseVolumeId(id string) (ID, error) ``` ```go id, err := volume.ParseVolumeId("myvol/3") // ID{myvol, 3} id, err = volume.ParseVolumeId("cg-abc") // ID{cg-abc, 0} ``` -------------------------------- ### ToDiskfullResourceCreate Source: https://github.com/piraeusdatastore/linstor-csi Prepares a Info to be deployed by linstor on a node with local storage. ```APIDOC ## ToDiskfullResourceCreate ### Description Prepares a Info to be deployed by linstor on a node with local storage. ### Method Function ### Signature ```go func (params *Parameters) ToDiskfullResourceCreate(id, node string) (lapi.ResourceCreate, error) ``` ### Parameters - **id** (string) - Required - Identifier for the resource. - **node** (string) - Required - Node where the resource will be deployed. ``` -------------------------------- ### CLI flag binding example Source: https://github.com/piraeusdatastore/linstor-csi Binds DefaultRemoteAccessPolicy to a CLI flag named "default-remote-access-policy". ```go flag.Var(&volume.DefaultRemoteAccessPolicy, "default-remote-access-policy", "") ``` -------------------------------- ### ControllerGetVolume Source: https://github.com/piraeusdatastore/linstor-csi Gets a volume's information. Returns NotFound when the volume is missing; otherwise returns VolumeId and CapacityBytes. ```APIDOC ## ControllerGetVolume ### Description `NotFound` when the volume is missing; otherwise returns `VolumeId` and `CapacityBytes`. ### Method Unary RPC ### Parameters - **VolumeId** (string) - Required - The ID of the volume to retrieve. ### Response - **csi.ControllerGetVolumeResponse** - The response object containing `VolumeId` and `CapacityBytes`. ``` -------------------------------- ### Prepare Alpine VM image with qemu-img and qemu-system-x86_64 Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/examples/kubevirt/README.md Creates a raw Alpine VM image for use with kubevirt. Requires an Alpine ISO and KVM support. The image is compressed with xz and copied to a storage node. ```bash $ qemu-img create -f qcow2 alpine.qcow2 950M $ qemu-system-x86_64 -m 512 -nic user -boot d -cdrom alpine-virt-3.11.5-x86_64.iso -enable-kvm -hda alpine.qcow2 # setup-alpine; no ssh; reboot; disable the NIC/eth0 in /etc/network/interfaces $ qemu-img convert -O raw alpine.qcow2 -f qcow2 alpine.raw $ xz alpine.raw # scp alpine.raw.xz to one of the storage nodes/hypervisors ``` -------------------------------- ### ToDisklessResourceCreate Source: https://github.com/piraeusdatastore/linstor-csi Prepares a Info to be deployed by linstor on a node without local storage. Adds KeyStorPoolName: params.DisklessStoragePool and the diskless flag from DisklessFlag() (errors when the layer list has no diskless flag). ```APIDOC ## ToDisklessResourceCreate ### Description Prepares a Info to be deployed by linstor on a node without local storage. Adds KeyStorPoolName: params.DisklessStoragePool and the diskless flag from DisklessFlag() (errors when the layer list has no diskless flag). ### Method Function ### Signature ```go func (params *Parameters) ToDisklessResourceCreate(id, node string) (lapi.ResourceCreate, error) ``` ### Parameters - **id** (string) - Required - Identifier for the resource. - **node** (string) - Required - Node where the resource will be deployed. ``` -------------------------------- ### Listen: Create a unix domain socket listener for the CSI endpoint Source: https://github.com/piraeusdatastore/linstor-csi Creates a listener for the given CSI endpoint, which must be a unix domain socket URL. A stale socket file from a previous run is removed first. Binding before Serve runs avoids racing clients against server startup. Returns an error for non-unix schemes, stale socket removal failures, or net.Listen errors. ```go // Listen creates a listener for the given CSI endpoint, which must be a unix // domain socket URL. A stale socket file from a previous run is removed first. // Binding before Serve runs avoids racing clients against server startup. func Listen(endpoint string) (net.Listener, error) ``` -------------------------------- ### VolumeContextFromParameters function signature Source: https://github.com/piraeusdatastore/linstor-csi Builds a VolumeContext from parsed volume parameters, extracting mount options, post-mount xfs options, and remote access policy. ```go func VolumeContextFromParameters(params *volume.Parameters) *VolumeContext ``` -------------------------------- ### ListVolumes Source: https://github.com/piraeusdatastore/linstor-csi Lists all volumes. Paginates with StartingToken and MaxEntries; returns Aborted on an invalid token. ```APIDOC ## ListVolumes ### Description Lists all volumes via `linstorClient.ListAll` and paginates with `StartingToken`/`MaxEntries` (`Aborted` on an invalid token). Entries contain `VolumeId` and `CapacityBytes` only — topology is deliberately excluded. ### Method Unary RPC ### Parameters - **StartingToken** (string) - Optional - The token for pagination. - **MaxEntries** (int32) - Optional - The maximum number of entries to return. ### Response - **csi.ListVolumesResponse** - The response object containing volume entries. ``` -------------------------------- ### Attach method signature Source: https://github.com/piraeusdatastore/linstor-csi Defines the Attach method that idempotently creates a resource on a node. Returns the device path of the volume on the node. ```go // Attach idempotently creates a resource on the given node. func (s *Linstor) Attach(ctx context.Context, id volume.ID, node string, rwxBlock bool) (string, error) ``` -------------------------------- ### PrunePattern function Source: https://github.com/piraeusdatastore/linstor-csi Removes duplicates and patterns already covered by a more general pattern. Example: [{a:1,b:1}, {a:1}] becomes [{a:1}]. ```go // PrunePattern returns the given list of pattern, removed of any duplicates or patterns which are already covered by // a more general pattern. func PrunePattern(sources ...map[string]string) []map[string]string ``` -------------------------------- ### nfs-helper dispatch function (cmd/nfs-helper/main.go) Source: https://github.com/piraeusdatastore/linstor-csi Dispatches the first argument to the corresponding subcommand handler. Returns an error with 'unknown command: ' for unrecognized commands. ```go // dispatch (cmd/nfs-helper/main.go) func run(ctx context.Context, args ...string) error { switch args[0] { case "start-stop-reactor": return startStopReactor(ctx, args[1:]) case "generate-ganesha-config": return generateGaneshaConfig(ctx, args[1:]) case "mount": return mount(ctx, args[1:]) case "prepare-device-links": return prepareDeviceLinks(ctx, args[1:]) case "growfs": return growfs(ctx, args[1:]) case "advertise": return advertise(ctx, args[1:]) case "check-socket": return checkSocket(ctx, args[1:]) default: return fmt.Errorf("unknown command: %s", args[0]) } } ``` -------------------------------- ### Utility function signatures in pkg/linstor/util Source: https://github.com/piraeusdatastore/linstor-csi Declares utility functions for checking diskful deployment, extracting DRBD layers, iterating consistency group volumes, and determining storage pool space sharing. Includes comments on behavior. ```go func DeployedDiskfullyNodes(res []lapi.Resource) []string // DeployedDiskfully returns true if the resource has volumes that are physically // present and the resource state is healthy. func DeployedDiskfully(res lapi.Resource) bool func GetDrbdLayer(layer *lapi.ResourceLayer) *lapi.DrbdResource func ConsistencyGroupVolumes(vds ...lapi.VolumeDefinition) iter.Seq2[int, string] func ConsistencyGroupVolumeNumberFor(volumeName string, vds ...lapi.VolumeDefinition) iter.Seq[int] func SpaceName(sp *lapi.StoragePool) string // IsSharedSpace reports whether the storage pool's space is shared with other nodes. func IsSharedSpace(sp *lapi.StoragePool) bool ``` -------------------------------- ### GetNodeTopologies function signature Source: https://github.com/piraeusdatastore/linstor-csi Builds the csi.Topology for a node: always includes linbit.com/hostname, plus storage-pool labels when labelBySP is set, and node properties under PropertyNamespace. ```go func (s *Linstor) GetNodeTopologies(ctx context.Context, nodename string) (*csi.Topology, error) ``` -------------------------------- ### GetPluginCapabilities: Advertise supported capabilities Source: https://github.com/piraeusdatastore/linstor-csi Advertises CONTROLLER_SERVICE, VOLUME_ACCESSIBILITY_CONSTRAINTS, GROUP_CONTROLLER_SERVICE, and ONLINE volume expansion. ```go func (d *Driver) GetPluginCapabilities(ctx context.Context, req *csi.GetPluginCapabilitiesRequest) (*csi.GetPluginCapabilitiesResponse, error) ``` -------------------------------- ### GetNodeTopologies Source: https://github.com/piraeusdatastore/linstor-csi Builds the csi.Topology for a node: always includes linbit.com/hostname, plus storage pool labels and node properties under the configured PropertyNamespace. The storage-pool labels allow storagePool-restricted scheduling. ```APIDOC ## GetNodeTopologies ### Description Builds the `csi.Topology` for a node: always `linbit.com/hostname: `; when `labelBySP` is set, one `linbit.com/sp-: "true"` key per storage pool; plus every node property under the configured `PropertyNamespace`. The storage-pool labels allow `storagePool`-restricted scheduling, see [topology.md](topology.md). ### Method Function ### Signature ```go func (s *Linstor) GetNodeTopologies(ctx context.Context, nodename string) (*csi.Topology, error) ``` ### Parameters - **ctx** (context.Context) - Required - The context for the operation. - **nodename** (string) - Required - The name of the node. ### Return Values - **csi.Topology** - Pointer to the topology object. - **error** - Error if any. ### Request Example ```go topo, err := c.GetNodeTopologies(ctx, "worker-3") // topo.Segments["linbit.com/hostname"] == "worker-3" // topo.Segments["linbit.com/sp-ssd"] == "true" ``` ``` -------------------------------- ### Attach Source: https://github.com/piraeusdatastore/linstor-csi Idempotently creates a resource on the given node and returns the device path of the volume on that node. ```APIDOC ## Attach ### Description Idempotently creates a resource on the given node and returns the device path of the volume on that node. ### Method Attach ### Endpoint `func (s *Linstor) Attach(ctx context.Context, id volume.ID, node string, rwxBlock bool) (string, error)` ### Parameters - `id` (volume.ID) - Required - Volume to attach - `node` (string) - Required - Target node - `rwxBlock` (bool) - Required - Enables `AutoManageDualPrimary` on make-available (RWX block volumes) ### Response Returns the device path of the volume on the node (`Resources.GetVolume` → `DevicePath`). ### Request Example ```go devPath, err := c.Attach(ctx, vol.ID, "worker-3", false) if err != nil { // e.g. node offline, no diskless storage pool } ``` ``` -------------------------------- ### NewLinstor usage example Source: https://github.com/piraeusdatastore/linstor-csi Constructs a Linstor client with custom API client, property namespace, labelBySP disabled, and debug log level. Returns an error if construction fails. ```go c, err := client.NewLinstor( client.APIClient(hlc), client.PropertyNamespace("Aux"), client.LabelBySP(false), client.LogLevel("debug"), ) if err != nil { return err } ``` -------------------------------- ### Package-level defaults constants and variables Source: https://github.com/piraeusdatastore/linstor-csi Defines the default diskless storage pool name and the default remote access policy used when none is specified. ```go const DefaultDisklessStoragePoolName = "DfltDisklessStorPool" // DefaultRemoteAccessPolicy is the access policy used by default when none is specified. var DefaultRemoteAccessPolicy = RemoteAccessPolicyAnywhere ``` -------------------------------- ### PVC label for consistency group membership Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/examples/consistency-groups/README.md Add this label to a PVC to make it part of a consistency group. The group key is (namespace, value); all PVCs with the same label value in the same namespace become volumes of one shared LINSTOR resource. ```yaml metadata: labels: linstor.csi.linbit.com/consistency-group: fedora-vm-1 ``` -------------------------------- ### GetPluginInfo: Return plugin name and version Source: https://github.com/piraeusdatastore/linstor-csi Returns the plugin name 'linstor.csi.linbit.com' and the vendor version. ```go func (d *Driver) GetPluginInfo(ctx context.Context, req *csi.GetPluginInfoRequest) (*csi.GetPluginInfoResponse, error) ``` -------------------------------- ### Probe: Return empty ready response Source: https://github.com/piraeusdatastore/linstor-csi Always returns an empty (ready) response. ```go func (d *Driver) Probe(ctx context.Context, req *csi.ProbeRequest) (*csi.ProbeResponse, error) ``` -------------------------------- ### ToResourceCreateList Source: https://github.com/piraeusdatastore/linstor-csi Prepares a list of lapi.ResourceCreate to be used to manually assign resources based on the node and client lists of the volume. Produces one ResourceCreate per NodeList entry (diskful) and per ClientList entry (diskless). ```APIDOC ## ToResourceCreateList ### Description Prepares a list of lapi.ResourceCreate to be used to manually assign resources based on the node and client lists of the volume. Produces one ResourceCreate per NodeList entry (diskful) and per ClientList entry (diskless). ### Method Function ### Signature ```go func (params *Parameters) ToResourceCreateList(id string) ([]lapi.ResourceCreate, error) ``` ### Parameters - **id** (string) - Required - Identifier for the resources. ``` -------------------------------- ### NewRWXBlockValidator(ctx, client, resyncAfter, log) Source: https://github.com/piraeusdatastore/linstor-csi Builds a PersistentVolume informer indexed by CSI volume handle, starts it, and blocks until its cache has synced. The informer keeps the index current for the lifetime of ctx. ```APIDOC ## NewRWXBlockValidator(ctx, client, resyncAfter, log) ### Description Builds a PersistentVolume informer indexed by CSI volume handle, starts it, and blocks until its cache has synced. The informer keeps the index current for the lifetime of ctx. ### Method Function ### Signature ```go func NewRWXBlockValidator(ctx context.Context, client kubernetes.Interface, resyncAfter time.Duration, log *logrus.Entry) (*RWXBlockValidator, error) ``` ### Parameters - `ctx` (context.Context) - Required - Lifetime of the informer - `client` (kubernetes.Interface) - Required - Cluster client - `resyncAfter` (time.Duration) - Required - Informer resync period - `log` (*logrus.Entry) - Required - Logger ### Returns - `*RWXBlockValidator` - The validator instance - `error` - Error if index registration or cache sync fails ### Example ```go validator, err := utils.NewRWXBlockValidator(ctx, kube, 5*time.Minute, logger) if err != nil { return err } ``` ``` -------------------------------- ### driver.NewDriver options Source: https://github.com/piraeusdatastore/linstor-csi Constructor options for the CSI driver. These functional options configure the driver's node ID, topology prefix, logging, snapshot client, consistency groups, and RWX support. ```APIDOC ## driver.NewDriver options ### Description Constructor options for the CSI driver. These functional options configure the driver's node ID, topology prefix, logging, snapshot client, consistency groups, and RWX support. ### Options - **NodeID** - `func NodeID(nodeID string) func(*Driver) error` - Default: `"localhost"` - **TopologyPrefix** - `func TopologyPrefix(prefix string) func(*Driver) error` - Default: `"Aux"` - **LogOut** - `func LogOut(out io.Writer) func(*Driver) error` - Default: discarded - **LogLevel** - `func LogLevel(s string) func(*Driver) error` - Default: — - **SnapshotClient** - `func SnapshotClient(client snapclientset.Interface) func(*Driver) error` - Default: disabled - **ConfigureConsistencyGroups** - `func ConfigureConsistencyGroups(client kubernetes.Interface) func(*Driver) error` - Default: disabled - **ConfigureRWX** - `func ConfigureRWX(cl kubernetes.Interface, namespace, reactorConfigMap string) func(*Driver) error` - Default: disabled - **EnableRWXBlockValidation** - `func EnableRWXBlockValidation(validator *utils.RWXBlockValidator) func(*Driver) error` - Default: disabled ``` -------------------------------- ### SnapDelete usage with retryable error check Source: https://github.com/piraeusdatastore/linstor-csi Example call to SnapDelete that checks for the retryable error message indicating the backup is still the base of a newer incremental backup. If that error occurs, deletion must be deferred until dependents are deleted first. ```go err := c.SnapDelete(ctx, &snap.SnapshotId) if strings.Contains(err.Error(), "still the base of a newer incremental backup") { // defer: dependents must be deleted first } ``` -------------------------------- ### NewParameters function signature Source: https://github.com/piraeusdatastore/linstor-csi Parses raw storage-class parameters into a Parameters struct. The topologyPrefix is prepended to topology keys. Returns an error for invalid QoS limits, unknown keys, unparseable values, invalid layer lists, invalid placement policies, unparseable xReplicasOnDifferent YAML, or negative/unparseable over-provisioning. ```go // NewParameters parses out the raw parameters we get and sets appropriate // zero values func NewParameters(params map[string]string, topologyPrefix string) (Parameters, error) ``` -------------------------------- ### NewVolumeContext function signature Source: https://github.com/piraeusdatastore/linstor-csi Returns a new default volume context with no mount options, no post-mount xfs options, and zero remote access policy. ```go // NewVolumeContext creates a new default volume context, which does not specify any fancy mkfs/mount/post-mount options func NewVolumeContext() *VolumeContext ``` -------------------------------- ### NewRWXBlockValidator function signature Source: https://github.com/piraeusdatastore/linstor-csi Builds a PersistentVolume informer indexed by CSI volume handle, starts it, and blocks until its cache has synced. The informer keeps the index current for the lifetime of ctx. Errors when the index cannot be registered or the cache does not sync in time. ```go // NewRWXBlockValidator builds a PersistentVolume informer indexed by CSI volume handle, starts it, and // blocks until its cache has synced. The informer keeps the index current for the lifetime of ctx. func NewRWXBlockValidator(ctx context.Context, client kubernetes.Interface, resyncAfter time.Duration, log *logrus.Entry) (*RWXBlockValidator, error) ``` -------------------------------- ### GetSnapshotRemoteAndReadiness Source: https://github.com/piraeusdatastore/linstor-csi Returns the snapshot's primary remote and readiness. Prefers the `BackupShipping/*` properties (LINSTOR ≥ 1.33), then legacy `Backup`/`Shipped` flags, and finally "local only" ready when the `Successful` flag is set. Malformed backup start timestamps produce an error. ```APIDOC ## GetSnapshotRemoteAndReadiness ### Description Returns `(remote, ready, error)` for a LINSTOR snapshot: prefers the `BackupShipping/*` properties (LINSTOR ≥ 1.33, target receives and source uploads, choosing the earliest start time), then the legacy `Backup`/`Shipped` flags, and finally "local only" ready when the `Successful` flag is set. Malformed backup start timestamps produce an error. ### Method Function ### Signature ```go func GetSnapshotRemoteAndReadiness(snap *lapi.Snapshot) (string, bool, error) ``` ### Parameters - **snap** (*lapi.Snapshot) - Required - The LINSTOR snapshot to inspect. ### Response - **string** - The primary remote name. - **bool** - Whether the snapshot is ready. - **error** - An error if the operation fails. ``` -------------------------------- ### MarshalText method Source: https://github.com/piraeusdatastore/linstor-csi Inverse of UnmarshalText: LocalOnly becomes "false", Anywhere becomes "true", otherwise serializes as YAML. ```go func (r RemoteAccessPolicy) MarshalText() (text []byte, err error) ``` -------------------------------- ### Deploy LINSTOR CSI components with kubectl kustomize Source: https://github.com/piraeusdatastore/linstor-csi/blob/master/README.md Deploys the LINSTOR CSI components using kubectl kustomize, replacing the LINSTOR_CONTROLLER_URL placeholder with the actual controller URL. Requires a working LINSTOR cluster and updates every occurrence of LINSTOR_CONTROLLER_URL. ```bash $ LINSTOR_CONTROLLER_URL=http://linstor-controller.example.com:3370 $ kubectl kustomize http://github.com/piraeusdatastore/linstor-csi/examples/k8s/deploy \ | sed "s#LINSTOR_CONTROLLER_URL#$LINSTOR_CONTROLLER_URL#" \ | kubectl apply --server-side -f - ``` -------------------------------- ### ControllerGetCapabilities Source: https://github.com/piraeusdatastore/linstor-csi Returns the capabilities supported by the controller. Advertises a specific set of volume and snapshot operations. ```APIDOC ## ControllerGetCapabilities ### Description Advertises: `CREATE_DELETE_VOLUME`, `PUBLISH_UNPUBLISH_VOLUME`, `LIST_VOLUMES`, `CREATE_DELETE_SNAPSHOT`, `CLONE_VOLUME`, `LIST_SNAPSHOTS`, `GET_CAPACITY`, `EXPAND_VOLUME`, `GET_VOLUME`, `SINGLE_NODE_MULTI_WRITER`, `GET_SNAPSHOT`, `GET_VOLUME_HEALTH`, `LIST_VOLUME_HEALTH`. Explicitly no `PUBLISH_READONLY`. ### Method ControllerGetCapabilities ### Signature ```go func (d *Driver) ControllerGetCapabilities(ctx context.Context, req *csi.ControllerGetCapabilitiesRequest) (*csi.ControllerGetCapabilitiesResponse, error) ``` ### Parameters - **ctx** (context.Context) - Required - The context for the request - **req** (*csi.ControllerGetCapabilitiesRequest) - Required - The capabilities request ### Response Returns a `*csi.ControllerGetCapabilitiesResponse` containing the list of supported capabilities. ``` -------------------------------- ### NodeAvailable function signature Source: https://github.com/piraeusdatastore/linstor-csi Errors unless the node's ConnectionStatus is "ONLINE"; error text is the status, e.g. 'node OFFLINE'. ```go func (s *Linstor) NodeAvailable(ctx context.Context, node string) error ``` -------------------------------- ### Parameters struct definition Source: https://github.com/piraeusdatastore/linstor-csi Defines the parsed form of the CSI CreateVolumeRequest.parameters storage-class map. Field semantics match the storage-class parameter reference in configuration.md. The IO QoS limits are held by meta.Limits from github.com/piraeusdatastore/nri-volume-qos/pkg/meta. ```go // Parameters configuration for linstor volumes. type Parameters struct { ClientList []string NodeList []string ReplicasOnDifferent []string ReplicasOnSame []string XReplicasOnDifferent map[string]int DisklessStoragePool string DoNotPlaceWithRegex string FSOpts string MountOpts string StoragePools []string PlacementCount int32 Disklessonremaining bool Encryption bool AllowRemoteVolumeAccess RemoteAccessPolicy LayerList []devicelayerkind.DeviceLayerKind PlacementPolicy topology.PlacementPolicy PostMountXfsOpts string ResourceGroup string Properties map[string]string UsePvcName bool OverProvision *float64 NfsConfigTemplatePath string NfsServiceName string NfsSquash string NfsRecoveryVolumeBytes int64 IOQoSLimits meta.Limits } ``` -------------------------------- ### PublishContextFromMap function signature Source: https://github.com/piraeusdatastore/linstor-csi Parses a map into a PublishContext. Returns nil when the marker is absent. QoS limits are decoded ignoring errors. ```go func PublishContextFromMap(ctx map[string]string) *PublishContext ``` -------------------------------- ### Mount Source: https://github.com/piraeusdatastore/linstor-csi Makes volumes consumable from the source to the target. Filesystems are formatted and block devices are bind mounted. Operates locally on the machine where it is called. ```APIDOC ## Mount ### Description Makes volumes consumable from the source to the target. Filesystems are formatted and block devices are bind mounted. Operates locally on the machine where it is called. ### Method Function ### Signature ```go func (s *Linstor) Mount(ctx context.Context, source, target, fsType string, readonly bool, mntOpts []string) error ``` ### Parameters - `ctx` (context.Context) - Required - Context for the operation. - `source` (string) - Required - Device path or NFS URL. - `target` (string) - Required - Mount point / bind target file. - `fsType` (string) - Required - Filesystem (`""` = block volume bind mount, `"nfs"` = NFS mount). - `readonly` (bool) - Required - Read-only mount (adds `ro,norecovery`, sets device ro). - `mntOpts` ([]string) - Required - Extra mount options. ### Response Returns an error if the mount operation fails. ``` -------------------------------- ### Probe Source: https://github.com/piraeusdatastore/linstor-csi Returns an empty (ready) response to indicate the plugin is ready. ```APIDOC ## Probe ### Description Returns an empty (ready) response to indicate the plugin is ready. ### Method Probe ### Endpoint func (d *Driver) Probe(ctx context.Context, req *csi.ProbeRequest) (*csi.ProbeResponse, error) ### Parameters #### Parameters - **ctx** (context.Context) - Required - Context for the request - **req** (*csi.ProbeRequest) - Required - CSI Probe request ### Response #### Success Response - **csi.ProbeResponse** - Always returns an empty (ready) response. ``` -------------------------------- ### GroupControllerGetCapabilities method signature Source: https://github.com/piraeusdatastore/linstor-csi Advertises the single capability RPC_CREATE_DELETE_GET_VOLUME_GROUP_SNAPSHOT. ```go func (d *Driver) GroupControllerGetCapabilities(ctx context.Context, in *csi.GroupControllerGetCapabilitiesRequest) (*csi.GroupControllerGetCapabilitiesResponse, error) ``` -------------------------------- ### ToMap method signature for PublishContext Source: https://github.com/piraeusdatastore/linstor-csi Serializes the PublishContext to a map, always emitting marker, device path, and fs type. Adds QoS keys when device is set. ```go func (p *PublishContext) ToMap() map[string]string ``` -------------------------------- ### CapacityBytes method signature Source: https://github.com/piraeusdatastore/linstor-csi Defines the CapacityBytes method that returns free space in storage pools, excluding nodes with missing topology keys. Parameters include storagePools, overProvision ratio, segments, and missingKeys. ```go // CapacityBytes returns the amount of free space in the storage pool specified by the params and topology. // Nodes that have any of the missingKeys topology keys are not counted. func (s *Linstor) CapacityBytes(ctx context.Context, storagePools []string, overProvision *float64, segments map[string]string, missingKeys ...string) (int64, error) ``` -------------------------------- ### VolumeContextFromMap function signature Source: https://github.com/piraeusdatastore/linstor-csi Parses a map into a VolumeContext. Returns nil, nil when the marker key is absent. Errors on unparseable remote access policy. ```go func VolumeContextFromMap(ctx map[string]string) (*VolumeContext, error) ``` -------------------------------- ### AllocationSize: Calculate allocation size Source: https://github.com/piraeusdatastore/linstor-csi Rounds up to KiB alignment and enforces minimum size and limit. Raises minimum to 2 MiB for ext4, 300 MiB for xfs, otherwise 4096 bytes. Errors when minimum exceeds limit or aligned size violates limit. ```go func (s *Linstor) AllocationSize(requiredBytes, limitBytes int64, fsType string) (int64, error) ``` ```go sz, err := c.AllocationSize(1024, 0, "ext4") // -> 2 MiB (ext4 minimum) sz, err = c.AllocationSize(1<<30, 512<<20, "xfs") // err: LINSTOR's minimum volume size (314572800) exceeds the maximum size limit ... ```