Repository navigation
Commit a347039
authored
fix(linstor): mark volumes restored from a snapshot DRBD-initialized (cozystack#4866)
## What this PR does
Restoring a PVC from a LINSTOR VolumeSnapshot can leave one replica with
an empty volume that DRBD marks UpToDate. A pod that lands on that
replica sees an empty filesystem.
linstor-csi restores the snapshot on one node and autoplaces the other
replicas. LINSTOR v1.33.3 copies `Linstor/Drbd/InitialUptoDateOn` from
the source volume into the restored volume definition and leaves the
definition without `DRBD_INITIALIZED`. When autoplace puts a replica on
the node that name points to, that empty new volume wins the initial
UpToDate race, gets a random current UUID and a fresh filesystem, and
goes StandAlone against the restored replica.
LINBIT fixed this in v1.34.0. This PR backports that commit,
LINBIT/linstor-server@9d8cdf5, to our v1.33.3 build as a patch. The
README entry says to drop the patch with the move to v1.34.0 or later.
Tested on a three-node cluster with a thick ZFS pool and
`placementCount: 3`. Before the change the reproduction failed every
time. With the patch I ran 5 restores, 4 of them with the source
volume's winner on a different node than the restore. I also restored a
snapshot taken before the change, and a 10Gi volume. Every time all
replicas came up connected and UpToDate, a pod on each node read the
same checksum, and the backing zvols were byte-identical when read with
`O_DIRECT`. The resync after a restore moves about the written data,
around 40 s for 1Gi and 134 s for 7.5 GiB on 10Gi. CSI clone PVC-to-PVC
was checked too and is not affected.
There is no e2e test. Every e2e lane runs without DRBD
(`COZY_LINSTOR_DRBD_ENABLED=false`, StorageClass `local`), so a suite
for replicated restores would never run in CI.
Volumes restored before this fix are not repaired by it. How to find and
fix them is in [this
comment](cozystack#4862 (comment)).
Known limitation, shared by upstream v1.34 since that code did not
change: the autoplaced replicas skip the initial sync and rely on the
source volume's day0 UUID still being in the restored replica's history.
If that ever fails, DRBD goes StandAlone with "unrelated data" instead
of syncing, which is visible, not silent. I could not reproduce it.
Closes cozystack#4862
### Screenshots
Not a UI change.
### Downstream repositories
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
```release-note
fix(linstor): a PVC restored from a VolumeSnapshot no longer gets a replica with an empty volume marked UpToDate. Volumes restored before this release may still have such a replica; cozystack#4862 (comment) explains how to check and repair them.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Restored snapshots with diskful DRBD resources are now recognized as
initialized by default, helping them resume without unnecessary metadata
recreation.
* When force-initial-sync is enabled, restored volumes continue through
the existing metadata-recreation path.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->2 files changed
Lines changed: 73 additions & 0 deletions
File tree
- packages/system/linstor/images/piraeus-server/patches
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
0 commit comments