Repository navigation
Commit 526f08e
authored
fix(mariadb): restore the dump a Backup was taken as (cozystack#5227)
## What this PR does
A RestoreJob that names a MariaDB `Backup` restored the newest dump of
the application, not the dump of that `Backup`. The driver created the
operator `Restore` with only `backupRef`, and mariadb-operator then
restores the dump nearest to the moment the restore starts, from the
prefix every run of the app shares. Restoring an older `Backup` brought
back the latest data, and when its own dump was already gone it silently
restored another one.
The `Backup` now records the dump its operator `Backup` wrote, the file
name in `k8s.mariadb.com/dump-file` and the timestamp from that name in
`k8s.mariadb.com/dump-time`. They come from the ownership-proven match
the size probe already makes, and nothing is recorded when ownership is
not proven. A restore first checks with a bounded S3 listing of the
operator `Backup`'s own storage that the dump still exists, then sets
`targetRecoveryTime` on the operator `Restore` to the dump's own
timestamp, which mariadb-operator matches exactly. If the dump is gone
the RestoreJob fails and names it. If the check cannot be made it waits
with `Ready=False/DumpCheckPending` until the restore deadline and then
fails with the cause.
`Backup`s taken before this stay restorable. The restore makes the same
ownership-proven match against their operator `Backup` and pins the dump
when it is provable, otherwise it restores the newest dump as before and
a `BestEffortDump` warning event on the RestoreJob says so. One case is
not covered: the `Backup`'s own dump is gone, a neighbour's dump from
within the minute of slack is still there and its operator `Backup` was
deleted, so that dump is taken as the match. Nothing in the platform
deletes operator `Backup`s, it needs someone to delete one by hand. A
`Backup` made of an operator `Backup` whose Job failed (`Complete` with
reason `JobFailed`) has no dump, so its RestoreJob now fails instead of
restoring another run's dump.
Stacked on cozystack#5223 and targets
its branch until that merges, only the last three commits belong here.
Checked on a test cluster with mariadb-operator 25.10.2. An operator
`Restore` of an older `Backup` with `targetRecoveryTime` set to its dump
timestamp restored exactly that dump (`obtained target backup` names it
and a row inserted after it is absent from the target), the same
`Restore` without the target restored the newest one. Against the real
bucket and operator `Backup`s the recording probe attributed to each
`Backup` the dump its backup pod logged, the existence check passed for
those dumps and refused a pruned one, legacy `Backup`s with a provable
dump got pinned while back-to-back runs fell back with the warning, and
the two failed runs on that cluster were refused. The controller itself
was not run on the cluster, only its code driven by a test against the
real bucket.
Fixes cozystack#5196
### Screenshots
Not a UI change.
### Downstream repositories
The diff is confined to the backup controller's Go code. It changes no
CRD or API type (`driverMetadata` is an opaque map), no chart, values,
package layout or `hack/` script, so no entry of the trigger map
applies.
- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:
### Release note
```release-note
fix(mariadb): a RestoreJob now restores the dump of the Backup it names instead of the newest dump of the application. It fails with a clear message when that dump no longer exists in storage, or when the Backup was made of a failed backup run. Backups taken before this release are restored by the one dump that can be matched to them, or else, as before, by the newest dump, with a BestEffortDump warning event on the RestoreJob.
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## Summary by CodeRabbit
* **New Features**
* MariaDB backups now record the selected dump’s filename and timestamp,
allowing restores to target the corresponding recovery time.
* For older backups without dump metadata, restores use a uniquely
matched dump when possible or fall back to the latest dump with a
warning.
* Restore processing verifies recorded dumps, retries temporary storage
checks, and reports failures for invalid or missing recorded dumps.
* **Bug Fixes**
* Deleting a MariaDB backup now removes its associated dump and operator
backup when ownership is verified and no restore is active. Cleanup
avoids deleting artifacts when it cannot safely identify them.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->13 files changed
Lines changed: 2717 additions & 83 deletions
File tree
- internal/backupcontroller
- packages/system/backupstrategy-controller
- templates
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
42 | 49 | | |
43 | 50 | | |
44 | 51 | | |
| |||
67 | 74 | | |
68 | 75 | | |
69 | 76 | | |
70 | | - | |
71 | | - | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
72 | 81 | | |
73 | 82 | | |
74 | 83 | | |
| |||
78 | 87 | | |
79 | 88 | | |
80 | 89 | | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
81 | 101 | | |
82 | 102 | | |
83 | 103 | | |
| |||
94 | 114 | | |
95 | 115 | | |
96 | 116 | | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
97 | 125 | | |
98 | 126 | | |
99 | 127 | | |
| |||
103 | 131 | | |
104 | 132 | | |
105 | 133 | | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
106 | 142 | | |
107 | 143 | | |
108 | 144 | | |
| |||
168 | 204 | | |
169 | 205 | | |
170 | 206 | | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
171 | 211 | | |
172 | 212 | | |
173 | 213 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
42 | | - | |
43 | | - | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
44 | 45 | | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
45 | 58 | | |
46 | 59 | | |
47 | 60 | | |
| |||
126 | 139 | | |
127 | 140 | | |
128 | 141 | | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
138 | | - | |
139 | | - | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
140 | 151 | | |
141 | 152 | | |
142 | 153 | | |
143 | 154 | | |
144 | | - | |
| 155 | + | |
145 | 156 | | |
146 | 157 | | |
147 | 158 | | |
| |||
154 | 165 | | |
155 | 166 | | |
156 | 167 | | |
157 | | - | |
| 168 | + | |
158 | 169 | | |
159 | 170 | | |
160 | 171 | | |
| |||
294 | 305 | | |
295 | 306 | | |
296 | 307 | | |
| 308 | + | |
297 | 309 | | |
298 | 310 | | |
299 | 311 | | |
300 | 312 | | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
176 | 176 | | |
177 | 177 | | |
178 | 178 | | |
179 | | - | |
180 | | - | |
181 | | - | |
182 | | - | |
183 | | - | |
| 179 | + | |
| 180 | + | |
184 | 181 | | |
185 | 182 | | |
186 | 183 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
51 | | - | |
52 | | - | |
| 51 | + | |
| 52 | + | |
53 | 53 | | |
54 | | - | |
| 54 | + | |
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
| |||
60 | 60 | | |
61 | 61 | | |
62 | 62 | | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
63 | 75 | | |
64 | 76 | | |
65 | 77 | | |
66 | 78 | | |
67 | | - | |
| 79 | + | |
68 | 80 | | |
69 | 81 | | |
70 | 82 | | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
75 | | - | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | 83 | | |
80 | 84 | | |
81 | 85 | | |
| |||
88 | 92 | | |
89 | 93 | | |
90 | 94 | | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
91 | 133 | | |
92 | 134 | | |
93 | 135 | | |
94 | 136 | | |
95 | 137 | | |
96 | 138 | | |
97 | 139 | | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
98 | 144 | | |
99 | 145 | | |
100 | 146 | | |
| |||
120 | 166 | | |
121 | 167 | | |
122 | 168 | | |
123 | | - | |
124 | | - | |
| 169 | + | |
| 170 | + | |
125 | 171 | | |
126 | 172 | | |
127 | 173 | | |
| |||
153 | 199 | | |
154 | 200 | | |
155 | 201 | | |
156 | | - | |
| 202 | + | |
157 | 203 | | |
158 | | - | |
159 | | - | |
160 | | - | |
161 | | - | |
162 | | - | |
163 | | - | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
164 | 213 | | |
165 | 214 | | |
166 | 215 | | |
| |||
177 | 226 | | |
178 | 227 | | |
179 | 228 | | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
180 | 232 | | |
181 | 233 | | |
182 | 234 | | |
| |||
0 commit comments