Repository navigation
Commit 59f0013
committed
fix(backups): attribute a MariaDB dump only when no other run can own it
Every backup of a MariaDB application writes its dump under one shared
prefix, so the probe took the dump whose file-name timestamp (the
backup pod's clock) and LastModified (the object store's clock) both
fall in the operator Backup's lifetime, with a minute of slack for
skew. That keeps a neighbour's dump out only while this run's own dump
is in the window. With the pod clock more than the slack behind, the
own dump is named before the window opens, and a previous run that
finished within the minute before this one was created becomes the
single match: a 123-byte dump was reported as 666 bytes. A pod clock
ahead or an object store clock behind does the same with the next run,
and an own dump that is gone by the time it is looked up, as it can be
for a restore long after the backup, leaves a neighbour as the single
match with no skew at all.
A dump is now attributed only when ownership is proven:
- it is the one dump that either clock places in the window. A dump
that one clock places there and the other does not may be this run's
own with a skewed clock, so it leaves any other match unproven, the
way a second dump in the window already did, and is not taken itself;
- both of its clocks place it in the window;
- neither clock places it in the window of another operator Backup
that writes under the same bucket and prefix and has completed. This
rejects a neighbour's dump when this run's own is missing, which the
clocks alone cannot tell.
Operator Backups still running are not counted as neighbours: their
window has no end, and one stuck short of Complete (a Job whose pod
never starts stays JobRunning for good) would otherwise keep every
later run of the application from being sized. A dump such a run
writes inside this run's window still competes with this run's own.
Narrower slack does not prove ownership: a neighbour that finished
within the narrower slack still matches, and a smaller skew pushes the
own dump out. The Job's start and completion times are stamped by the
control plane, like the operator Backup's own, so they narrow the
window by seconds without removing the dependence on the two clocks a
dump carries. What is left: a neighbour's dump is taken only when this
run's own is missing from the listing or misplaced on both clocks and
the neighbour's operator Backup no longer exists or has not completed.
Every other unproven case leaves the size unset, as two runs that
overlap or follow each other within the slack already did.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <[email protected]>1 parent 560c44f commit 59f0013
3 files changed
Lines changed: 250 additions & 23 deletions
File tree
- internal/backupcontroller
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
60 | 60 | | |
61 | 61 | | |
62 | 62 | | |
63 | | - | |
| 63 | + | |
| 64 | + | |
64 | 65 | | |
65 | 66 | | |
66 | 67 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
11 | 15 | | |
12 | 16 | | |
13 | 17 | | |
| |||
60 | 64 | | |
61 | 65 | | |
62 | 66 | | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
63 | 71 | | |
64 | 72 | | |
65 | 73 | | |
| |||
76 | 84 | | |
77 | 85 | | |
78 | 86 | | |
79 | | - | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
80 | 108 | | |
81 | 109 | | |
82 | | - | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | | - | |
89 | | - | |
90 | | - | |
91 | | - | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
92 | 127 | | |
93 | 128 | | |
94 | 129 | | |
95 | | - | |
| 130 | + | |
96 | 131 | | |
97 | 132 | | |
98 | | - | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
99 | 137 | | |
100 | | - | |
101 | | - | |
| 138 | + | |
102 | 139 | | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | | - | |
107 | 140 | | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
108 | 185 | | |
109 | 186 | | |
110 | 187 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
239 | 239 | | |
240 | 240 | | |
241 | 241 | | |
| 242 | + | |
242 | 243 | | |
243 | | - | |
| 244 | + | |
244 | 245 | | |
245 | 246 | | |
246 | 247 | | |
| |||
255 | 256 | | |
256 | 257 | | |
257 | 258 | | |
258 | | - | |
| 259 | + | |
259 | 260 | | |
260 | 261 | | |
261 | 262 | | |
262 | | - | |
| 263 | + | |
263 | 264 | | |
264 | 265 | | |
265 | 266 | | |
| |||
303 | 304 | | |
304 | 305 | | |
305 | 306 | | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
306 | 455 | | |
307 | 456 | | |
308 | 457 | | |
| |||
0 commit comments