Repository navigation
Commit 3dbe4d3
authored
[fix](cloud) Invalidate version caches on visible commit retries (#67813)
### What problem does this PR solve?
Issue Number: close #67099
Related PR: None
Problem Summary: After a cloud commit response is lost, a retry can
return a VISIBLE transaction without version results. FE can then keep
using stale partition versions and the table version used to validate
SQL caches.
Invalidate the affected table and partition version caches, including
the MoW already-visible shortcut, and notify other FEs through the
existing version synchronization RPC. Subsequent reads fetch fresh
versions from MS. Commit processing performs no extra MS version query,
so a version-service outage does not turn an already committed
transaction into a commit error or require replaying its callbacks.
Publish batch cache updates under the tables' version write locks.
Invalidation epochs prevent old in-flight MS responses and delayed
commit notifications from restoring stale caches. The synchronization
RPC uses table version -1 for invalidation; older FEs ignore it and
retain their existing periodic synchronization. Peer notification keeps
its existing asynchronous behavior and configuration.
### Release note
Prevent stale reads and false commit failures after cloud commit
retries.
### Check List (For Author)
- Test: Unit Test
- 58 FE tests passed in both ordinary and JaCoCo coverage modes via
run-fe-ut.sh (CloudGlobalTransactionMgrTest, CloudPartitionTest,
OlapTableTest, VersionHelperTest); FE Checkstyle: 0 violations; PR diff
whitespace check passed.
- Tests cover invalidation and batch-publication locks, table/partition
cache recovery, old RPC completions, successful commit callbacks during
a version outage, and FE notification serialization/receiver handling.
- No live cluster deployment or SQL regression run.
- Behavior changed: Yes. VISIBLE retries without versions invalidate
local and peer caches; subsequent reads retrieve versions without
changing the successful commit result.
- Does this need documentation: No1 parent 7b9693d commit 3dbe4d3
5 files changed
Lines changed: 520 additions & 14 deletions
File tree
- fe/fe-core/src
- main/java/org/apache/doris
- catalog
- cloud
- catalog
- transaction
- test/java/org/apache/doris/cloud/transaction
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
248 | 248 | | |
249 | 249 | | |
250 | 250 | | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
251 | 255 | | |
252 | 256 | | |
253 | 257 | | |
| |||
3591 | 3595 | | |
3592 | 3596 | | |
3593 | 3597 | | |
3594 | | - | |
| 3598 | + | |
3595 | 3599 | | |
3596 | 3600 | | |
3597 | 3601 | | |
| |||
3605 | 3609 | | |
3606 | 3610 | | |
3607 | 3611 | | |
3608 | | - | |
| 3612 | + | |
| 3613 | + | |
3609 | 3614 | | |
3610 | 3615 | | |
3611 | 3616 | | |
3612 | 3617 | | |
3613 | 3618 | | |
3614 | | - | |
| 3619 | + | |
| 3620 | + | |
| 3621 | + | |
| 3622 | + | |
| 3623 | + | |
3615 | 3624 | | |
3616 | 3625 | | |
3617 | 3626 | | |
| |||
3632 | 3641 | | |
3633 | 3642 | | |
3634 | 3643 | | |
| 3644 | + | |
3635 | 3645 | | |
3636 | 3646 | | |
3637 | 3647 | | |
| |||
3658 | 3668 | | |
3659 | 3669 | | |
3660 | 3670 | | |
| 3671 | + | |
3661 | 3672 | | |
3662 | 3673 | | |
3663 | 3674 | | |
| |||
3722 | 3733 | | |
3723 | 3734 | | |
3724 | 3735 | | |
| 3736 | + | |
3725 | 3737 | | |
3726 | 3738 | | |
3727 | 3739 | | |
| 3740 | + | |
3728 | 3741 | | |
3729 | 3742 | | |
3730 | 3743 | | |
| |||
3733 | 3746 | | |
3734 | 3747 | | |
3735 | 3748 | | |
| 3749 | + | |
3736 | 3750 | | |
3737 | 3751 | | |
3738 | 3752 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
52 | 55 | | |
53 | 56 | | |
54 | 57 | | |
| |||
57 | 60 | | |
58 | 61 | | |
59 | 62 | | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
60 | 111 | | |
61 | 112 | | |
62 | 113 | | |
| |||
154 | 205 | | |
155 | 206 | | |
156 | 207 | | |
| 208 | + | |
157 | 209 | | |
158 | 210 | | |
159 | 211 | | |
160 | 212 | | |
161 | 213 | | |
162 | 214 | | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
163 | 219 | | |
164 | 220 | | |
165 | 221 | | |
166 | 222 | | |
167 | 223 | | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
168 | 227 | | |
169 | 228 | | |
170 | 229 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
| 51 | + | |
51 | 52 | | |
52 | 53 | | |
53 | 54 | | |
| |||
66 | 67 | | |
67 | 68 | | |
68 | 69 | | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
69 | 74 | | |
70 | 75 | | |
71 | 76 | | |
| |||
127 | 132 | | |
128 | 133 | | |
129 | 134 | | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
130 | 139 | | |
131 | 140 | | |
132 | | - | |
| 141 | + | |
133 | 142 | | |
134 | 143 | | |
135 | 144 | | |
| |||
155 | 164 | | |
156 | 165 | | |
157 | 166 | | |
| 167 | + | |
158 | 168 | | |
159 | 169 | | |
160 | 170 | | |
| |||
183 | 193 | | |
184 | 194 | | |
185 | 195 | | |
| 196 | + | |
186 | 197 | | |
187 | 198 | | |
188 | 199 | | |
| |||
193 | 204 | | |
194 | 205 | | |
195 | 206 | | |
| 207 | + | |
196 | 208 | | |
197 | 209 | | |
198 | 210 | | |
| |||
254 | 266 | | |
255 | 267 | | |
256 | 268 | | |
| 269 | + | |
257 | 270 | | |
258 | 271 | | |
259 | 272 | | |
260 | 273 | | |
| 274 | + | |
261 | 275 | | |
262 | 276 | | |
263 | 277 | | |
| |||
266 | 280 | | |
267 | 281 | | |
268 | 282 | | |
269 | | - | |
270 | | - | |
271 | | - | |
272 | | - | |
273 | | - | |
274 | | - | |
275 | | - | |
276 | | - | |
277 | | - | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
278 | 304 | | |
279 | 305 | | |
280 | 306 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
491 | 491 | | |
492 | 492 | | |
493 | 493 | | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
494 | 498 | | |
495 | 499 | | |
496 | 500 | | |
| |||
597 | 601 | | |
598 | 602 | | |
599 | 603 | | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
600 | 611 | | |
601 | 612 | | |
602 | 613 | | |
| |||
0 commit comments