Repository navigation
Commit 6cb029c
[fix](ivm) Keep the keep-alive TSOs of a stream in a concurrent map
### What problem does this PR solve?
Issue Number: None
Related PR: #69831
Problem Summary:
`OlapTableStream.partitionKeepAlive` was a `HashMap` that the keep-alive daemon writes under the stream write lock. The delta scan of an IVM refresh reads it without the stream lock: `IvmDeltaRewriteState` looks the stream up and creates its `OlapTableStreamWrapper` after the statement has locked its tables, so the stream is never among them, and the base table read lock the daemon also holds in shared-nothing mode does not exclude the refresh, which holds it too. A put that adds a partition and resizes the map can make a concurrent get miss the keep-alive TSO of another partition. The wrapper keeps the miss for the whole refresh, and when that partition's offset is older than the TTL the read is refused: an AUTO refresh falls back to COMPLETE and a strict INCREMENTAL refresh fails, although a fresh keep-alive TSO was recorded. No row is read wrong.
The field is now declared as a `ConcurrentMap`, so a get never misses an entry while another one is added. Declared as such, GSON reads it back from an image as a concurrent map too, as it does `Database.name2Function`.
### Release note
None
### Check List (For Author)
- Test: Unit Test
- `TableStreamManagerKeepAliveTest`: a keep-alive TSO written to an image reads back, into a concurrent map; the cases that set the field directly set a concurrent map. The race itself has no deterministic test.
- Behavior changed: No
- Does this need documentation: No
Co-Authored-By: Claude Opus 5.5 <[email protected]>1 parent 754db89 commit 6cb029c
2 files changed
Lines changed: 33 additions & 8 deletions
File tree
- fe/fe-core/src
- main/java/org/apache/doris/catalog/stream
- test/java/org/apache/doris/catalog/stream
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| 35 | + | |
35 | 36 | | |
36 | 37 | | |
37 | 38 | | |
| |||
43 | 44 | | |
44 | 45 | | |
45 | 46 | | |
| 47 | + | |
46 | 48 | | |
47 | 49 | | |
48 | 50 | | |
| |||
59 | 61 | | |
60 | 62 | | |
61 | 63 | | |
62 | | - | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
63 | 67 | | |
64 | | - | |
| 68 | + | |
65 | 69 | | |
66 | 70 | | |
67 | 71 | | |
68 | 72 | | |
69 | | - | |
| 73 | + | |
70 | 74 | | |
71 | 75 | | |
72 | 76 | | |
| |||
75 | 79 | | |
76 | 80 | | |
77 | 81 | | |
78 | | - | |
| 82 | + | |
79 | 83 | | |
80 | 84 | | |
81 | 85 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
| 56 | + | |
| 57 | + | |
56 | 58 | | |
57 | 59 | | |
58 | 60 | | |
| |||
89 | 91 | | |
90 | 92 | | |
91 | 93 | | |
92 | | - | |
| 94 | + | |
93 | 95 | | |
94 | 96 | | |
95 | 97 | | |
| |||
140 | 142 | | |
141 | 143 | | |
142 | 144 | | |
143 | | - | |
144 | | - | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
145 | 148 | | |
146 | 149 | | |
147 | 150 | | |
| |||
208 | 211 | | |
209 | 212 | | |
210 | 213 | | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
211 | 232 | | |
212 | 233 | | |
213 | 234 | | |
| |||
287 | 308 | | |
288 | 309 | | |
289 | 310 | | |
290 | | - | |
| 311 | + | |
291 | 312 | | |
292 | 313 | | |
293 | 314 | | |
| |||
0 commit comments