Repository navigation
Commit 9214167
committed
[fix](ivm) Treat a partition_sync_limit window change as a baseline change
A partition_sync_limit window decides which base partitions the MV maintains, so changing it changes
which rows the MV owes: a partition that leaves the window keeps its MV partition, the deltas of the
range it skipped were never applied, and a later widening makes partition sync keep it. The
invalidation path tried to answer that after the fact -- whether the mapping it had just built was
filtered by a window -- by reading the window from mutable properties. A window set and cleared while
that mapping was built is seen by neither read, so the mapping could be windowed while the guard read
no limit, the selection came out empty, and no barrier was written for a change whose rows no row
binlog can repair.
The window now gets the answer the two properties around it already get: an effective change to it
forces a complete baseline rebuild, recorded by the ALTER itself, in the same critical section that
applies it, so no ordering of the ALTER against a partition change can leave a windowed mapping
trusted.
Key changes:
- Compare the three window properties across alterMvProperties, as they are stored, so restating the same window forces nothing, and require a complete baseline rebuild when they change, for IVM MVs
- Keep the property vocabulary where the rest of it lives: MTMVPropertyUtil gains the key list and the two helpers, next to hasPartitionSyncLimit
- Release the barrier the ALTER now records in the three tests that set the window and then test what happens while it is in effect, so their assertions keep holding for the reason they were written for
Unit Test:
- IvmBaselineRebuildTest.testChangingTheSyncWindowRequiresCompleteBaselineRebuild: setting the window requires a complete rebuild, restating the same window does not, widening it does. Without the rule it fails with expected: <true> but was: <false>
- Regression: mtmv_p0/ivm test_ivm_partition_sync_limit and test_ivm_partition_sync_limit_with_window (both set the window at CREATE, so the rule does not reach them) plus the eight marker suites of the previous commit green on a local cluster built from this head; IvmBaselineRebuildTest 37/37, MTMVPropertyUtilTest 13/13, MTMVTest 23/23; checkstyle clean1 parent 57a1b3d commit 9214167
3 files changed
Lines changed: 83 additions & 0 deletions
File tree
- fe/fe-core/src
- main/java/org/apache/doris
- catalog
- mtmv
- test/java/org/apache/doris/mtmv/ivm
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
384 | 384 | | |
385 | 385 | | |
386 | 386 | | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
387 | 396 | | |
388 | 397 | | |
389 | 398 | | |
| |||
428 | 437 | | |
429 | 438 | | |
430 | 439 | | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
431 | 444 | | |
432 | 445 | | |
433 | 446 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
48 | 54 | | |
49 | 55 | | |
50 | 56 | | |
| |||
231 | 237 | | |
232 | 238 | | |
233 | 239 | | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
234 | 263 | | |
235 | 264 | | |
236 | 265 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
216 | 216 | | |
217 | 217 | | |
218 | 218 | | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
219 | 223 | | |
220 | 224 | | |
221 | 225 | | |
| |||
263 | 267 | | |
264 | 268 | | |
265 | 269 | | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
266 | 274 | | |
267 | 275 | | |
268 | 276 | | |
269 | 277 | | |
270 | 278 | | |
271 | 279 | | |
272 | 280 | | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
273 | 310 | | |
274 | 311 | | |
275 | 312 | | |
| |||
290 | 327 | | |
291 | 328 | | |
292 | 329 | | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
293 | 334 | | |
294 | 335 | | |
295 | 336 | | |
| |||
0 commit comments