Repository navigation
Commit 4f3abce
authored
[fix](ivm) Resolve IVM identity keys by slot identity and materialize unprojected keys (#67669)
Two follow-ups from the #62606 review round, both around same-named
identity keys of aggregate/full-keys MVs:
1. **Resolve IVM aggregate group keys by slot identity instead of name**
— an aggregate delta over a join of two tables that both expose the same
column name (`GROUP BY l.id, r.id`) resolved its group keys through
name-keyed lookups. A name index collapses same-named slots onto the
last match, so both keys were bound to one delta slot and a `(10, 20)`
group was written as `(20, 20)`, diverging from COMPLETE. The delta-side
group keys are now resolved and carried by slot identity (ordered with
the aggregate metadata), the apply project emits each key output from
its own delta slot, and the `ivm_use_full_keys` identity conjuncts
resolve their delta side by identity first. The full-keys regression
with a non-full-keys counterpart covers plain INSERT, full-keys, and MOW
UPDATE refresh, each cross-checked against COMPLETE.
2. **Materialize unprojected full-keys identity keys at the project
layer** — with `ivm_use_full_keys` every identity key must be
materialized in the stored layout, either as a visible output or under a
hidden key column. The result sink judged "already projected" by column
name, so a same-named key from another table (selecting only `l.id`
while grouping by `l.id, r.id`) was silently dropped and the key set
lost one dimension. The sink now checks projection by slot identity, and
materialization moves down to the first project that drops a key, so
CREATE (result sink) and refresh (olap-table sink) plans produce the
same hidden-key layout. A new regression asserts the hidden key column
exists (DESC with hidden columns) and that INCREMENTAL matches COMPLETE.
Tests: `IvmAggDeltaHandlerTest` (33) and `IvmNormalizeMTMVJoinTest` new
cases pass; the new regression cases pass and the full ivm regression
set (85 suites) is green on the materialization change.
Trace issue: #654181 parent c1bff0d commit 4f3abce
10 files changed
Lines changed: 750 additions & 33 deletions
File tree
- fe/fe-core/src
- main/java/org/apache/doris
- mtmv/ivm
- nereids/rules/analysis
- test/java/org/apache/doris
- mtmv/ivm
- nereids/rules/analysis
- regression-test
- data/mtmv_p0/ivm
- suites/mtmv_p0/ivm
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
110 | 110 | | |
111 | 111 | | |
112 | 112 | | |
113 | | - | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
114 | 123 | | |
115 | 124 | | |
116 | 125 | | |
117 | | - | |
| 126 | + | |
| 127 | + | |
118 | 128 | | |
119 | 129 | | |
120 | 130 | | |
121 | 131 | | |
| 132 | + | |
122 | 133 | | |
123 | 134 | | |
124 | 135 | | |
| |||
237 | 248 | | |
238 | 249 | | |
239 | 250 | | |
240 | | - | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
241 | 255 | | |
242 | | - | |
| 256 | + | |
243 | 257 | | |
244 | 258 | | |
245 | 259 | | |
246 | | - | |
| 260 | + | |
247 | 261 | | |
248 | | - | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
249 | 268 | | |
250 | 269 | | |
251 | 270 | | |
252 | | - | |
| 271 | + | |
253 | 272 | | |
254 | 273 | | |
255 | 274 | | |
| |||
304 | 323 | | |
305 | 324 | | |
306 | 325 | | |
307 | | - | |
308 | | - | |
309 | | - | |
| 326 | + | |
| 327 | + | |
310 | 328 | | |
311 | 329 | | |
312 | 330 | | |
| |||
330 | 348 | | |
331 | 349 | | |
332 | 350 | | |
333 | | - | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
334 | 358 | | |
335 | 359 | | |
336 | 360 | | |
| |||
350 | 374 | | |
351 | 375 | | |
352 | 376 | | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
353 | 384 | | |
354 | 385 | | |
355 | 386 | | |
356 | 387 | | |
357 | | - | |
| 388 | + | |
358 | 389 | | |
359 | 390 | | |
360 | 391 | | |
| |||
367 | 398 | | |
368 | 399 | | |
369 | 400 | | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
370 | 416 | | |
371 | 417 | | |
372 | 418 | | |
| |||
449 | 495 | | |
450 | 496 | | |
451 | 497 | | |
452 | | - | |
453 | | - | |
454 | | - | |
455 | | - | |
456 | | - | |
457 | | - | |
458 | | - | |
459 | | - | |
460 | | - | |
461 | 498 | | |
462 | 499 | | |
463 | 500 | | |
| |||
487 | 524 | | |
488 | 525 | | |
489 | 526 | | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
490 | 538 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
81 | 81 | | |
82 | 82 | | |
83 | 83 | | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
84 | 98 | | |
85 | 99 | | |
86 | 100 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
383 | 383 | | |
384 | 384 | | |
385 | 385 | | |
386 | | - | |
387 | | - | |
| 386 | + | |
| 387 | + | |
388 | 388 | | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
389 | 399 | | |
390 | 400 | | |
391 | 401 | | |
| |||
416 | 426 | | |
417 | 427 | | |
418 | 428 | | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
419 | 444 | | |
420 | 445 | | |
421 | 446 | | |
| |||
927 | 952 | | |
928 | 953 | | |
929 | 954 | | |
930 | | - | |
931 | | - | |
932 | | - | |
933 | 955 | | |
934 | 956 | | |
935 | 957 | | |
936 | 958 | | |
937 | | - | |
| 959 | + | |
| 960 | + | |
| 961 | + | |
| 962 | + | |
| 963 | + | |
| 964 | + | |
| 965 | + | |
938 | 966 | | |
939 | 967 | | |
940 | | - | |
941 | | - | |
942 | | - | |
| 968 | + | |
943 | 969 | | |
944 | 970 | | |
945 | 971 | | |
| |||
0 commit comments