Repository navigation
Commit 61c910f
authored
[fix](fe) Fix stream scan normalization and snapshot/reset pruning (#65468)
### What problem does this PR solve?
Issue Number: close #65469
Problem Summary: This PR fixes two FE issues in the stream scan path.
First, when a stream scan appears inside a CTE,
`NormalizeOlapTableStreamScan` may not be applied on the reused CTE
branch, so the stream virtual columns are not lowered consistently and
later rewrite steps see the wrong scan shape.
Second, `PruneEmptyPartition` uses `selectNonEmptyPartitionIds` before
stream scan normalization. For `stream@snapshot` and `stream@reset`, the
old stream-wrapper pruning logic still used incremental stream
visibility semantics, so a stream with no new delta could be pruned as
empty even though snapshot/reset should still read snapshot data.
This change keeps stream read mode as explicit logical scan state,
threads that mode through `LogicalCatalogRelation` into partition
pruning, preserves stream scan helper predicates on the logical node,
and lets snapshot/reset pruning follow the base table path while
incremental reads still use stream delta visibility. It also keeps
stream scan normalization behavior consistent when the scan is
referenced through a CTE.
As a follow-up for the alias copy path, this PR also fills the remaining
`LogicalOlapTableStreamScan` rebuild methods inherited from
`LogicalOlapScan`, so alias / virtual-column / topn rewrite paths do not
accidentally rebuild a stream scan as a plain `LogicalOlapScan` and skip
stream normalization.1 parent 9b26dc6 commit 61c910f
16 files changed
Lines changed: 351 additions & 63 deletions
File tree
- fe/fe-core/src
- main/java/org/apache/doris
- catalog
- stream
- nereids
- jobs/executor
- rules
- analysis
- exploration/mv
- rewrite
- trees/plans
- commands
- logical
- planner
- test/java/org/apache/doris/nereids
- rules/exploration/mv
- trees/plans
- regression-test
- data/table_stream_p0
- suites/table_stream_p0
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
33 | 33 | | |
34 | 34 | | |
35 | 35 | | |
| 36 | + | |
36 | 37 | | |
37 | 38 | | |
38 | 39 | | |
| |||
1564 | 1565 | | |
1565 | 1566 | | |
1566 | 1567 | | |
1567 | | - | |
| 1568 | + | |
| 1569 | + | |
1568 | 1570 | | |
1569 | 1571 | | |
1570 | 1572 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
20 | 21 | | |
21 | 22 | | |
22 | 23 | | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
26 | 27 | | |
| 28 | + | |
27 | 29 | | |
28 | 30 | | |
29 | 31 | | |
| |||
146 | 148 | | |
147 | 149 | | |
148 | 150 | | |
149 | | - | |
150 | | - | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
151 | 154 | | |
152 | 155 | | |
153 | 156 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
| 37 | + | |
37 | 38 | | |
38 | 39 | | |
39 | 40 | | |
| |||
174 | 175 | | |
175 | 176 | | |
176 | 177 | | |
177 | | - | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
178 | 184 | | |
179 | 185 | | |
180 | 186 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
21 | 20 | | |
22 | 21 | | |
23 | 22 | | |
| |||
408 | 407 | | |
409 | 408 | | |
410 | 409 | | |
| 410 | + | |
411 | 411 | | |
412 | 412 | | |
413 | 413 | | |
| |||
728 | 728 | | |
729 | 729 | | |
730 | 730 | | |
| 731 | + | |
| 732 | + | |
| 733 | + | |
731 | 734 | | |
732 | 735 | | |
733 | 736 | | |
| |||
892 | 895 | | |
893 | 896 | | |
894 | 897 | | |
895 | | - | |
896 | | - | |
897 | | - | |
898 | | - | |
899 | | - | |
900 | | - | |
901 | | - | |
902 | | - | |
903 | 898 | | |
904 | 899 | | |
905 | 900 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| 42 | + | |
42 | 43 | | |
43 | 44 | | |
44 | 45 | | |
| |||
341 | 342 | | |
342 | 343 | | |
343 | 344 | | |
344 | | - | |
| 345 | + | |
345 | 346 | | |
346 | | - | |
| 347 | + | |
347 | 348 | | |
348 | 349 | | |
349 | 350 | | |
| |||
1001 | 1002 | | |
1002 | 1003 | | |
1003 | 1004 | | |
1004 | | - | |
| 1005 | + | |
1005 | 1006 | | |
1006 | 1007 | | |
1007 | 1008 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
| 53 | + | |
53 | 54 | | |
54 | 55 | | |
55 | 56 | | |
| |||
176 | 177 | | |
177 | 178 | | |
178 | 179 | | |
179 | | - | |
| 180 | + | |
| 181 | + | |
180 | 182 | | |
181 | 183 | | |
182 | 184 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
52 | | - | |
| 52 | + | |
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
330 | 330 | | |
331 | 331 | | |
332 | 332 | | |
333 | | - | |
| 333 | + | |
334 | 334 | | |
335 | 335 | | |
336 | 336 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| 28 | + | |
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
| |||
194 | 195 | | |
195 | 196 | | |
196 | 197 | | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
197 | 202 | | |
198 | 203 | | |
199 | 204 | | |
| |||
0 commit comments