Repository navigation
Commit 5c84f63
[refactor](Nereids) refactor column pruning (apache#17579)
This pr refactor the column pruning by the visitor, the good sides
1. easy to provide ability of column pruning for new plan by implement the interface `OutputPrunable` if the plan contains output field or do nothing if not contains output field, don't need to add new rule like `PruneXxxChildColumns`, few scenarios need to override the visit function to write special logic, like prune the LogicalSetOperation and Aggregate
2. support shrink output field in some plans, this can skip some useless operations so improvement
example:
```sql
select id
from (
select id, sum(age)
from student
group by id
)a
```
we should prune the useless `sum (age)` in the aggregate.
before refactor:
```
LogicalProject ( distinct=false, projects=[id#0], excepts=[], canEliminate=true )
+--LogicalSubQueryAlias ( qualifier=[a] )
+--LogicalAggregate ( groupByExpr=[id#0], outputExpr=[id#0, sum(age#2) AS `sum(age)`apache#4], hasRepeat=false )
+--LogicalProject ( distinct=false, projects=[id#0, age#2], excepts=[], canEliminate=true )
+--LogicalOlapScan ( qualified=default_cluster:test.student, indexName=<index_not_selected>, selectedIndexId=10007, preAgg=ON )
```
after refactor:
```
LogicalProject ( distinct=false, projects=[id#0], excepts=[], canEliminate=true )
+--LogicalSubQueryAlias ( qualifier=[a] )
+--LogicalAggregate ( groupByExpr=[id#0], outputExpr=[id#0], hasRepeat=false )
+--LogicalProject ( distinct=false, projects=[id#0], excepts=[], canEliminate=true )
+--LogicalOlapScan ( qualified=default_cluster:test.student, indexName=<index_not_selected>, selectedIndexId=10007, preAgg=ON )
```1 parent 7a67c95 commit 5c84f63
44 files changed
Lines changed: 628 additions & 583 deletions
File tree
- fe/fe-core/src
- main/java/org/apache/doris
- analysis
- nereids
- glue/translator
- jobs/batch
- rules
- rewrite/logical
- trees/plans
- algebra
- logical
- physical
- visitor
- util
- planner
- test/java/org/apache/doris/nereids
- postprocess
- rules/rewrite/logical
- util
- regression-test
- data/nereids_syntax_p0
- framework/src/main/groovy/org/apache/doris/regression/action
- suites
- mtmv_p0
- mv_p0
- multi_slot_multi_mv
- testAggQueryOnAggMV2
- mysql_ssl_p0
- nereids_p0
- aggregate
- sql_functions
- aggregate_functions
- array_functions
- json_function
- nereids_syntax_p0
- query_p0
- aggregate
- sql_functions/aggregate_functions
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
844 | 844 | | |
845 | 845 | | |
846 | 846 | | |
847 | | - | |
| 847 | + | |
848 | 848 | | |
849 | 849 | | |
850 | 850 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1671 | 1671 | | |
1672 | 1672 | | |
1673 | 1673 | | |
1674 | | - | |
| 1674 | + | |
| 1675 | + | |
| 1676 | + | |
| 1677 | + | |
| 1678 | + | |
1675 | 1679 | | |
1676 | 1680 | | |
1677 | 1681 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
22 | 23 | | |
23 | 24 | | |
24 | 25 | | |
| |||
56 | 57 | | |
57 | 58 | | |
58 | 59 | | |
| 60 | + | |
59 | 61 | | |
60 | 62 | | |
61 | 63 | | |
62 | 64 | | |
63 | 65 | | |
64 | 66 | | |
65 | 67 | | |
| 68 | + | |
| 69 | + | |
66 | 70 | | |
67 | 71 | | |
68 | 72 | | |
| |||
89 | 93 | | |
90 | 94 | | |
91 | 95 | | |
92 | | - | |
| 96 | + | |
93 | 97 | | |
94 | 98 | | |
95 | 99 | | |
| |||
117 | 121 | | |
118 | 122 | | |
119 | 123 | | |
120 | | - | |
| 124 | + | |
121 | 125 | | |
122 | 126 | | |
123 | | - | |
| 127 | + | |
| 128 | + | |
124 | 129 | | |
125 | 130 | | |
126 | 131 | | |
| |||
138 | 143 | | |
139 | 144 | | |
140 | 145 | | |
141 | | - | |
142 | | - | |
143 | | - | |
144 | | - | |
145 | | - | |
146 | | - | |
147 | | - | |
148 | | - | |
149 | | - | |
150 | | - | |
151 | | - | |
152 | | - | |
153 | | - | |
154 | | - | |
155 | | - | |
156 | | - | |
157 | | - | |
158 | | - | |
159 | | - | |
160 | | - | |
161 | | - | |
162 | | - | |
163 | | - | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
164 | 169 | | |
165 | 170 | | |
166 | 171 | | |
167 | | - | |
| 172 | + | |
168 | 173 | | |
169 | | - | |
| 174 | + | |
170 | 175 | | |
171 | | - | |
172 | | - | |
173 | | - | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
174 | 179 | | |
175 | | - | |
176 | | - | |
| 180 | + | |
| 181 | + | |
177 | 182 | | |
178 | | - | |
| 183 | + | |
179 | 184 | | |
180 | | - | |
181 | | - | |
182 | | - | |
183 | | - | |
184 | | - | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
185 | 190 | | |
186 | 191 | | |
187 | 192 | | |
188 | 193 | | |
189 | 194 | | |
190 | 195 | | |
191 | 196 | | |
192 | | - | |
| 197 | + | |
| 198 | + | |
193 | 199 | | |
194 | 200 | | |
195 | 201 | | |
196 | | - | |
197 | | - | |
198 | | - | |
199 | | - | |
200 | 202 | | |
201 | 203 | | |
202 | 204 | | |
203 | | - | |
204 | 205 | | |
205 | | - | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
206 | 226 | | |
207 | 227 | | |
208 | 228 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
88 | 88 | | |
89 | 89 | | |
90 | 90 | | |
| 91 | + | |
91 | 92 | | |
92 | 93 | | |
93 | 94 | | |
| |||
128 | 129 | | |
129 | 130 | | |
130 | 131 | | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
138 | | - | |
| 132 | + | |
139 | 133 | | |
140 | 134 | | |
141 | 135 | | |
| |||
0 commit comments