Repository navigation
Commit df08808
authored
[fix](mv) Fix NPE when materialized view aggregate rewrite treats a derived group by projection as a group by key (#67362)
### What problem does this PR solve?
Problem Summary:
Given a sync materialized view on `sync_tz_base`:
```sql
CREATE MATERIALIZED VIEW sync_tz_day AS
SELECT date_trunc(ts, 'day') AS day_ts, sum(v) AS day_sum
FROM sync_tz_base WHERE ts IS NOT NULL
GROUP BY date_trunc(ts, 'day');
```
the following query fails during planning with an NPE:
```sql
SELECT CAST(date_trunc(ts, 'day') AS STRING) AS day_ts, SUM(v)
FROM sync_tz_base WHERE ts IS NOT NULL
GROUP BY date_trunc(ts, 'day');
```
```
java.lang.NullPointerException: Cannot invoke "org.apache.doris.analysis.Expr.getChildren()" because "root" is null
at org.apache.doris.analysis.Expr.extractSlots(Expr.java:173)
at org.apache.doris.nereids.glue.translator.PhysicalPlanTranslator.visitPhysicalProject(PhysicalPlanTranslator.java:2173)
```
**Root cause**
In `AbstractMaterializedViewAggregateRule.aggregateRewriteByView`, the
group by keys of the rewritten aggregate were collected from the query
**top plan** output expressions. For a `project -> aggregate` structure,
`topPlanSplitToGroupAndFunction` classifies the derived projection
`cast(date_trunc(ts, 'day') AS STRING)` as a group-by expression,
because it is derived from the real group key. This derived projection
was then wrongly added as a group by key of the rewritten aggregate:
```
finalGroupExpressions = [cast(day_ts#7 as TEXT) AS #9, day_ts#7]
finalOutputExpressions = [cast(day_ts#7 as TEXT) AS #9, sum(day_sum#8) AS #10]
```
This led to two problems:
1. A redundant group by key `cast(day_ts#7 as TEXT)` which is only a
projection of the real group key `day_ts#7`.
2. The real group key `day_ts#7` was added by the group-by compensation
logic but was **not** present in the aggregate output expressions, so
the top project referenced a slot that the physical aggregate never
produced, causing the `"root" is null` NPE during physical plan
translation.
**Fix**
The rewritten aggregate is now built directly from the query bottom
aggregate:
- The group by keys are the query bottom aggregate's group by
expressions rewritten against the MV scan (always correct, so the
previous group-by compensation is no longer needed).
- The aggregate output contains the rewritten group keys and the
rolled-up aggregate functions.
- The query top plan output expressions — including derived projections
of group keys such as `cast(date_trunc(ts, 'day') AS STRING)` — are
recomputed by a `LogicalProject` above the rewritten aggregate when they
cannot be produced by the aggregate directly.
After the fix the rewritten plan is valid and the query returns correct
results:
```
Project [cast(day_ts#7 as TEXT) AS day_ts#3, sum(day_sum#8) AS SUM(v)#4]
Aggregate [group by [day_ts#7], output [day_ts#7, sum(day_sum#8)]]
MVScan(sync_tz_day)
```1 parent d9f8373 commit df08808
3 files changed
Lines changed: 416 additions & 54 deletions
File tree
- fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv
- regression-test
- data/nereids_rules_p0/mv/agg_without_roll_up
- suites/nereids_rules_p0/mv/agg_without_roll_up
0 commit comments