You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 07bcd31
Browse filesBrowse the repository at this point in the historyBrowse files
[improvement](fe) Prune partition keys from partition topn sort (apache#63379)
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary: PartitionTopN still passed partition key expressions to
BE sort info when the same original slot also appeared in window order
keys. These keys are constant inside each partition, so keeping them
causes redundant comparison work and noisy explain output, especially
after slot remapping changes ExprIds.
``` before
| 1:VPartitionTopN(42) |
| | functions: rank |
| | partition by: c_region[#3] |
| | order by: c_region[#5] ASC, c_custkey[#6] ASC |
| | has global limit: false |
| | partition limit: 1 |
| | partition topn phase: TWO_PHASE_LOCAL_PTOPN |
| | distribute expr lists: c_custkey[#4]
```
after
```
| 1:VPartitionTopN(42) |
| | functions: rank |
| | partition by: c_region[#3] |
| | order by: c_custkey[#6] ASC |
| | has global limit: false |
| | partition limit: 1 |
| | partition topn phase: TWO_PHASE_LOCAL_PTOPN |
| | distribute expr lists: c_custkey[#4] |
| |
```
This PR also adds a guard for self-join cases where the partition key
and order key come from the same original table column, but from
different join inputs.
Example:
```sql
EXPLAIN
SELECT *
FROM (
SELECT
l.c2 AS lc2,
r.c2 AS rc2,
row_number() OVER (
PARTITION BY l.c2
ORDER BY r.c2
) AS rn
FROM test_global_partition_topn_plan l
JOIN test_global_partition_topn_plan r
ON l.c1 = r.c1
) tmp
WHERE rn <= 1;
```
Here l.c2 and r.c2 both come from test_global_partition_topn_plan.c2, but they are different slots after the self join. PARTITION BY l.c2 only guarantees that
the left-side c2 is constant inside each partition. It does not guarantee that the right-side r.c2 is constant, so ORDER BY r.c2 must not be pruned.
The expected EXPLAIN should still keep the order key in VPartitionTopN:
```
1:VPartitionTopN
| functions: row_number
| partition by: c2[#0]
| order by: c2[#3] ASC
| has global limit: false
| partition limit: 1
| partition topn phase: TWO_PHASE_LOCAL_PTOPN
The different slot ids, c2[#0] and c2[#3], show that the partition key
and order key are different join outputs. If the order key were pruned
here, the row
selected by rn <= 1 could become nondeterministic or incorrect.
```
Copy file name to clipboardExpand all lines: fe/fe-core/src/main/java/org/apache/doris/nereids/rules/implementation/LogicalPartitionTopNToPhysicalPartitionTopN.java
0 commit comments