You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit bcfe457
Browse filesBrowse the repository at this point in the historyBrowse files
feat: teach AggregateUDFImpl how it treats DISTINCT, and strip the flag when it is a no-op
Nothing recorded whether an aggregate cares about duplicates in its input.
`min(DISTINCT x)` already built a plain `MinAccumulator`, so the flag was a
no-op at execution time, but `SingleDistinctToGroupBy` still saw it and
rewrote `SELECT g, min(DISTINCT x) FROM t GROUP BY g` into an inner group by
at `(g, x)` grain. Callers also had no way to ask the question: the rule
identified sum/min/max by lowercased string name.
Add `AggregateUDFImpl::distinct_handling`, returning a three-valued
`DistinctHandling`:
- `Ignored` - the merge is idempotent, so `DISTINCT` cannot change the
result and the planner may drop it
- `Honored` - the accumulator reads `is_distinct` and deduplicates
- `Unsupported` - the accumulator does not implement `DISTINCT`
`Honored` is the default, so external UDFs are unaffected. The property is
forwarded from both the `AggregateUDF` wrapper and `AliasedAggregateUDFImpl`.
Tag the built-ins. `Ignored`: min, max, bool_and, bool_or, approx_distinct,
and the AND/OR arms of the bitwise operation (XOR cancels duplicate pairs and
stays `Honored`). `Unsupported`: the aggregates that either reject `DISTINCT`
already or silently return the non-distinct answer. first_value, last_value
and any_value keep the default with a TODO, because whether they ignore
duplicates depends on `ORDER BY`.
Add `EliminateAggregateDistinct`, inserted immediately before
`SingleDistinctToGroupBy`. It returns `Transformed::no` for every node that is
not an `Aggregate`, and uses `NamePreserver` to keep the output schema intact,
so the plan reads `min(t.v) AS min(DISTINCT t.v)`.
`Unsupported` is declared but not yet enforced. Rejecting `corr(DISTINCT x, y)`
fixes a silent wrong answer but turns queries that are accepted today into
planning errors, which belongs in its own change.
Over 4,000,000 rows in 2,000 groups, `min(DISTINCT x)` needed a 192M pool
before this change and completes under 1M after it.
Part of apache#24929, tracked as apache#11686.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_015YwFpxheiiUNMadTNyeHiu
| 15 |`propagate_empty_relation`| Pushes empty-relation knowledge upward so operators fed by no rows collapse early. |
55
-
| 16 |`filter_null_join_keys`| Adds `IS NOT NULL` filters to nullable equijoin keys that can never match. |
56
-
| 17 |`eliminate_outer_join`| Rewrites outer joins to inner joins when later filters reject the NULL-extended rows. |
57
-
| 18 |`push_down_limit`| Moves literal limits to scans/unions, merges limits, and pushes `Sort(fetch=N)` onto a preserved join side only when every sort key is from it; the outer Sort stays. |
58
-
| 19 |`push_down_filter`| Moves filters as early as possible through filter-commutative operators. |
0 commit comments