Repository navigation
Commit 55ed434
fix: omit ordering fields from the state of order-insensitive aggregates (apache#25402)
## Which issue does this PR close?
- Part of apache#25401. This fixes the `min`/`max` schema mismatch. The
`avg`/`bit_*`/`stddev`/`var_*` panics have a different cause and are
fixed in apache#25403; the two PRs are independent.
## Rationale for this change
An order-insensitive aggregate called with an `ORDER BY` fails whenever
the aggregation runs in two phases (`Partial` →
`Final`/`FinalPartitioned`), grouped or not:
```sql
CREATE TABLE d (g INT, k INT, v INT) AS VALUES (1, 2, 20), (1, 1, 10), (2, 4, 40), (2, 3, 30);
SELECT g, min(v ORDER BY k) FROM d GROUP BY g;
-- Arrow error: Invalid argument error: number of columns(2) must match number of fields(3) in schema
```
`AggregateExprBuilder::build()` derived the aggregate's
`ordering_fields` from its `ORDER BY` even when the function is
order-insensitive. `AggregateFunctionExpr::order_bys()` hides those
expressions from the accumulator, but `state_fields()` still passed
`ordering_fields`. `Min` and `Max` use the default
`AggregateUDFImpl::state_fields`, which appends them, while their
accumulators only emit the value. The partial state schema therefore
declared fields that no column filled. `sum`, `count`, `bool_and` and
`bool_or` override `state_fields` without ordering fields, so they
already worked.
## What changes are included in this PR?
`AggregateExprBuilder::build()` now drops `order_bys` when the
function's `order_sensitivity()` is insensitive. `ordering_fields` is
then empty from the start, so the aggregate carries no ordering state
that later code has to ignore. Order-sensitive aggregates are unchanged.
None of the built-in order-insensitive aggregates reads `order_bys` from
`AccumulatorArgs`, and plans display aggregates by `name()`, so EXPLAIN
output does not change.
## What is the testing strategy for this PR?
Two queries in `group_by.slt`, in the section that sets
`target_partitions = 8`, each selecting `MIN`, `MAX` and `SUM` with
`ORDER BY ts DESC` over `sales_global`, with `SUM` as a control:
- Grouped by `country`. The plan is `Partial` → `FinalPartitioned`. On
`main` it fails with `number of columns(4) must match number of
fields(6)`.
- Ungrouped, with `WHERE amount > 0` so the input is multi-partition and
the plan is `Partial` → `Final`. On `main` it fails with `number of
columns(3) must match number of fields(5)`. Without the filter, the
single-partition input runs in `Single` mode and never builds a partial
state.
Both queries pass with this change. The other `.slt` files covering
aggregates still pass (`group_by`, `aggregate`, `array_agg`,
`first_last_*`, `distinct_on`, `agg_func_substitute`, `window`,
`order`).
## Are there any user-facing changes?
Queries that failed now return results. No API changes.
## Notes for reviewers
The state field *names* for ordering fields are being reworked in
apache#25196. This PR only changes whether order-insensitive aggregates have
ordering fields at all, so the two shouldn't interact beyond a possible
textual rebase.
---------
Co-authored-by: Neil Conway <[email protected]>1 parent 8fa2c55 commit 55ed434
2 files changed
Lines changed: 135 additions & 10 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
264 | 264 | | |
265 | 265 | | |
266 | 266 | | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
267 | 276 | | |
268 | 277 | | |
269 | 278 | | |
| |||
772 | 781 | | |
773 | 782 | | |
774 | 783 | | |
775 | | - | |
776 | | - | |
777 | | - | |
778 | | - | |
779 | | - | |
| 784 | + | |
780 | 785 | | |
781 | 786 | | |
782 | 787 | | |
| |||
1052 | 1057 | | |
1053 | 1058 | | |
1054 | 1059 | | |
1055 | | - | |
1056 | | - | |
1057 | | - | |
1058 | | - | |
| 1060 | + | |
1059 | 1061 | | |
1060 | 1062 | | |
1061 | 1063 | | |
| |||
1192 | 1194 | | |
1193 | 1195 | | |
1194 | 1196 | | |
1195 | | - | |
| 1197 | + | |
| 1198 | + | |
| 1199 | + | |
1196 | 1200 | | |
1197 | 1201 | | |
1198 | 1202 | | |
| |||
1263 | 1267 | | |
1264 | 1268 | | |
1265 | 1269 | | |
| 1270 | + | |
| 1271 | + | |
| 1272 | + | |
| 1273 | + | |
| 1274 | + | |
| 1275 | + | |
| 1276 | + | |
| 1277 | + | |
| 1278 | + | |
| 1279 | + | |
| 1280 | + | |
| 1281 | + | |
| 1282 | + | |
| 1283 | + | |
| 1284 | + | |
| 1285 | + | |
| 1286 | + | |
| 1287 | + | |
| 1288 | + | |
| 1289 | + | |
| 1290 | + | |
| 1291 | + | |
| 1292 | + | |
| 1293 | + | |
| 1294 | + | |
| 1295 | + | |
| 1296 | + | |
| 1297 | + | |
| 1298 | + | |
| 1299 | + | |
| 1300 | + | |
| 1301 | + | |
| 1302 | + | |
| 1303 | + | |
| 1304 | + | |
| 1305 | + | |
| 1306 | + | |
| 1307 | + | |
| 1308 | + | |
| 1309 | + | |
| 1310 | + | |
| 1311 | + | |
| 1312 | + | |
| 1313 | + | |
| 1314 | + | |
| 1315 | + | |
| 1316 | + | |
| 1317 | + | |
| 1318 | + | |
| 1319 | + | |
| 1320 | + | |
| 1321 | + | |
| 1322 | + | |
| 1323 | + | |
| 1324 | + | |
| 1325 | + | |
| 1326 | + | |
| 1327 | + | |
| 1328 | + | |
| 1329 | + | |
| 1330 | + | |
| 1331 | + | |
| 1332 | + | |
| 1333 | + | |
| 1334 | + | |
| 1335 | + | |
| 1336 | + | |
| 1337 | + | |
| 1338 | + | |
| 1339 | + | |
| 1340 | + | |
| 1341 | + | |
| 1342 | + | |
| 1343 | + | |
| 1344 | + | |
| 1345 | + | |
| 1346 | + | |
| 1347 | + | |
| 1348 | + | |
| 1349 | + | |
| 1350 | + | |
| 1351 | + | |
| 1352 | + | |
| 1353 | + | |
| 1354 | + | |
| 1355 | + | |
| 1356 | + | |
| 1357 | + | |
| 1358 | + | |
| 1359 | + | |
| 1360 | + | |
| 1361 | + | |
| 1362 | + | |
| 1363 | + | |
1266 | 1364 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3075 | 3075 | | |
3076 | 3076 | | |
3077 | 3077 | | |
| 3078 | + | |
| 3079 | + | |
| 3080 | + | |
| 3081 | + | |
| 3082 | + | |
| 3083 | + | |
| 3084 | + | |
| 3085 | + | |
| 3086 | + | |
| 3087 | + | |
| 3088 | + | |
| 3089 | + | |
| 3090 | + | |
| 3091 | + | |
| 3092 | + | |
| 3093 | + | |
| 3094 | + | |
| 3095 | + | |
| 3096 | + | |
| 3097 | + | |
| 3098 | + | |
| 3099 | + | |
| 3100 | + | |
| 3101 | + | |
| 3102 | + | |
| 3103 | + | |
| 3104 | + | |
3078 | 3105 | | |
3079 | 3106 | | |
3080 | 3107 | | |
| |||
0 commit comments