Skip to content

Commit 224cc56

Browse files
fix: prevent panic and incorrect results for COUNT with ORDER BY (apache#24997)
## Which issue does this PR close? - Closes apache#25055. ## Rationale for this change `COUNT` does not depend on input order, but it inherited the default `AggregateOrderSensitivity::HardRequirement`. As detailed in apache#25055, this could pass ordering keys to count accumulators as additional arguments, causing incorrect results or a panic, and could introduce unnecessary sorting. ## What changes are included in this PR? Override `Count::order_sensitivity` to return `AggregateOrderSensitivity::Insensitive`. This makes aggregate ordering keys ineffective for physical planning: they are excluded from accumulator inputs and no `SortExec` is required solely for `COUNT`'s `ORDER BY`. ## What is the testing strategy for this PR? Regression tests in `aggregate.slt` cover grouped, multi-argument, and non-grouped counts. The data includes both a null ordering key, which must not affect the count, and a null counted argument, which must still be excluded. A bare global `COUNT(a ORDER BY b)` over the in-memory test table is folded by `AggregateStatistics` to `num_rows - null_count(a)`, so `CountAccumulator` never runs. The test uses `a + 0`, which preserves `a`'s nullness but prevents that fold, ensuring the regression exercises the accumulator. An `EXPLAIN` assertion verifies that the count does not require a sort. ## Are there any user-facing changes? `COUNT(... ORDER BY ...)` returns the correct count without panicking. Nulls in counted arguments retain their usual behavior; nulls in ordering keys no longer incorrectly exclude rows.
1 parent 517ac6d commit 224cc56

2 files changed

Lines changed: 48 additions & 1 deletion

File tree

‎datafusion/functions-aggregate/src/count.rs‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ use datafusion_expr::{
4040
TypeSignature, Volatility, WindowFunctionDefinition,
4141
expr::WindowFunction,
4242
function::{AccumulatorArgs, StateFieldsArgs},
43-
utils::format_state_name,
43+
utils::{AggregateOrderSensitivity, format_state_name},
4444
};
4545
use datafusion_functions_aggregate_common::aggregate::count_distinct::PrimitiveDistinctCountGroupsAccumulator;
4646
use datafusion_functions_aggregate_common::aggregate::{
@@ -396,6 +396,10 @@ impl AggregateUDFImpl for Count {
396396
ReversedUDAF::Identical
397397
}
398398

399+
fn order_sensitivity(&self) -> AggregateOrderSensitivity {
400+
AggregateOrderSensitivity::Insensitive
401+
}
402+
399403
fn default_value(&self, _data_type: &DataType) -> Result<ScalarValue> {
400404
Ok(ScalarValue::Int64(Some(0)))
401405
}

‎datafusion/sqllogictest/test_files/aggregate.slt‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9883,3 +9883,46 @@ SELECT sum(s) FROM (SELECT sum(column2) OVER () AS s FROM nested_agg_t);
98839883

98849884
statement ok
98859885
DROP TABLE nested_agg_t;
9886+
9887+
# Tests for COUNT(... ORDER BY ...)
9888+
statement ok
9889+
CREATE TABLE count_order_by_t (a INT, b INT, c INT) AS VALUES
9890+
(1, NULL, 10),
9891+
(2, 20, 10),
9892+
(3, 30, 20),
9893+
(NULL, 40, 20);
9894+
9895+
# Single-argument grouped count with ORDER BY
9896+
query II rowsort
9897+
SELECT c, COUNT(a ORDER BY b) FROM count_order_by_t GROUP BY c;
9898+
----
9899+
10 2
9900+
20 1
9901+
9902+
# Multi-argument count with nullable ORDER BY key
9903+
query I
9904+
SELECT COUNT(a, c ORDER BY b) FROM count_order_by_t;
9905+
----
9906+
3
9907+
9908+
# Single-argument non-grouped count with nullable ORDER BY key
9909+
# A bare COUNT(a ORDER BY b) is folded to num_rows - null_count(a), bypassing
9910+
# CountAccumulator. a + 0 preserves nullness and forces execution.
9911+
query I
9912+
SELECT COUNT(a + 0 ORDER BY b) FROM count_order_by_t;
9913+
----
9914+
3
9915+
9916+
# COUNT does not require sorting its input, even with an explicit ORDER BY.
9917+
query TT
9918+
EXPLAIN SELECT COUNT(a + 0 ORDER BY b) FROM count_order_by_t;
9919+
----
9920+
logical_plan
9921+
01)Aggregate: groupBy=[[]], aggr=[[count(CAST(count_order_by_t.a AS Int64) + Int64(0)) ORDER BY [count_order_by_t.b ASC NULLS LAST]]]
9922+
02)--TableScan: count_order_by_t projection=[a, b]
9923+
physical_plan
9924+
01)AggregateExec: mode=Single, gby=[], aggr=[count(count_order_by_t.a + Int64(0)) ORDER BY [count_order_by_t.b ASC NULLS LAST]]
9925+
02)--DataSourceExec: partitions=1, partition_sizes=[1]
9926+
9927+
statement ok
9928+
DROP TABLE count_order_by_t;

0 commit comments

Comments
 (0)