Skip to content

Commit 06ede00

Browse files
adriangbclaude
authored andcommitted
test(proto): split roundtrip_physical_plan.rs by plan category (apache#24223)
`roundtrip_physical_plan.rs` had grown to 5,682 lines, so finding the tests for a given plan — or noticing that a plan has none — meant scrolling through every other plan's tests. apache#24171 (e) proposed splitting it by plan category; this does that. The tests now live in `tests/cases/plans/`, grouped by what they cover: leaves, dispatch, limits, filters, joins, sorts, aggregates, windows, sources, sinks, udfs, exprs, dynamic_filters, scalar_subquery, misc and tpch. The shared round-trip helpers (`roundtrip_test` and friends) stay in `plans/mod.rs`; every other item moved into exactly one file, and each file carries its own imports rather than inheriting one 140-line block. Pure code motion: no test was added, removed, renamed or edited. Every moved item is byte-identical to its previous form, and the test inventory (`--list`) is unchanged at 222 entries. ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## Rationale for this change <!-- Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed. Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes. Please explain the problem you are trying to solve in terms of the user-visible behavior, rather than the implementation. For example, "The code in `foo.rs` doesn't handle nulls" is a symptom of the implementation. "COUNT(DISTINCT) returns wrong results when the column contains nulls" is the user-visible problem. --> ## What changes are included in this PR? <!-- There is no need to duplicate the description in the issue here, but it is sometimes worth providing a summary of the individual changes in this PR. --> ## Are these changes tested? <!-- We typically require tests for all PRs in order to: 1. Prevent the code from being accidentally broken by subsequent changes 2. Serve as another way to document the expected behavior of the code If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> ## Are there any user-facing changes? <!-- If there are user-facing changes then we may require documentation to be updated before approving the PR. If there are any breaking changes to public APIs, please add the `api change` label. --> Co-authored-by: Claude <[email protected]>
1 parent 6529744 commit 06ede00

19 files changed

Lines changed: 6438 additions & 5786 deletions

‎datafusion/proto/tests/cases/mod.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,8 @@ use std::fmt::Debug;
3232
use std::hash::Hash;
3333
use std::sync::Arc;
3434

35+
mod plans;
3536
mod roundtrip_logical_plan;
36-
mod roundtrip_physical_plan;
3737
mod serialize;
3838
mod stack_safety;
3939

Lines changed: 256 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,256 @@
1+
// Licensed to the Apache Software Foundation (ASF) under one
2+
// or more contributor license agreements. See the NOTICE file
3+
// distributed with this work for additional information
4+
// regarding copyright ownership. The ASF licenses this file
5+
// to you under the Apache License, Version 2.0 (the
6+
// "License"); you may not use this file except in compliance
7+
// with the License. You may obtain a copy of the License at
8+
//
9+
// http://www.apache.org/licenses/LICENSE-2.0
10+
//
11+
// Unless required by applicable law or agreed to in writing,
12+
// software distributed under the License is distributed on an
13+
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
// KIND, either express or implied. See the License for the
15+
// specific language governing permissions and limitations
16+
// under the License.
17+
18+
//! `AggregateExec`.
19+
20+
use super::{roundtrip_test, roundtrip_test_with_context};
21+
use datafusion::arrow::array::ArrayRef;
22+
use datafusion::arrow::compute::kernels::sort::SortOptions;
23+
use datafusion::arrow::datatypes::{DataType, Field, Schema};
24+
use datafusion::logical_expr::Volatility;
25+
use datafusion::physical_expr::aggregate::AggregateExprBuilder;
26+
use datafusion::physical_plan::PhysicalExpr;
27+
use datafusion::physical_plan::aggregates::{
28+
AggregateExec, AggregateMode, LimitOptions, PhysicalGroupBy,
29+
};
30+
use datafusion::physical_plan::empty::EmptyExec;
31+
use datafusion::physical_plan::expressions::{PhysicalSortExpr, col, lit};
32+
use datafusion::prelude::SessionContext;
33+
use datafusion::scalar::ScalarValue;
34+
use datafusion_common::Result;
35+
use datafusion_expr::{
36+
Accumulator, AccumulatorFactoryFunction, AggregateUDF, Signature, SimpleAggregateUDF,
37+
};
38+
use datafusion_functions_aggregate::approx_percentile_cont::approx_percentile_cont_udaf;
39+
use datafusion_functions_aggregate::array_agg::array_agg_udaf;
40+
use datafusion_functions_aggregate::average::avg_udaf;
41+
use datafusion_functions_aggregate::nth_value::nth_value_udaf;
42+
use datafusion_functions_aggregate::string_agg::string_agg_udaf;
43+
use std::sync::Arc;
44+
use std::vec;
45+
46+
#[test]
47+
fn roundtrip_aggregate() -> Result<()> {
48+
let field_a = Field::new("a", DataType::Int64, false);
49+
let field_b = Field::new("b", DataType::Int64, false);
50+
let schema = Arc::new(Schema::new(vec![field_a, field_b]));
51+
52+
let groups: Vec<(Arc<dyn PhysicalExpr>, String)> =
53+
vec![(col("a", &schema)?, "unused".to_string())];
54+
55+
let avg_expr = AggregateExprBuilder::new(avg_udaf(), vec![col("b", &schema)?])
56+
.schema(Arc::clone(&schema))
57+
.alias("AVG(b)")
58+
.build()?;
59+
let nth_expr =
60+
AggregateExprBuilder::new(nth_value_udaf(), vec![col("b", &schema)?, lit(1u64)])
61+
.schema(Arc::clone(&schema))
62+
.alias("NTH_VALUE(b, 1)")
63+
.build()?;
64+
let str_agg_expr =
65+
AggregateExprBuilder::new(string_agg_udaf(), vec![col("b", &schema)?, lit(1u64)])
66+
.schema(Arc::clone(&schema))
67+
.alias("NTH_VALUE(b, 1)")
68+
.build()?;
69+
70+
let test_cases = vec![
71+
// AVG
72+
vec![Arc::new(avg_expr)],
73+
// NTH_VALUE
74+
vec![Arc::new(nth_expr)],
75+
// STRING_AGG
76+
vec![Arc::new(str_agg_expr)],
77+
];
78+
79+
for aggregates in test_cases {
80+
let schema = schema.clone();
81+
roundtrip_test(Arc::new(AggregateExec::try_new(
82+
AggregateMode::Final,
83+
PhysicalGroupBy::new_single(groups.clone()),
84+
aggregates,
85+
vec![None],
86+
Arc::new(EmptyExec::new(schema.clone())),
87+
schema,
88+
)?))?;
89+
}
90+
91+
Ok(())
92+
}
93+
94+
#[test]
95+
fn roundtrip_aggregate_with_limit() -> Result<()> {
96+
let field_a = Field::new("a", DataType::Int64, false);
97+
let field_b = Field::new("b", DataType::Int64, false);
98+
let schema = Arc::new(Schema::new(vec![field_a, field_b]));
99+
100+
let groups: Vec<(Arc<dyn PhysicalExpr>, String)> =
101+
vec![(col("a", &schema)?, "unused".to_string())];
102+
103+
let aggregates = vec![
104+
AggregateExprBuilder::new(avg_udaf(), vec![col("b", &schema)?])
105+
.schema(Arc::clone(&schema))
106+
.alias("AVG(b)")
107+
.build()
108+
.map(Arc::new)?,
109+
];
110+
111+
let agg = AggregateExec::try_new(
112+
AggregateMode::Final,
113+
PhysicalGroupBy::new_single(groups.clone()),
114+
aggregates,
115+
vec![None],
116+
Arc::new(EmptyExec::new(schema.clone())),
117+
schema,
118+
)?;
119+
let agg = agg.with_limit_options(Some(LimitOptions::new_with_order(12, false)));
120+
roundtrip_test(Arc::new(agg))
121+
}
122+
123+
#[test]
124+
fn roundtrip_aggregate_with_approx_pencentile_cont() -> Result<()> {
125+
let field_a = Field::new("a", DataType::Int64, false);
126+
let field_b = Field::new("b", DataType::Int64, false);
127+
let schema = Arc::new(Schema::new(vec![field_a, field_b]));
128+
129+
let groups: Vec<(Arc<dyn PhysicalExpr>, String)> =
130+
vec![(col("a", &schema)?, "unused".to_string())];
131+
132+
let aggregates = vec![
133+
AggregateExprBuilder::new(
134+
approx_percentile_cont_udaf(),
135+
vec![col("b", &schema)?, lit(0.5)],
136+
)
137+
.schema(Arc::clone(&schema))
138+
.alias("APPROX_PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY b)")
139+
.build()
140+
.map(Arc::new)?,
141+
];
142+
143+
let agg = AggregateExec::try_new(
144+
AggregateMode::Final,
145+
PhysicalGroupBy::new_single(groups.clone()),
146+
aggregates,
147+
vec![None],
148+
Arc::new(EmptyExec::new(schema.clone())),
149+
schema,
150+
)?;
151+
roundtrip_test(Arc::new(agg))
152+
}
153+
154+
#[test]
155+
fn roundtrip_aggregate_with_sort() -> Result<()> {
156+
let field_a = Field::new("a", DataType::Int64, false);
157+
let field_b = Field::new("b", DataType::Int64, false);
158+
let schema = Arc::new(Schema::new(vec![field_a, field_b]));
159+
160+
let groups: Vec<(Arc<dyn PhysicalExpr>, String)> =
161+
vec![(col("a", &schema)?, "unused".to_string())];
162+
let sort_exprs = vec![PhysicalSortExpr {
163+
expr: col("b", &schema)?,
164+
options: SortOptions {
165+
descending: false,
166+
nulls_first: true,
167+
},
168+
}];
169+
170+
let aggregates = vec![
171+
AggregateExprBuilder::new(array_agg_udaf(), vec![col("b", &schema)?])
172+
.schema(Arc::clone(&schema))
173+
.alias("ARRAY_AGG(b)")
174+
.order_by(sort_exprs)
175+
.build()
176+
.map(Arc::new)?,
177+
];
178+
179+
let agg = AggregateExec::try_new(
180+
AggregateMode::Final,
181+
PhysicalGroupBy::new_single(groups.clone()),
182+
aggregates,
183+
vec![None],
184+
Arc::new(EmptyExec::new(schema.clone())),
185+
schema,
186+
)?;
187+
roundtrip_test(Arc::new(agg))
188+
}
189+
190+
#[test]
191+
fn roundtrip_aggregate_udaf() -> Result<()> {
192+
let field_a = Field::new("a", DataType::Int64, false);
193+
let field_b = Field::new("b", DataType::Int64, false);
194+
let schema = Arc::new(Schema::new(vec![field_a, field_b]));
195+
196+
#[derive(Debug)]
197+
struct Example;
198+
impl Accumulator for Example {
199+
fn state(&mut self) -> Result<Vec<ScalarValue>> {
200+
Ok(vec![ScalarValue::Int64(Some(0))])
201+
}
202+
203+
fn update_batch(&mut self, _values: &[ArrayRef]) -> Result<()> {
204+
Ok(())
205+
}
206+
207+
fn merge_batch(&mut self, _states: &[ArrayRef]) -> Result<()> {
208+
Ok(())
209+
}
210+
211+
fn evaluate(&mut self) -> Result<ScalarValue> {
212+
Ok(ScalarValue::Int64(Some(0)))
213+
}
214+
215+
fn size(&self) -> usize {
216+
0
217+
}
218+
}
219+
220+
let return_type = DataType::Int64;
221+
let accumulator: AccumulatorFactoryFunction = Arc::new(|_| Ok(Box::new(Example)));
222+
223+
let udaf = AggregateUDF::from(SimpleAggregateUDF::new_with_signature(
224+
"example",
225+
Signature::exact(vec![DataType::Int64], Volatility::Immutable),
226+
return_type,
227+
accumulator,
228+
vec![Field::new("value", DataType::Int64, true).into()],
229+
));
230+
231+
let ctx = SessionContext::new();
232+
ctx.register_udaf(udaf.clone());
233+
234+
let groups: Vec<(Arc<dyn PhysicalExpr>, String)> =
235+
vec![(col("a", &schema)?, "unused".to_string())];
236+
237+
let aggregates = vec![
238+
AggregateExprBuilder::new(Arc::new(udaf), vec![col("b", &schema)?])
239+
.schema(Arc::clone(&schema))
240+
.alias("example_agg")
241+
.build()
242+
.map(Arc::new)?,
243+
];
244+
245+
roundtrip_test_with_context(
246+
Arc::new(AggregateExec::try_new(
247+
AggregateMode::Final,
248+
PhysicalGroupBy::new_single(groups.clone()),
249+
aggregates,
250+
vec![None],
251+
Arc::new(EmptyExec::new(schema.clone())),
252+
schema,
253+
)?),
254+
&ctx,
255+
)
256+
}

0 commit comments

Comments
 (0)