Repository navigation
Commit 5f21582
fix: plan aggregates over CASE conditions with multiple columns (#25684)
## Which issue does this PR close?
- Closes #25369.
## Rationale for this change
A query that sums over a repeated CASE expression whose condition uses
two columns fails during planning. The reported query now returns 1
instead of an internal schema error.
## What changes are included in this PR?
CASE nullability inference now uses the existing three-valued bounds for
AND, OR and NOT when a condition cannot be evaluated without input rows.
The aggregate's schema check is unchanged.
The deduction that a branch is unreachable relies on NULL AND x never
being true, for any x, including NULL. OR and NOT keep the distinction
between false and NULL, so a branch that can still be reached with a
NULL value keeps the CASE nullable.
## What is the testing strategy for this PR?
The reported query runs through a small program on upstream main and on
this branch, and the same query is in cse.slt with SUM, COUNT, an
expanded BETWEEN, NULL inputs, a nullable or missing ELSE and an empty
input. The unit tests in case.rs cover independent columns in both
operand orders, a nested AND over OR condition and its negation checked
against evaluation on every combination of NULL inputs, and Boolean-cast
failures in literal conditions.
## Testing Done
Put the source below in src/main.rs of a new crate, add these
dependencies to its Cargo.toml with the path pointing at a checkout, and
run it with cargo run.
```toml
[dependencies]
datafusion = { path = "../datafusion/datafusion/core", default-features = false, features = ["sql"] }
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }
```
Before, on upstream main
(0d79252):
```text
SELECT sum(CASE WHEN (CASE WHEN a > 1 AND b < 3 THEN a ELSE 0 END) BETWEEN 1 AND 2 THEN 1 ELSE 0 END) AS total FROM cse_nullable_case
Error: Internal("Physical input schema should be the same as the one converted from logical input schema. Differences: \n\t- field nullability at index 0 [__common_expr_1]: (physical) true vs (logical) false")
exit=1
```
After, on this branch:
```text
SELECT sum(CASE WHEN (CASE WHEN a > 1 AND b < 3 THEN a ELSE 0 END) BETWEEN 1 AND 2 THEN 1 ELSE 0 END) AS total FROM cse_nullable_case
+-------+
| total |
+-------+
| 1 |
+-------+
exit=0
```
<details><summary>Reproducer source: main.rs</summary>
```rust
use datafusion::error::Result;
use datafusion::prelude::SessionContext;
#[tokio::main]
async fn main() -> Result<()> {
let ctx = SessionContext::new();
ctx.sql("CREATE TABLE cse_nullable_case(a INT, b INT) AS VALUES (1, 1), (2, 2), (3, 3)")
.await?
.collect()
.await?;
let sql = "SELECT sum(CASE WHEN (CASE WHEN a > 1 AND b < 3 THEN a ELSE 0 END) \
BETWEEN 1 AND 2 THEN 1 ELSE 0 END) AS total FROM cse_nullable_case";
println!("{sql}");
ctx.sql(sql).await?.show().await?;
Ok(())
}
```
</details>
## Are there any user-facing changes?
The affected aggregate queries plan and run. There is no public API
change.
---------
Signed-off-by: 1fanwang <[email protected]>
Co-authored-by: kosiew <[email protected]>1 parent 2ff1f76 commit 5f21582
2 files changed
Lines changed: 230 additions & 25 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| |||
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| 38 | + | |
| 39 | + | |
38 | 40 | | |
39 | 41 | | |
40 | 42 | | |
| |||
1292 | 1294 | | |
1293 | 1295 | | |
1294 | 1296 | | |
1295 | | - | |
1296 | | - | |
| 1297 | + | |
| 1298 | + | |
| 1299 | + | |
1297 | 1300 | | |
1298 | 1301 | | |
1299 | 1302 | | |
1300 | 1303 | | |
1301 | | - | |
1302 | | - | |
1303 | | - | |
1304 | | - | |
1305 | | - | |
1306 | | - | |
1307 | | - | |
| 1304 | + | |
1308 | 1305 | | |
1309 | 1306 | | |
1310 | 1307 | | |
| |||
1516 | 1513 | | |
1517 | 1514 | | |
1518 | 1515 | | |
1519 | | - | |
1520 | | - | |
1521 | | - | |
1522 | | - | |
1523 | | - | |
1524 | | - | |
| 1516 | + | |
| 1517 | + | |
1525 | 1518 | | |
1526 | 1519 | | |
1527 | 1520 | | |
| |||
1531 | 1524 | | |
1532 | 1525 | | |
1533 | 1526 | | |
1534 | | - | |
1535 | | - | |
1536 | | - | |
| 1527 | + | |
| 1528 | + | |
| 1529 | + | |
| 1530 | + | |
| 1531 | + | |
| 1532 | + | |
| 1533 | + | |
| 1534 | + | |
| 1535 | + | |
| 1536 | + | |
| 1537 | + | |
| 1538 | + | |
| 1539 | + | |
1537 | 1540 | | |
1538 | | - | |
1539 | | - | |
1540 | | - | |
| 1541 | + | |
| 1542 | + | |
1541 | 1543 | | |
1542 | | - | |
| 1544 | + | |
1543 | 1545 | | |
1544 | 1546 | | |
1545 | 1547 | | |
| |||
1594 | 1596 | | |
1595 | 1597 | | |
1596 | 1598 | | |
1597 | | - | |
| 1599 | + | |
1598 | 1600 | | |
1599 | 1601 | | |
1600 | 1602 | | |
| |||
1603 | 1605 | | |
1604 | 1606 | | |
1605 | 1607 | | |
1606 | | - | |
1607 | 1608 | | |
1608 | 1609 | | |
1609 | 1610 | | |
| |||
2796 | 2797 | | |
2797 | 2798 | | |
2798 | 2799 | | |
| 2800 | + | |
| 2801 | + | |
| 2802 | + | |
| 2803 | + | |
| 2804 | + | |
| 2805 | + | |
| 2806 | + | |
| 2807 | + | |
| 2808 | + | |
| 2809 | + | |
| 2810 | + | |
| 2811 | + | |
| 2812 | + | |
| 2813 | + | |
| 2814 | + | |
| 2815 | + | |
| 2816 | + | |
| 2817 | + | |
| 2818 | + | |
| 2819 | + | |
| 2820 | + | |
| 2821 | + | |
| 2822 | + | |
| 2823 | + | |
| 2824 | + | |
| 2825 | + | |
| 2826 | + | |
| 2827 | + | |
| 2828 | + | |
| 2829 | + | |
| 2830 | + | |
| 2831 | + | |
| 2832 | + | |
| 2833 | + | |
| 2834 | + | |
| 2835 | + | |
| 2836 | + | |
| 2837 | + | |
| 2838 | + | |
| 2839 | + | |
| 2840 | + | |
| 2841 | + | |
| 2842 | + | |
| 2843 | + | |
| 2844 | + | |
| 2845 | + | |
| 2846 | + | |
| 2847 | + | |
| 2848 | + | |
| 2849 | + | |
| 2850 | + | |
| 2851 | + | |
| 2852 | + | |
| 2853 | + | |
| 2854 | + | |
| 2855 | + | |
| 2856 | + | |
| 2857 | + | |
| 2858 | + | |
| 2859 | + | |
| 2860 | + | |
| 2861 | + | |
| 2862 | + | |
| 2863 | + | |
| 2864 | + | |
| 2865 | + | |
| 2866 | + | |
| 2867 | + | |
| 2868 | + | |
| 2869 | + | |
| 2870 | + | |
| 2871 | + | |
| 2872 | + | |
| 2873 | + | |
| 2874 | + | |
| 2875 | + | |
| 2876 | + | |
| 2877 | + | |
| 2878 | + | |
| 2879 | + | |
| 2880 | + | |
| 2881 | + | |
| 2882 | + | |
| 2883 | + | |
| 2884 | + | |
| 2885 | + | |
| 2886 | + | |
| 2887 | + | |
| 2888 | + | |
| 2889 | + | |
| 2890 | + | |
| 2891 | + | |
| 2892 | + | |
| 2893 | + | |
| 2894 | + | |
| 2895 | + | |
| 2896 | + | |
| 2897 | + | |
| 2898 | + | |
| 2899 | + | |
| 2900 | + | |
| 2901 | + | |
| 2902 | + | |
| 2903 | + | |
| 2904 | + | |
| 2905 | + | |
| 2906 | + | |
| 2907 | + | |
| 2908 | + | |
| 2909 | + | |
| 2910 | + | |
| 2911 | + | |
| 2912 | + | |
| 2913 | + | |
| 2914 | + | |
| 2915 | + | |
| 2916 | + | |
| 2917 | + | |
| 2918 | + | |
| 2919 | + | |
| 2920 | + | |
| 2921 | + | |
| 2922 | + | |
| 2923 | + | |
| 2924 | + | |
| 2925 | + | |
| 2926 | + | |
| 2927 | + | |
| 2928 | + | |
| 2929 | + | |
| 2930 | + | |
| 2931 | + | |
| 2932 | + | |
| 2933 | + | |
| 2934 | + | |
| 2935 | + | |
| 2936 | + | |
| 2937 | + | |
| 2938 | + | |
2799 | 2939 | | |
2800 | 2940 | | |
2801 | 2941 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
214 | 214 | | |
215 | 215 | | |
216 | 216 | | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
217 | 282 | | |
218 | 283 | | |
219 | 284 | | |
| |||
0 commit comments