Repository navigation
Commit b39f3af
fix: preserve uncertainty in singleton filter statistics (apache#25863)
## Which issue does this PR close?
Related to apache#25620. Split from apache#25648 following [the review
request](apache#25648 (comment)).
This PR can be reviewed and merged independently.
## Rationale for this change
An equality filter can narrow a column's bounds to one value without
proving that any row contains that value. With input bounds [1, 100] and
actual values 1 and 100, `a = 5` returns no rows, but currently reports
an exact distinct count of one when its estimated row count is positive.
## What changes are included in this PR?
Report `Inexact(1)` for a singleton domain when the filtered row count
is an estimate or is unknown. Preserve exact zero for an exactly empty
output and exact one when an exact positive row count proves that the
singleton is present.
Add an execution regression for the absent value and update affected
unit-test and Parquet EXPLAIN expectations.
## What is the testing strategy for this PR?
Passed on this independent branch:
- `cargo test --profile ci -p datafusion-physical-plan --lib` (2,364
tests)
- `cargo clippy --profile ci -p datafusion-physical-plan --all-targets
--all-features -- -D warnings`
- `cargo test --profile ci -p datafusion-sqllogictest --test
sqllogictests -- parquet_statistics.slt`
- `cargo fmt --all` and `git diff --check`
Benchmarks were not rerun for this split. This extracts the
singleton-precision implementation already present in apache#25648.
## Are there any user-facing changes?
EXPLAIN reports an estimated distinct count when a matching row is not
proven. No public API changes.1 parent cfc602f commit b39f3af
2 files changed
Lines changed: 63 additions & 30 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1254 | 1254 | | |
1255 | 1255 | | |
1256 | 1256 | | |
1257 | | - | |
1258 | | - | |
1259 | | - | |
| 1257 | + | |
| 1258 | + | |
| 1259 | + | |
1260 | 1260 | | |
1261 | 1261 | | |
1262 | 1262 | | |
1263 | 1263 | | |
1264 | 1264 | | |
1265 | 1265 | | |
1266 | 1266 | | |
1267 | | - | |
1268 | | - | |
1269 | | - | |
1270 | | - | |
| 1267 | + | |
| 1268 | + | |
| 1269 | + | |
| 1270 | + | |
1271 | 1271 | | |
1272 | 1272 | | |
1273 | 1273 | | |
| |||
2990 | 2990 | | |
2991 | 2991 | | |
2992 | 2992 | | |
2993 | | - | |
| 2993 | + | |
2994 | 2994 | | |
2995 | 2995 | | |
2996 | 2996 | | |
| |||
3006 | 3006 | | |
3007 | 3007 | | |
3008 | 3008 | | |
3009 | | - | |
| 3009 | + | |
3010 | 3010 | | |
3011 | 3011 | | |
3012 | 3012 | | |
| |||
3022 | 3022 | | |
3023 | 3023 | | |
3024 | 3024 | | |
3025 | | - | |
| 3025 | + | |
3026 | 3026 | | |
3027 | 3027 | | |
3028 | 3028 | | |
| |||
3036 | 3036 | | |
3037 | 3037 | | |
3038 | 3038 | | |
3039 | | - | |
| 3039 | + | |
3040 | 3040 | | |
3041 | 3041 | | |
3042 | 3042 | | |
| |||
3093 | 3093 | | |
3094 | 3094 | | |
3095 | 3095 | | |
3096 | | - | |
| 3096 | + | |
3097 | 3097 | | |
3098 | 3098 | | |
3099 | 3099 | | |
| |||
3109 | 3109 | | |
3110 | 3110 | | |
3111 | 3111 | | |
3112 | | - | |
| 3112 | + | |
3113 | 3113 | | |
3114 | 3114 | | |
3115 | 3115 | | |
| |||
3130 | 3130 | | |
3131 | 3131 | | |
3132 | 3132 | | |
3133 | | - | |
| 3133 | + | |
3134 | 3134 | | |
3135 | 3135 | | |
3136 | 3136 | | |
| |||
3166 | 3166 | | |
3167 | 3167 | | |
3168 | 3168 | | |
3169 | | - | |
| 3169 | + | |
3170 | 3170 | | |
3171 | 3171 | | |
3172 | 3172 | | |
| |||
3231 | 3231 | | |
3232 | 3232 | | |
3233 | 3233 | | |
3234 | | - | |
| 3234 | + | |
3235 | 3235 | | |
3236 | 3236 | | |
3237 | 3237 | | |
| |||
3529 | 3529 | | |
3530 | 3530 | | |
3531 | 3531 | | |
3532 | | - | |
| 3532 | + | |
3533 | 3533 | | |
3534 | 3534 | | |
3535 | 3535 | | |
| |||
3544 | 3544 | | |
3545 | 3545 | | |
3546 | 3546 | | |
3547 | | - | |
| 3547 | + | |
3548 | 3548 | | |
3549 | 3549 | | |
3550 | 3550 | | |
| |||
3582 | 3582 | | |
3583 | 3583 | | |
3584 | 3584 | | |
3585 | | - | |
| 3585 | + | |
3586 | 3586 | | |
3587 | 3587 | | |
3588 | 3588 | | |
| |||
3616 | 3616 | | |
3617 | 3617 | | |
3618 | 3618 | | |
3619 | | - | |
| 3619 | + | |
3620 | 3620 | | |
3621 | 3621 | | |
3622 | 3622 | | |
| |||
3650 | 3650 | | |
3651 | 3651 | | |
3652 | 3652 | | |
3653 | | - | |
| 3653 | + | |
3654 | 3654 | | |
3655 | 3655 | | |
3656 | 3656 | | |
| |||
3684 | 3684 | | |
3685 | 3685 | | |
3686 | 3686 | | |
3687 | | - | |
| 3687 | + | |
3688 | 3688 | | |
3689 | 3689 | | |
3690 | 3690 | | |
| |||
3719 | 3719 | | |
3720 | 3720 | | |
3721 | 3721 | | |
3722 | | - | |
| 3722 | + | |
3723 | 3723 | | |
3724 | 3724 | | |
3725 | 3725 | | |
| |||
3766 | 3766 | | |
3767 | 3767 | | |
3768 | 3768 | | |
3769 | | - | |
| 3769 | + | |
3770 | 3770 | | |
3771 | 3771 | | |
3772 | 3772 | | |
| |||
4174 | 4174 | | |
4175 | 4175 | | |
4176 | 4176 | | |
| 4177 | + | |
| 4178 | + | |
| 4179 | + | |
| 4180 | + | |
| 4181 | + | |
| 4182 | + | |
| 4183 | + | |
| 4184 | + | |
| 4185 | + | |
| 4186 | + | |
| 4187 | + | |
| 4188 | + | |
| 4189 | + | |
| 4190 | + | |
| 4191 | + | |
| 4192 | + | |
| 4193 | + | |
| 4194 | + | |
| 4195 | + | |
| 4196 | + | |
| 4197 | + | |
| 4198 | + | |
| 4199 | + | |
| 4200 | + | |
| 4201 | + | |
| 4202 | + | |
| 4203 | + | |
| 4204 | + | |
| 4205 | + | |
| 4206 | + | |
| 4207 | + | |
| 4208 | + | |
| 4209 | + | |
4177 | 4210 | | |
4178 | 4211 | | |
4179 | 4212 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
62 | | - | |
| 62 | + | |
63 | 63 | | |
64 | 64 | | |
65 | 65 | | |
| |||
84 | 84 | | |
85 | 85 | | |
86 | 86 | | |
87 | | - | |
| 87 | + | |
88 | 88 | | |
89 | 89 | | |
90 | 90 | | |
| |||
152 | 152 | | |
153 | 153 | | |
154 | 154 | | |
155 | | - | |
| 155 | + | |
156 | 156 | | |
157 | 157 | | |
158 | 158 | | |
| |||
161 | 161 | | |
162 | 162 | | |
163 | 163 | | |
164 | | - | |
| 164 | + | |
165 | 165 | | |
166 | 166 | | |
167 | 167 | | |
| |||
170 | 170 | | |
171 | 171 | | |
172 | 172 | | |
173 | | - | |
| 173 | + | |
174 | 174 | | |
175 | 175 | | |
176 | 176 | | |
| |||
179 | 179 | | |
180 | 180 | | |
181 | 181 | | |
182 | | - | |
| 182 | + | |
183 | 183 | | |
184 | 184 | | |
185 | 185 | | |
| |||
0 commit comments