Repository navigation
Estimate Decimal filter selectivity #25613
Description
Describe the bug
Decimal types are excluded from interval analysis. Adding quantity and discount
restrictions to a date range causes the entire filter to receive a blanket 20%
estimate.
To Reproduce
From the repository root, with the CLI fix from
PR #25570 applied:
cargo build --profile ci --locked -p datafusion-benchmarks --bin dfbench
cargo install tpchgen-cli --version 1.1.1 --locked # if not already installed
repro_dir=$(mktemp -d)
tpchgen-cli --scale-factor 1 --format parquet \
--parquet-compression 'ZSTD(1)' --parts 1 --output-dir "$repro_dir/data"
cat > "$repro_dir/repro.sql" <<'SQL'
SET datafusion.execution.target_partitions = 1;
SET datafusion.optimizer.enable_dynamic_filter_pushdown = false;
SELECT l_orderkey FROM lineitem WHERE l_shipdate >= DATE '1994-01-01' AND l_shipdate < DATE '1995-01-01';
SELECT l_orderkey FROM lineitem WHERE l_shipdate >= DATE '1994-01-01' AND l_shipdate < DATE '1995-01-01' AND l_discount >= 0.05 AND l_discount <= 0.07 AND l_quantity < 24;
SQL
target/ci/dfbench statistics \
--path "$repro_dir/data" --query_path "$repro_dir/repro.sql"Observed with tpchgen-cli 1.1.1 at
6c320561b5.
Inspect the SELECT reports; ignore the empty SET reports.
| SELECT | FilterExec node | Estimated rows | Actual rows |
|---|---|---|---|
| Date range | 0 |
869,535 | 909,455 |
| Date + Decimal conditions | 0 |
1,200,243 | 114,160 |
Expected behavior
Support Decimal comparison selectivity with correct scales, coercions and
boundaries. Preserving the supported date condition is a separate improvement
from estimating the Decimal restrictions.
Additional context
This isolates TPC-H Q6’s predicate. Related:
filter selectivity #14237.
Part of #25610.
Activity
shrivasshankar commentedon Sep 27, 2026
take
feat: estimate Decimal filter selectivity with interval analysis (apa…