Skip to content

Commit c120619

Browse files
authored
fix: use null counts to estimate IS NULL and IS NOT NULL (apache#25648)
## Which issue does this PR close? Closes apache#25620. ## Rationale for this change Bare `IS NULL` and `IS NOT NULL` filters use the default selectivity even when the input has a null count. For five rows containing two NULLs, both filters estimate one row instead of two and three. ## What changes are included in this PR? Use available null counts to estimate bare column null checks, while retaining the existing fallback for missing statistics and other predicates. Keep the output column statistics consistent with all-null, non-null, and empty results. Preserve the checked column's distinct count through the statistics provider when all its non-null values survive. Add regression coverage for exact and estimated counts, missing statistics, projection, fetch, and null filters above outer joins. The supporting changes in apache#25861, apache#25862, and apache#25863 have merged. This PR now contains only the null-filter change on current `main`. ## What is the testing strategy for this PR? The null-count and statistics-provider regressions fail against unchanged `main` production code. Compared 21 optimized `compute_statistics` coalesce, filter, and mixed-chain cases against `main` at `2717e25625ff0c51c1805f8df52d317979db74ec`, using the `release-nonlto` profile and 30 samples per case. Median time changes were +0.4% for coalesce controls, -1.1% for filter chains, and -5.1% for mixed chains. These local measurements cover statistics computation. ## Are there any user-facing changes? Null-check row estimates and related EXPLAIN statistics improve. Plan choices can change as a result. No public API changes.
1 parent caaf0fa commit c120619

3 files changed

Lines changed: 533 additions & 8 deletions

File tree

0 commit comments

Comments
 (0)