Join planning inserts IS NOT NULL filters that provably pass 100% of rows when input statistics report zero nulls #24942

Summary

Join planning inserts null-rejecting IS NOT NULL filters on nullable join keys (filter_null_join_keys), which are then pushed down to scans. When the underlying data contains no NULLs — the overwhelmingly common case for join keys, e.g. TPC-H primary/foreign keys — these filters pass 100% of rows, yet every input batch still pays a full predicate evaluation plus a full batch copy.

On TPC-H SF=10 we measured selectivity = 100% (6.00M/6.00M) for such filters on join keys.

Why this is wasteful

FilterExec::statistics_helper already derives the forward direction: a surviving IS NOT NULL conjunct implies null_count = Exact(0) for the output. The inverse is missing: when input statistics already prove null_count = Exact(0) for a column, a bare Column IS NOT NULL conjunct is vacuously true and could be dropped from the predicate at plan-construction time (FilterExecBuilder::build). If all conjuncts are dropped, the predicate degenerates to lit(true) and the filter becomes a no-op pass-through (or can be elided entirely).

Non-provable cases (statistics Absent or null_count Inexact) must of course keep the predicate unchanged — this is strictly a statistics-driven refinement.

Impact

• Every join query over sources that report exact zero null-counts (Parquet with statistics, custom TableProviders with exact stats) executes no-op filter evaluations per batch.
• For plans with many joins (TPC-H), the redundant filters stack up on the scan side.

Related

• PR implementing the conjunct-drop at FilterExecBuilder::build: feat: drop provably-true IS NOT NULL conjuncts from FilterExec predicates via input null_count statistics #24821 (filed this issue per review discussion there)