Skip to content

Commit 3fa5bc7

Browse files
adriangbclaude
andauthored
test: close five mutation-testing gaps in leaf expression pushdown guards (apache#25452)
## Which issue does this PR close? - Part of apache#25451. This PR closes 5 of the 14 real test gaps that the mutation run found. It does not close the issue. ## Rationale for this change `extract_leaf_expressions.rs` holds several guards that stop a rewrite in one specific case. Each guard went in with a bug fix, and each fix came with a plan snapshot test. A plan snapshot test asserts the shape of one output plan. It does not assert that the guard makes its decision for the reason the guard documents. A `cargo-mutants` run over the file shows the result. Three guards let a mutation through with no test failure: | Function | Mutation that survived | Effect if this happened for real | |---|---|---| | `passthrough_column` | guard `col.name == alias.name` to `true` | A renaming alias (`test.a AS b`) counts as a pass-through column. The recovery projection that restores the name is then dropped, and the rename is lost. | | `is_pure_extraction_projection` | body to `true`, and the `EXTRACTED_EXPR_PREFIX` guard to `true` | A projection that holds a `__common_expr_1` alias is pushed down a second time. The expression is re-extracted under a new alias that the parent cannot resolve. | | `merge_would_duplicate_kept_expr` | `>` to `>=` | The guard fires on a single reference site. Every merge into a child projection is then blocked, and leaf extraction stops working over a projection. | | `merge_would_duplicate_kept_expr` | `&&` to `\|\|` | Plain columns count as reference sites. A merge that touches only plain columns is blocked, although a plain column is cheap to duplicate. | Each of these is a silent behaviour change. The suite stays green. ## What changes are included in this PR? Three unit tests, one per function, in the existing `tests` module of `datafusion/optimizer/src/extract_leaf_expressions.rs`. Each test calls the guard directly and asserts both answers: the case where the guard must fire, and the case where it must not. The file already uses this style for `find_owning_input`. - `test_passthrough_column_rejects_renaming_alias` covers a bare column, a trivial rename that keeps the name, a rename that changes the name, and an alias over a non-column expression. - `test_is_pure_extraction_projection` covers a plan that is not a projection, a columns-only projection, an extraction projection, a projection that uses a `CommonSubexprEliminate` alias, and a projection with a bare expression. - `test_merge_would_duplicate_kept_expr_counts_only_kept_columns` covers one reference site to a `KeepInPlace` column, two sites, one site plus a pass-through reference, and two sites that reference a plain column only. The pass-through case is the one that apache#23655 reported. The PR adds a small `keep_in_place_udf` test helper. It gives a non-volatile `KeepInPlace` expression, which the existing `volatile_udf` helper cannot. There is no change to product code. ## What is the testing strategy for this PR? This PR is tests only. `cargo test --profile ci -p datafusion-optimizer` goes from 882 to 885 library tests. The 26 integration tests and 5 documentation tests are unchanged. All pass. To show that the new tests close the gaps, re-run `cargo-mutants` on the three functions: ```bash cargo mutants -p datafusion-optimizer \ -f 'datafusion/optimizer/src/extract_leaf_expressions.rs' \ -F 'passthrough_column|is_pure_extraction_projection|merge_would_duplicate_kept_expr' \ --profile ci --timeout 600 --jobs 2 ``` | | Before | After | |---|---|---| | Caught | 16 | 21 | | Missed | 5 | 0 | | Unviable | 1 | 1 | ## Are there any user-facing changes? No. Note: the test for `merge_would_duplicate_kept_expr` becomes dead when apache#25456 lands, because that PR deletes the function. Whichever merges second drops that one test; the other two tests apply unchanged. Part of the leaf-pushdown EPIC: apache#25459 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.1 <[email protected]>
1 parent 84b84e5 commit 3fa5bc7

1 file changed

Lines changed: 161 additions & 0 deletions

File tree

‎datafusion/optimizer/src/extract_leaf_expressions.rs‎

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3601,4 +3601,165 @@ mod tests {
36013601

36023602
Ok(())
36033603
}
3604+
3605+
/// `passthrough_column` tells `split_and_push_projection` which projection
3606+
/// expressions the input already produces. A renaming alias (`test.a AS b`)
3607+
/// does not qualify: it is a new output column. If the function accepted
3608+
/// it, the rename would count as captured, and the recovery projection
3609+
/// that restores the name would be dropped.
3610+
#[test]
3611+
fn test_passthrough_column_rejects_renaming_alias() {
3612+
// A bare column is a pass-through.
3613+
let bare = Expr::Column(Column::new(Some("test"), "a"));
3614+
assert_eq!(
3615+
passthrough_column(&bare).map(|c| c.flat_name()),
3616+
Some("test.a".to_string())
3617+
);
3618+
3619+
// `test.a AS a` keeps the name, so it is a pass-through.
3620+
let trivial_rename = Expr::Column(Column::new(Some("test"), "a")).alias("a");
3621+
assert_eq!(
3622+
passthrough_column(&trivial_rename).map(|c| c.flat_name()),
3623+
Some("test.a".to_string())
3624+
);
3625+
3626+
// `test.a AS b` changes the name, so it is not a pass-through.
3627+
let renaming = Expr::Column(Column::new(Some("test"), "a")).alias("b");
3628+
assert_eq!(passthrough_column(&renaming), None);
3629+
3630+
// An alias over anything other than a column is not a pass-through.
3631+
let not_a_column = lit(1).alias("a");
3632+
assert_eq!(passthrough_column(&not_a_column), None);
3633+
}
3634+
3635+
/// `is_pure_extraction_projection` decides if a merged projection can go
3636+
/// down one more level. It must accept only `__datafusion_extracted`
3637+
/// aliases and plain columns, and only with at least one extraction. An
3638+
/// alias with a different prefix must not count: a second push would
3639+
/// re-extract the expression under a new alias that the parent cannot
3640+
/// resolve.
3641+
#[test]
3642+
fn test_is_pure_extraction_projection() -> Result<()> {
3643+
let scan = test_table_scan_with_struct()?;
3644+
3645+
// Not a projection.
3646+
assert!(!is_pure_extraction_projection(&scan));
3647+
3648+
// Columns only: nothing was extracted.
3649+
let columns_only = LogicalPlanBuilder::from(scan.clone())
3650+
.project(vec![col("id"), col("user")])?
3651+
.build()?;
3652+
assert!(!is_pure_extraction_projection(&columns_only));
3653+
3654+
// One extraction alias and one pass-through column.
3655+
let pure = LogicalPlanBuilder::from(scan.clone())
3656+
.project(vec![
3657+
leaf_udf(col("user"), "status").alias("__datafusion_extracted_1"),
3658+
col("id"),
3659+
])?
3660+
.build()?;
3661+
assert!(is_pure_extraction_projection(&pure));
3662+
3663+
// The same shape under a `CommonSubexprEliminate` alias is not an
3664+
// extraction projection.
3665+
let common_expr = LogicalPlanBuilder::from(scan.clone())
3666+
.project(vec![
3667+
leaf_udf(col("user"), "status").alias("__common_expr_1"),
3668+
col("id"),
3669+
])?
3670+
.build()?;
3671+
assert!(!is_pure_extraction_projection(&common_expr));
3672+
3673+
// A bare expression is never allowed.
3674+
let bare_expr = LogicalPlanBuilder::from(scan)
3675+
.project(vec![leaf_udf(col("user"), "status"), col("id")])?
3676+
.build()?;
3677+
assert!(!is_pure_extraction_projection(&bare_expr));
3678+
3679+
Ok(())
3680+
}
3681+
3682+
/// A non-volatile `KeepInPlace` expression: the kind that
3683+
/// `CommonSubexprEliminate` hoists into a column so it runs one time.
3684+
fn keep_in_place_udf(expr: Expr) -> Expr {
3685+
Expr::ScalarFunction(ScalarFunction::new_udf(
3686+
Arc::new(ScalarUDF::new_from_impl(
3687+
PlacementTestUDF::new().with_placement(ExpressionPlacement::KeepInPlace),
3688+
)),
3689+
vec![expr],
3690+
))
3691+
}
3692+
3693+
/// `merge_would_duplicate_kept_expr` must block a merge only when it would
3694+
/// inline one `KeepInPlace` definition into more than one reference site.
3695+
/// It must count reference sites for `KeepInPlace` columns alone: a plain
3696+
/// column is cheap to duplicate, and a merge that touches only plain
3697+
/// columns must still go ahead.
3698+
#[test]
3699+
fn test_merge_would_duplicate_kept_expr_counts_only_kept_columns() -> Result<()> {
3700+
let scan = test_table_scan_with_struct()?;
3701+
// `k` holds a `KeepInPlace` definition. `id` is a plain column.
3702+
let child = Projection::try_new(
3703+
vec![keep_in_place_udf(col("user")).alias("k"), col("id")],
3704+
Arc::new(scan),
3705+
)?;
3706+
let no_columns = IndexSet::new();
3707+
3708+
// One reference site: the merge inlines one copy, so it is free.
3709+
let one_site = vec![(
3710+
get_field_like(col("k"), "a"),
3711+
"__datafusion_extracted_1".to_string(),
3712+
)];
3713+
assert!(!merge_would_duplicate_kept_expr(
3714+
&one_site,
3715+
&no_columns,
3716+
&child
3717+
));
3718+
3719+
// Two reference sites: the merge would evaluate `k` two times.
3720+
let two_sites = vec![
3721+
(
3722+
get_field_like(col("k"), "a"),
3723+
"__datafusion_extracted_1".to_string(),
3724+
),
3725+
(
3726+
get_field_like(col("k"), "b"),
3727+
"__datafusion_extracted_2".to_string(),
3728+
),
3729+
];
3730+
assert!(merge_would_duplicate_kept_expr(
3731+
&two_sites,
3732+
&no_columns,
3733+
&child
3734+
));
3735+
3736+
// A pass-through reference to `k` is a second site too (#23655).
3737+
let kept_column: IndexSet<Column> =
3738+
std::iter::once(Column::new_unqualified("k")).collect();
3739+
assert!(merge_would_duplicate_kept_expr(
3740+
&one_site,
3741+
&kept_column,
3742+
&child
3743+
));
3744+
3745+
// Two sites that reference the plain column `id` are cheap to
3746+
// duplicate, so the merge must still go ahead.
3747+
let plain_column_sites = vec![
3748+
(
3749+
get_field_like(col("id"), "a"),
3750+
"__datafusion_extracted_1".to_string(),
3751+
),
3752+
(
3753+
get_field_like(col("id"), "b"),
3754+
"__datafusion_extracted_2".to_string(),
3755+
),
3756+
];
3757+
assert!(!merge_would_duplicate_kept_expr(
3758+
&plain_column_sites,
3759+
&no_columns,
3760+
&child
3761+
));
3762+
3763+
Ok(())
3764+
}
36043765
}

0 commit comments

Comments
 (0)