Skip to content

Commit 8511e18

Browse files
refactor: centralize join-input table-scan filter extraction before u… (apache#23166)
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123. --> - Closes apache#23159 ## Rationale for this change <!-- Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed. Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes. --> Join unparsing handles the left and right inputs inline in `select_to_sql_recursively` (`datafusion/sql/src/unparser/plan.rs`). Both sides repeated the same table-scan filter extraction (`try_transform_to_simple_table_scan_with_filters`: extend the collected filters, then return the simplified or original plan), while the passthrough-`Projection(Join)` handling intentionally diverges per side (the left input can be unwrapped; the right input may need a nested-join relation so SQL preserves the required join grouping). Centralizing only the duplicated common setup — and keeping the side-specific behavior explicit — makes the two paths easier to read and keeps the alias-scope invariant in one obvious place. apache#23159 tracks this. ## What changes are included in this PR? <!-- There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR. --> - Extract the duplicated filter-extraction dispatch into a small helper `extract_join_input_table_scan_filters(plan, &mut table_scan_filters)`, and call it from both the left and right join-input paths. - Keep the side-specific passthrough-projection handling inline and explicit (left: `unwrap_qualified_passthrough_join_projection`; right: `qualified_passthrough_join_projection_to_nested_relation`). - Tests: factor the shared three-table setup into a `nested_passthrough_join_tables()` helper, and add `test_unparse_projected_join_unwraps_left_nested_passthrough_projection` to mirror the existing right-input test (this also fills a left-side coverage gap). This is a pure refactor with no behavior change. ## Are these changes tested? <!-- We typically require tests for all PRs in order to: 1. Prevent the code from being accidentally broken by subsequent changes 2. Serve as another way to document the expected behavior of the code If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> Covered by existing tests plus the new left-input test, all passing: - `cargo test -p datafusion-sql --test sql_integration` (includes both `..._left_..._passthrough_projection` and `..._right_..._passthrough_projection`) - `cargo clippy -p datafusion-sql --tests -- -D warnings` - `cargo fmt --all` Existing inline snapshots are unchanged (no behavior change); the new test locks in the left-input behavior the refactor touches. ## Are there any user-facing changes? <!-- If there are user-facing changes then we may require documentation to be updated before approving the PR. --> <!-- If there are any breaking changes to public APIs, please add the `api change` label. --> No. This is an internal refactor of the SQL unparser; the generated SQL is unchanged and there are no public API changes. Signed-off-by: Jiawei Zhao <[email protected]>
1 parent cae5f2e commit 8511e18

2 files changed

Lines changed: 77 additions & 19 deletions

File tree

‎datafusion/sql/src/unparser/plan.rs‎

Lines changed: 21 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -584,6 +584,19 @@ impl Unparser<'_> {
584584
self.project_window_output(&window_expr, select, None)
585585
}
586586

587+
fn extract_join_input_table_scan_filters(
588+
plan: &Arc<LogicalPlan>,
589+
table_scan_filters: &mut Vec<Expr>,
590+
) -> Result<Arc<LogicalPlan>> {
591+
match try_transform_to_simple_table_scan_with_filters(plan)? {
592+
Some((plan, filters)) => {
593+
table_scan_filters.extend(filters);
594+
Ok(Arc::new(plan))
595+
}
596+
None => Ok(Arc::clone(plan)),
597+
}
598+
}
599+
587600
#[cfg_attr(feature = "recursive_protection", recursive::recursive)]
588601
fn select_to_sql_recursively(
589602
&self,
@@ -1153,14 +1166,10 @@ impl Unparser<'_> {
11531166
// The outer projection plan will handle projecting the correct columns.
11541167
let already_projected = select.already_projected();
11551168

1156-
let left_plan =
1157-
match try_transform_to_simple_table_scan_with_filters(left_plan)? {
1158-
Some((plan, filters)) => {
1159-
table_scan_filters.extend(filters);
1160-
Arc::new(plan)
1161-
}
1162-
None => Arc::clone(left_plan),
1163-
};
1169+
let left_plan = Self::extract_join_input_table_scan_filters(
1170+
left_plan,
1171+
&mut table_scan_filters,
1172+
)?;
11641173
let left_plan = if already_projected {
11651174
Self::unwrap_qualified_passthrough_join_projection(left_plan)
11661175
} else {
@@ -1181,14 +1190,10 @@ impl Unparser<'_> {
11811190
None
11821191
};
11831192

1184-
let right_plan =
1185-
match try_transform_to_simple_table_scan_with_filters(right_plan)? {
1186-
Some((plan, filters)) => {
1187-
table_scan_filters.extend(filters);
1188-
Arc::new(plan)
1189-
}
1190-
None => Arc::clone(right_plan),
1191-
};
1193+
let right_plan = Self::extract_join_input_table_scan_filters(
1194+
right_plan,
1195+
&mut table_scan_filters,
1196+
)?;
11921197

11931198
let mut right_relation = RelationBuilder::default();
11941199
if already_projected

‎datafusion/sql/tests/cases/plan_to_sql.rs‎

Lines changed: 56 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2736,9 +2736,9 @@ fn test_unparse_inner_join_with_table_scan_projection() -> Result<()> {
27362736
Ok(())
27372737
}
27382738

2739-
#[test]
2740-
fn test_unparse_projected_join_unwraps_right_nested_passthrough_projection() -> Result<()>
2741-
{
2739+
/// Build the three base table scans (`left_table`, `mid_table`, `right_table`)
2740+
/// shared by the nested passthrough-projection join unparsing tests.
2741+
fn nested_passthrough_join_tables() -> Result<(LogicalPlan, LogicalPlan, LogicalPlan)> {
27422742
let left_schema = Schema::new(vec![
27432743
Field::new("left_id", DataType::Int32, false),
27442744
Field::new("mid_id", DataType::Int32, false),
@@ -2755,6 +2755,13 @@ fn test_unparse_projected_join_unwraps_right_nested_passthrough_projection() ->
27552755
let left = table_scan(Some("left_table"), &left_schema, None)?.build()?;
27562756
let mid = table_scan(Some("mid_table"), &mid_schema, None)?.build()?;
27572757
let right = table_scan(Some("right_table"), &right_schema, None)?.build()?;
2758+
Ok((left, mid, right))
2759+
}
2760+
2761+
#[test]
2762+
fn test_unparse_projected_join_unwraps_right_nested_passthrough_projection() -> Result<()>
2763+
{
2764+
let (left, mid, right) = nested_passthrough_join_tables()?;
27582765

27592766
let nested_right = LogicalPlanBuilder::from(mid)
27602767
.join(
@@ -2793,6 +2800,52 @@ fn test_unparse_projected_join_unwraps_right_nested_passthrough_projection() ->
27932800
Ok(())
27942801
}
27952802

2803+
#[test]
2804+
fn test_unparse_projected_join_unwraps_left_nested_passthrough_projection() -> Result<()>
2805+
{
2806+
let (left, mid, right) = nested_passthrough_join_tables()?;
2807+
2808+
// Left join input is a qualified passthrough `Projection(Join)`, and the
2809+
// outer join condition (`mid_table.right_id`) references an alias from
2810+
// inside it. The unparser must not wrap this in a derived table that would
2811+
// hide `mid_table.right_id` from the outer condition.
2812+
let nested_left = LogicalPlanBuilder::from(left)
2813+
.join(
2814+
mid,
2815+
datafusion_expr::JoinType::Inner,
2816+
(vec!["left_table.mid_id"], vec!["mid_table.mid_id"]),
2817+
None,
2818+
)?
2819+
.project(vec![
2820+
col("left_table.left_id"),
2821+
col("mid_table.mid_id"),
2822+
col("mid_table.right_id"),
2823+
])?
2824+
.build()?;
2825+
2826+
let plan = LogicalPlanBuilder::from(nested_left)
2827+
.join(
2828+
right,
2829+
datafusion_expr::JoinType::Inner,
2830+
(vec!["mid_table.right_id"], vec!["right_table.right_id"]),
2831+
None,
2832+
)?
2833+
.project(vec![
2834+
col("left_table.left_id"),
2835+
col("mid_table.mid_id"),
2836+
col("right_table.value"),
2837+
])?
2838+
.build()?;
2839+
2840+
let sql = plan_to_sql(&plan)?;
2841+
assert_snapshot!(
2842+
sql,
2843+
@r#"SELECT left_table.left_id, mid_table.mid_id, right_table."value" FROM left_table INNER JOIN mid_table ON left_table.mid_id = mid_table.mid_id INNER JOIN right_table ON mid_table.right_id = right_table.right_id"#
2844+
);
2845+
2846+
Ok(())
2847+
}
2848+
27962849
#[test]
27972850
fn test_unparse_left_semi_join_with_table_scan_projection() -> Result<()> {
27982851
let schema = Schema::new(vec![

0 commit comments

Comments
 (0)