Skip to content

Commit f2869f3

Browse files
adriangbclaude
andcommitted
fix: keep the recovery projection for a computed same-name column
`split_and_push_projection` decided if it needs a recovery projection from the set of unqualified field names of the pushed plan. A name says nothing about the value behind it. A projection such as `(- t.a) AS a, t.s, get_field(t.s, "b") AS __datafusion_extracted_1` keeps every name when the extraction goes below it, but the pushed plan exposes the table column `t.a` where the projection computed `- t.a`. The recovery projection went away and the computed column became its own input column. The recovery projection now also stays when a recovery expression computes a value, that is, when it is not a pass-through of a column. The name comparison stays for the leaked-column case that it was written for. Closes apache#25414 Co-Authored-By: Claude Fable 5.1 <[email protected]>
1 parent 242b96c commit f2869f3

1 file changed

Lines changed: 16 additions & 1 deletion

File tree

‎datafusion/optimizer/src/extract_leaf_expressions.rs‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1111,6 +1111,18 @@ fn split_and_push_projection(
11111111
// `SubqueryAlias` re-qualification (`sub.__datafusion_extracted_1` vs
11121112
// `__datafusion_extracted_1`) that a qualified/ordered comparison would
11131113
// spuriously treat as drift, stacking redundant recovery projections.
1114+
//
1115+
// A name comparison alone is not sufficient. A name says nothing about the
1116+
// *value* behind it. Take the projection
1117+
// `(- t.a) AS a, t.s, get_field(t.s, "b") AS __datafusion_extracted_1`. When
1118+
// the extraction goes below it, the pushed plan keeps every name, but it
1119+
// exposes the table column `t.a` where the projection computed `- t.a`. If
1120+
// the recovery projection goes away, the computed column becomes its own
1121+
// input column and the query gives wrong results. See
1122+
// <https://github.com/apache/datafusion/issues/25414>.
1123+
//
1124+
// So the recovery projection also stays when a recovery expression computes
1125+
// a value, that is, when it is not a pass-through of a column.
11141126
let base_names: BTreeSet<&str> = base_plan
11151127
.schema()
11161128
.fields()
@@ -1122,7 +1134,10 @@ fn split_and_push_projection(
11221134
.iter()
11231135
.map(|f| f.name().as_str())
11241136
.collect();
1125-
let needs_recovery = base_names != original_names;
1137+
let computes_a_value = recovery_exprs
1138+
.iter()
1139+
.any(|expr| passthrough_column(expr).is_none());
1140+
let needs_recovery = base_names != original_names || computes_a_value;
11261141

11271142
// Wrap with recovery projection if the output schema changed
11281143
if needs_recovery {

0 commit comments

Comments
 (0)