Repository navigation
Open
Labels
Prolific Authorauthor opened PRs or issues in more than 20 repositories in the last 30 daysauthor opened PRs or issues in more than 20 repositories in the last 30 daysbugSomething isn't workingSomething isn't working
Description
Describe the bug
A query that groups by both CAST(a AS VARCHAR) and a, and has a common subexpression such as a + 1, fails to plan:
Optimizer rule 'optimize_projections' failed ... No field named t.a.
A slightly different query panics in optimize_projections instead.
To Reproduce
CREATE TABLE t (a INT) AS VALUES (1);
SELECT a, a + 1, (a + 1) * 2 FROM t GROUP BY CAST(a AS VARCHAR), a;Optimizer rule 'optimize_projections' failed
caused by
Schema error: No field named t.a.
Valid fields are __common_expr_1.
This variant panicked in about half of the runs (5 of 10, and 4 of 10 on a later check). The other runs returned the same schema error (it depends on HashMap iteration order):
CREATE TABLE t (a INT) AS VALUES (1);
SELECT a, a + 1, COALESCE(min('x'), CAST(a AS VARCHAR)) FROM t GROUP BY CAST(a AS VARCHAR), a;thread 'main' panicked at datafusion/optimizer/src/optimize_projections/mod.rs:590:82:
called `Result::unwrap()` on an `Err` value: SchemaError(FieldNotFound { field: Column { relation: Some(Bare { table: "t" }), name: "a" }, valid_fields: [Column { relation: None, name: "__common_expr_1" }, Column { relation: None, name: "t.a" }, Column { relation: None, name: "min(Utf8(\"x\"))" }] }, Some(""))
Expected behavior
The query returns one row:
SELECT a, a + 1, (a + 1) * 2 FROM t GROUP BY CAST(a AS VARCHAR), a;
# postgres 18.6: 1 | 2 | 4
# duckdb 1.5.6: 1 | 2 | 4
# datafusion main: Optimizer rule 'optimize_projections' failed ... No field named t.a.
Without the CAST group key (GROUP BY a), DataFusion returns 1 | 2 | 4.
Additional context
- Tested on
mainat c0ec512 (2026-10-22) withdatafusion-cli, default settings. - On 55.2.0 the same query fails earlier, in
eliminate_group_by_constant("Projections require unique expression names"). fix: keep EliminateGroupByConstant from requalifying group expressions #26356 fixed that error, and the query now reaches this bug. - Related, not duplicates:
optimize_projectionsfails with "No field named ..." afterpropagate_empty_relationremoves the first branch of a UNION #26516 (same error text, UNION afterpropagate_empty_relation) and Aggregates ofxandCAST(x AS ...)in one query fail with "duplicate unqualified field name", even with aliases #26221 (xandCAST(x AS ...)share a name). - Where:
build_common_expr_project_planincommon_subexpr_eliminate.rsdedups input fields by flat name (qualified_name(qualifier, name)). The aggregate outputs an unqualified field namedt.a(from the CAST) next to the qualifiedt.a. Both have the flat name"t.a", so the qualifiedt.ais dropped from the intermediate projection while the outer projection still uses it.
Found with an automated differential/metamorphic testing tool; reproduced on current main.
Willingness to contribute
I would like to contribute a fix for this bug, but request guidance from the DataFusion community
Metadata
Metadata
Assignees
Labels
Prolific Authorauthor opened PRs or issues in more than 20 repositories in the last 30 daysauthor opened PRs or issues in more than 20 repositories in the last 30 daysbugSomething isn't workingSomething isn't working
Activity
Sriram-PR commentedon Oct 24, 2026
take
Sriram-PR commentedon Oct 30, 2026
untake