Repository navigation
Commit bb038a6
authored
fix: ScalarSubqueryToJoin fails when CTE inside correlated scalar subquery (#24363)
## Which issue does this PR close?
- Closes #24139
## Rationale for this change
The `scalar_subquery_to_join` optimizer rule was failing with a `Schema
error: No field named __always_true` when a CTE was used inside a
correlated scalar subquery.
This occurred because the `SubqueryAlias` logical plan node wasn't
refreshing its schema after the `__always_true` column was injected by
the correlated expression
pull-up logic (`PullUpCorrelatedExpr`). Because the schema became stale,
the parent `Projection` node would attempt to reference the newly
injected `__always_true`
column and fail with a schema error.
## What changes are included in this PR?
- Updated `PullUpCorrelatedExpr::f_up` logic for
`LogicalPlan::SubqueryAlias` in
`datafusion/optimizer/src/decorrelate.rs`.
- We now check if the `SubqueryAlias`'s input schema has a different
number of fields than its own recorded schema. If so, we recreate the
`SubqueryAlias` using
`LogicalPlanBuilder` so that its schema correctly reflects any newly
added columns (such as `__always_true`).
## Are these changes tested?
Yes, they are covered by the existing sqllogictests and optimizer
integration tests. Queries with CTEs inside correlated scalar subqueries
that previously crashed
the optimizer will now correctly optimize and execute.
## Are there any user-facing changes?
No, this is strictly an internal optimizer bug fix.1 parent 9a83691 commit bb038a6
2 files changed
Lines changed: 46 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
350 | 350 | | |
351 | 351 | | |
352 | 352 | | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
353 | 364 | | |
354 | | - | |
| 365 | + | |
355 | 366 | | |
356 | 367 | | |
357 | 368 | | |
358 | 369 | | |
359 | | - | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
360 | 377 | | |
361 | | - | |
362 | 378 | | |
363 | 379 | | |
364 | 380 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2657 | 2657 | | |
2658 | 2658 | | |
2659 | 2659 | | |
| 2660 | + | |
| 2661 | + | |
| 2662 | + | |
| 2663 | + | |
| 2664 | + | |
| 2665 | + | |
| 2666 | + | |
| 2667 | + | |
| 2668 | + | |
| 2669 | + | |
| 2670 | + | |
| 2671 | + | |
| 2672 | + | |
| 2673 | + | |
| 2674 | + | |
| 2675 | + | |
| 2676 | + | |
| 2677 | + | |
| 2678 | + | |
| 2679 | + | |
| 2680 | + | |
| 2681 | + | |
| 2682 | + | |
| 2683 | + | |
| 2684 | + | |
| 2685 | + | |
| 2686 | + | |
0 commit comments