Repository navigation
Commit 16dbb9d
fix: preserve projection field metadata during physical planning (apache#23981)
## Which issue does this PR close?
- Closes apache#23529.
## Rationale for this change
Logical projection aliases can attach field metadata, but physical
projection planning derived its output schema only from the unwrapped
physical expressions. This caused schema validation to reject
aggregates above such projections.
## What changes are included in this PR?
- Apply logical projection schema and field metadata when constructing
`ProjectionExec`.
- Preserve the existing physical derivation of field names, types, and
nullability.
- Add metadata-aware constructors across the physical-expression and
physical-plan crate boundary.
- Cover projections with alias metadata followed by aggregation.
## Are these changes tested?
Yes. A focused physical planner regression test reproduces the original
schema mismatch and passes with the fix. The relevant core and
physical-plan unit suites, targeted all-targets/all-features
clippy, formatting, and diff checks also pass.
## Are there any user-facing changes?
Projection field metadata is now preserved in physical plans. Two
additive constructor methods support passing projection schema metadata
across crate boundaries; no existing API is changed.1 parent 3bac7e0 commit 16dbb9d
3 files changed
Lines changed: 171 additions & 4 deletions
File tree
- datafusion
- core/src
- physical-expr/src
- physical-plan/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1106 | 1106 | | |
1107 | 1107 | | |
1108 | 1108 | | |
| 1109 | + | |
1109 | 1110 | | |
1110 | 1111 | | |
1111 | 1112 | | |
| |||
1329 | 1330 | | |
1330 | 1331 | | |
1331 | 1332 | | |
| 1333 | + | |
1332 | 1334 | | |
1333 | 1335 | | |
1334 | 1336 | | |
| |||
1343 | 1345 | | |
1344 | 1346 | | |
1345 | 1347 | | |
| 1348 | + | |
1346 | 1349 | | |
1347 | 1350 | | |
1348 | 1351 | | |
| |||
1727 | 1730 | | |
1728 | 1731 | | |
1729 | 1732 | | |
| 1733 | + | |
1730 | 1734 | | |
1731 | 1735 | | |
1732 | 1736 | | |
| |||
2958 | 2962 | | |
2959 | 2963 | | |
2960 | 2964 | | |
| 2965 | + | |
2961 | 2966 | | |
2962 | 2967 | | |
2963 | 2968 | | |
| |||
3015 | 3020 | | |
3016 | 3021 | | |
3017 | 3022 | | |
3018 | | - | |
| 3023 | + | |
| 3024 | + | |
| 3025 | + | |
| 3026 | + | |
| 3027 | + | |
3019 | 3028 | | |
3020 | 3029 | | |
3021 | 3030 | | |
| |||
3027 | 3036 | | |
3028 | 3037 | | |
3029 | 3038 | | |
3030 | | - | |
3031 | | - | |
| 3039 | + | |
| 3040 | + | |
| 3041 | + | |
| 3042 | + | |
| 3043 | + | |
3032 | 3044 | | |
3033 | 3045 | | |
3034 | 3046 | | |
| |||
3579 | 3591 | | |
3580 | 3592 | | |
3581 | 3593 | | |
| 3594 | + | |
| 3595 | + | |
| 3596 | + | |
| 3597 | + | |
| 3598 | + | |
| 3599 | + | |
| 3600 | + | |
| 3601 | + | |
| 3602 | + | |
| 3603 | + | |
| 3604 | + | |
| 3605 | + | |
| 3606 | + | |
| 3607 | + | |
| 3608 | + | |
| 3609 | + | |
| 3610 | + | |
| 3611 | + | |
| 3612 | + | |
| 3613 | + | |
| 3614 | + | |
| 3615 | + | |
| 3616 | + | |
| 3617 | + | |
| 3618 | + | |
| 3619 | + | |
| 3620 | + | |
| 3621 | + | |
| 3622 | + | |
| 3623 | + | |
| 3624 | + | |
| 3625 | + | |
| 3626 | + | |
| 3627 | + | |
| 3628 | + | |
| 3629 | + | |
| 3630 | + | |
3582 | 3631 | | |
3583 | 3632 | | |
3584 | 3633 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
539 | 539 | | |
540 | 540 | | |
541 | 541 | | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
542 | 592 | | |
543 | 593 | | |
544 | 594 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
43 | | - | |
| 43 | + | |
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
| |||
143 | 143 | | |
144 | 144 | | |
145 | 145 | | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
146 | 174 | | |
147 | 175 | | |
148 | 176 | | |
| |||
1395 | 1423 | | |
1396 | 1424 | | |
1397 | 1425 | | |
| 1426 | + | |
| 1427 | + | |
| 1428 | + | |
| 1429 | + | |
| 1430 | + | |
| 1431 | + | |
| 1432 | + | |
| 1433 | + | |
| 1434 | + | |
| 1435 | + | |
| 1436 | + | |
| 1437 | + | |
| 1438 | + | |
| 1439 | + | |
| 1440 | + | |
| 1441 | + | |
| 1442 | + | |
| 1443 | + | |
| 1444 | + | |
| 1445 | + | |
| 1446 | + | |
| 1447 | + | |
| 1448 | + | |
| 1449 | + | |
| 1450 | + | |
| 1451 | + | |
| 1452 | + | |
| 1453 | + | |
| 1454 | + | |
| 1455 | + | |
| 1456 | + | |
| 1457 | + | |
| 1458 | + | |
| 1459 | + | |
| 1460 | + | |
| 1461 | + | |
| 1462 | + | |
| 1463 | + | |
| 1464 | + | |
| 1465 | + | |
1398 | 1466 | | |
1399 | 1467 | | |
1400 | 1468 | | |
| |||
0 commit comments