Repository navigation
Commit a0124a4
authored
fix: preserve projection field metadata during physical planning (apache#23981)
## Which issue does this PR close?
- Closes apache#23529.
## Rationale for this change
Logical projection aliases can attach field metadata, but physical
projection planning derived its output schema only from the unwrapped
physical expressions. This caused schema validation to reject
aggregates above such projections.
## What changes are included in this PR?
- Apply logical projection schema and field metadata when constructing
`ProjectionExec`.
- Preserve the existing physical derivation of field names, types, and
nullability.
- Add metadata-aware constructors across the physical-expression and
physical-plan crate boundary.
- Cover projections with alias metadata followed by aggregation.
## Are these changes tested?
Yes. A focused physical planner regression test reproduces the original
schema mismatch and passes with the fix. The relevant core and
physical-plan unit suites, targeted all-targets/all-features
clippy, formatting, and diff checks also pass.
## Are there any user-facing changes?
Projection field metadata is now preserved in physical plans. Two
additive constructor methods support passing projection schema metadata
across crate boundaries; no existing API is changed.1 parent e7304ee commit a0124a4
3 files changed
Lines changed: 171 additions & 4 deletions
File tree
- datafusion
- core/src
- physical-expr/src
- physical-plan/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1106 | 1106 | | |
1107 | 1107 | | |
1108 | 1108 | | |
| 1109 | + | |
1109 | 1110 | | |
1110 | 1111 | | |
1111 | 1112 | | |
| |||
1329 | 1330 | | |
1330 | 1331 | | |
1331 | 1332 | | |
| 1333 | + | |
1332 | 1334 | | |
1333 | 1335 | | |
1334 | 1336 | | |
| |||
1343 | 1345 | | |
1344 | 1346 | | |
1345 | 1347 | | |
| 1348 | + | |
1346 | 1349 | | |
1347 | 1350 | | |
1348 | 1351 | | |
| |||
1727 | 1730 | | |
1728 | 1731 | | |
1729 | 1732 | | |
| 1733 | + | |
1730 | 1734 | | |
1731 | 1735 | | |
1732 | 1736 | | |
| |||
2958 | 2962 | | |
2959 | 2963 | | |
2960 | 2964 | | |
| 2965 | + | |
2961 | 2966 | | |
2962 | 2967 | | |
2963 | 2968 | | |
| |||
3015 | 3020 | | |
3016 | 3021 | | |
3017 | 3022 | | |
3018 | | - | |
| 3023 | + | |
| 3024 | + | |
| 3025 | + | |
| 3026 | + | |
| 3027 | + | |
3019 | 3028 | | |
3020 | 3029 | | |
3021 | 3030 | | |
| |||
3027 | 3036 | | |
3028 | 3037 | | |
3029 | 3038 | | |
3030 | | - | |
3031 | | - | |
| 3039 | + | |
| 3040 | + | |
| 3041 | + | |
| 3042 | + | |
| 3043 | + | |
3032 | 3044 | | |
3033 | 3045 | | |
3034 | 3046 | | |
| |||
3579 | 3591 | | |
3580 | 3592 | | |
3581 | 3593 | | |
| 3594 | + | |
| 3595 | + | |
| 3596 | + | |
| 3597 | + | |
| 3598 | + | |
| 3599 | + | |
| 3600 | + | |
| 3601 | + | |
| 3602 | + | |
| 3603 | + | |
| 3604 | + | |
| 3605 | + | |
| 3606 | + | |
| 3607 | + | |
| 3608 | + | |
| 3609 | + | |
| 3610 | + | |
| 3611 | + | |
| 3612 | + | |
| 3613 | + | |
| 3614 | + | |
| 3615 | + | |
| 3616 | + | |
| 3617 | + | |
| 3618 | + | |
| 3619 | + | |
| 3620 | + | |
| 3621 | + | |
| 3622 | + | |
| 3623 | + | |
| 3624 | + | |
| 3625 | + | |
| 3626 | + | |
| 3627 | + | |
| 3628 | + | |
| 3629 | + | |
| 3630 | + | |
3582 | 3631 | | |
3583 | 3632 | | |
3584 | 3633 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
539 | 539 | | |
540 | 540 | | |
541 | 541 | | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
542 | 592 | | |
543 | 593 | | |
544 | 594 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
43 | | - | |
| 43 | + | |
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
| |||
143 | 143 | | |
144 | 144 | | |
145 | 145 | | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
146 | 174 | | |
147 | 175 | | |
148 | 176 | | |
| |||
1395 | 1423 | | |
1396 | 1424 | | |
1397 | 1425 | | |
| 1426 | + | |
| 1427 | + | |
| 1428 | + | |
| 1429 | + | |
| 1430 | + | |
| 1431 | + | |
| 1432 | + | |
| 1433 | + | |
| 1434 | + | |
| 1435 | + | |
| 1436 | + | |
| 1437 | + | |
| 1438 | + | |
| 1439 | + | |
| 1440 | + | |
| 1441 | + | |
| 1442 | + | |
| 1443 | + | |
| 1444 | + | |
| 1445 | + | |
| 1446 | + | |
| 1447 | + | |
| 1448 | + | |
| 1449 | + | |
| 1450 | + | |
| 1451 | + | |
| 1452 | + | |
| 1453 | + | |
| 1454 | + | |
| 1455 | + | |
| 1456 | + | |
| 1457 | + | |
| 1458 | + | |
| 1459 | + | |
| 1460 | + | |
| 1461 | + | |
| 1462 | + | |
| 1463 | + | |
| 1464 | + | |
| 1465 | + | |
1398 | 1466 | | |
1399 | 1467 | | |
1400 | 1468 | | |
| |||
0 commit comments