Repository navigation
Commit 1504f92
committed
fix: skip empty metadata in intersect_metadata_for_union to prevent schema mismatch
When a UNION ALL combines columns from sources with field metadata
(e.g. Parquet) and NULL literals (which have no metadata), the
intersect_metadata_for_union function was dropping all metadata
because intersecting anything with an empty set yields an empty set.
After optimize_projections prunes unused columns and recompute_schema
rebuilds the Union via Union::try_new, the logical schema ends up
with empty metadata while the physical schema retains the original
field metadata from Parquet, causing a physical/logical schema
mismatch error.
The fix treats empty metadata as a non-vote in the intersection:
branches with no metadata (NULL literals, computed expressions) are
skipped, so only branches with actual metadata participate. When
non-empty branches conflict, their metadata is still correctly
intersected as before.
Closes apache#190491 parent 646d183 commit 1504f92
1 file changed
Lines changed: 79 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
512 | 512 | | |
513 | 513 | | |
514 | 514 | | |
515 | | - | |
516 | | - | |
517 | | - | |
518 | | - | |
| 515 | + | |
519 | 516 | | |
520 | 517 | | |
521 | | - | |
522 | | - | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
523 | 532 | | |
524 | 533 | | |
525 | | - | |
| 534 | + | |
526 | 535 | | |
527 | 536 | | |
528 | 537 | | |
| |||
4127 | 4136 | | |
4128 | 4137 | | |
4129 | 4138 | | |
| 4139 | + | |
| 4140 | + | |
| 4141 | + | |
| 4142 | + | |
| 4143 | + | |
| 4144 | + | |
| 4145 | + | |
| 4146 | + | |
| 4147 | + | |
| 4148 | + | |
| 4149 | + | |
| 4150 | + | |
| 4151 | + | |
| 4152 | + | |
| 4153 | + | |
| 4154 | + | |
| 4155 | + | |
| 4156 | + | |
| 4157 | + | |
| 4158 | + | |
| 4159 | + | |
| 4160 | + | |
| 4161 | + | |
| 4162 | + | |
| 4163 | + | |
| 4164 | + | |
| 4165 | + | |
| 4166 | + | |
| 4167 | + | |
| 4168 | + | |
| 4169 | + | |
| 4170 | + | |
| 4171 | + | |
| 4172 | + | |
| 4173 | + | |
| 4174 | + | |
| 4175 | + | |
| 4176 | + | |
| 4177 | + | |
| 4178 | + | |
| 4179 | + | |
| 4180 | + | |
| 4181 | + | |
| 4182 | + | |
| 4183 | + | |
| 4184 | + | |
| 4185 | + | |
| 4186 | + | |
| 4187 | + | |
| 4188 | + | |
| 4189 | + | |
| 4190 | + | |
| 4191 | + | |
| 4192 | + | |
| 4193 | + | |
| 4194 | + | |
| 4195 | + | |
| 4196 | + | |
| 4197 | + | |
| 4198 | + | |
| 4199 | + | |
| 4200 | + | |
| 4201 | + | |
4130 | 4202 | | |
0 commit comments