Repository navigation
Commit 90dc9a3
committed
fix: guard nested float normalization at every recursion level
Once an enclosing struct contained a float, normalize_nested_floats
rebuilt every child regardless of whether that child held any float
data. An empty struct child then reached StructArray::new, which cannot
infer a length from zero columns and panics. Iceberg exposes _partition
as struct<> on an unpartitioned table, so arrays_overlap over a struct
element carrying that column failed with a native exception where Spark
returns a result.
Apply the has_float_leaf guard inside normalize_nested_floats rather
than only at the call sites, so float-free subtrees are returned
unchanged. This fixes the panic and keeps the rebuild proportional to
the float data.1 parent e98a8f9 commit 90dc9a3
2 files changed
Lines changed: 116 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
546 | 546 | | |
547 | 547 | | |
548 | 548 | | |
549 | | - | |
| 549 | + | |
| 550 | + | |
550 | 551 | | |
551 | 552 | | |
552 | | - | |
| 553 | + | |
553 | 554 | | |
554 | 555 | | |
555 | 556 | | |
| |||
1095 | 1096 | | |
1096 | 1097 | | |
1097 | 1098 | | |
| 1099 | + | |
| 1100 | + | |
| 1101 | + | |
| 1102 | + | |
| 1103 | + | |
| 1104 | + | |
| 1105 | + | |
| 1106 | + | |
| 1107 | + | |
| 1108 | + | |
| 1109 | + | |
| 1110 | + | |
| 1111 | + | |
| 1112 | + | |
| 1113 | + | |
| 1114 | + | |
| 1115 | + | |
| 1116 | + | |
| 1117 | + | |
| 1118 | + | |
| 1119 | + | |
| 1120 | + | |
| 1121 | + | |
| 1122 | + | |
| 1123 | + | |
| 1124 | + | |
| 1125 | + | |
| 1126 | + | |
| 1127 | + | |
| 1128 | + | |
| 1129 | + | |
| 1130 | + | |
| 1131 | + | |
| 1132 | + | |
| 1133 | + | |
| 1134 | + | |
1098 | 1135 | | |
1099 | 1136 | | |
1100 | 1137 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
39 | 45 | | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
40 | 50 | | |
41 | 51 | | |
42 | 52 | | |
| |||
96 | 106 | | |
97 | 107 | | |
98 | 108 | | |
| 109 | + | |
99 | 110 | | |
100 | | - | |
| 111 | + | |
101 | 112 | | |
102 | 113 | | |
103 | 114 | | |
| |||
160 | 171 | | |
161 | 172 | | |
162 | 173 | | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
163 | 239 | | |
164 | 240 | | |
165 | 241 | | |
| |||
0 commit comments