Repository navigation
Commit ff89288
committed
fix: keep extension types when skip_metadata clears Parquet field metadata
With `skip_metadata = true` (the default), Parquet schema inference cleared all
field metadata, including the `ARROW:extension:name` and
`ARROW:extension:metadata` keys. Columns with an extension type were read as
their storage type.
Keep the extension type keys and clear the rest: the extension type is part of
the type of a field, not a writer annotation. Only top-level fields are handled;
nested fields still lose their metadata.
This is the fix proposed in apache#26517.
It changes how existing files with extension types (for example `arrow.uuid`)
are read.1 parent 4b8a6d5 commit ff89288
1 file changed
Lines changed: 13 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
| 37 | + | |
37 | 38 | | |
38 | 39 | | |
39 | 40 | | |
| |||
255 | 256 | | |
256 | 257 | | |
257 | 258 | | |
258 | | - | |
| 259 | + | |
| 260 | + | |
259 | 261 | | |
260 | 262 | | |
261 | 263 | | |
| |||
264 | 266 | | |
265 | 267 | | |
266 | 268 | | |
267 | | - | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
268 | 279 | | |
269 | 280 | | |
270 | 281 | | |
| |||
0 commit comments