-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
I am trying to convert Parquet data written using Avro Parquet Writer of Java. This parquet includes the Avro Schema inside the key parquet.avro.schema of the metadata, I want to convert this parquet data back to Avro using this schema and programmed in Rust.
I have tried to solve this challenge by converting Parquet into a json with string arrow_json::LineDelimitedWriter and then read this data with serde_json and serialise to Avro. In Avro Serializer I need that nullable fields are provided as Option<T>. The problem is that when I write the data into JSON null fields are skipped and I can't recover them as Options with serde_json
Describe the solution you'd like
I would like that arrow_json has a parameter for specifying if you want to output the null field.