Skip to content

arrow_json::LineDelimitedWriter: Write nulls #3774

@WaterKnight1998

Description

@WaterKnight1998

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

I am trying to convert Parquet data written using Avro Parquet Writer of Java. This parquet includes the Avro Schema inside the key parquet.avro.schema of the metadata, I want to convert this parquet data back to Avro using this schema and programmed in Rust.

I have tried to solve this challenge by converting Parquet into a json with string arrow_json::LineDelimitedWriter and then read this data with serde_json and serialise to Avro. In Avro Serializer I need that nullable fields are provided as Option<T>. The problem is that when I write the data into JSON null fields are skipped and I can't recover them as Options with serde_json

Describe the solution you'd like

I would like that arrow_json has a parameter for specifying if you want to output the null field.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementAny new improvement worthy of a entry in the changeloghelp wanted

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions