Repository navigation
Add examples for JSON and BSON logical types #101
Description
Recently, @emkornfield asked apache/parquet-site#153 (comment)
Just wanted to check for JSON and BSON other engines do support this with the caveat that they just pass through. Does polars not support them at all?
This was in the context of if polars supported a particular operation on such files However, there were no such files I could find in the parquet-testing repo that I could check with.
Thus I would like some example parquet files with JSON and BSON logical types, as specified https://parquet.apache.org/docs/file-format/types/logicaltypes/
These should be small files and come with a small readme that explains how they were created and what they contain
This would also help others implementing parquet to test their implementations with such files
Activity
[-]Add examples for JSON and BSON encoded files[/-][+]Add examples for JSON and BSON logical types[/+]emkornfield commentedon Feb 10, 2026
IIRC, I think Arrow C++ supports serializing canonical arrow JSON extension type to be marked with JSON (or at least there was an effort to do so that might have stalled).
alamb commentedon Feb 11, 2026
I am pretty sure the Rust writer can also annotate a record batch with the correct logical types (if you write an Arrow array with the Json canonical type: https://docs.rs/arrow-schema/57.3.0/arrow_schema/extension/enum.CanonicalExtensionType.html#variant.Json
I had a half written example to do this but I ran out of time
nssalian commentedon Jul 21, 2026
Happy to do this. @alamb do you mind assigning this to me?
alamb commentedon Jul 26, 2026
Thanks @nssalian and @emkornfield -- for examples I would like to recommend:
NULLvalue converage, as well as some JSON / BSON that have null as field values within non null fieldsnssalian commentedon Jul 26, 2026
Thanks @alamb. That makes sense. I'll put up a PR for this next week.