You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Expose public async bloom filter reader (metadata + AsyncFileReader) #9067
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
I'm integrating Parquet bloom filters into an async pruning pipeline and found a gap in the public API.
For current situation,
There is a sync API: Sbbf::read_from_column_chunk(column_meta, reader)
There is an async method, but only on the async Arrow builder: ParquetRecordBatchStreamBuilder::get_row_group_column_bloom_filter(...)
The helper used internally to parse bloom filter headers is pub(crate): chunk_read_bloom_filter_header_and_offset (in parquet::bloom_filter)
If parquet crate only has ParquetMetaData + an AsyncFileReader, downstream applications can't read bloom filters without re‑implementing Parquet bloom header parsing.
This blocks async metadata‑only pruning libraries (like me) from using bloom filters safely and efficiently.
Describe the solution you'd like
Expose a public async bloom reader that mirrors the sync API:
allow async pruning without coupling to Arrow builder
avoid duplicate parsing logic in downstream crates
be backwards compatible (pure API addition)
Alternative
Make chunk_read_bloom_filter_header_and_offset public, but this is a low‑level parsing helper and would bake in more implementation detail.
Activity
alamb commentedon Jan 5, 2026
FYI @RoseZhang123 and @ODukhno made some improvements to the API for reading bloom filteres in this PR:
It isn't quite the super nice async api you envision, but it may be enough
alamb commentedon Mar 11, 2026
This came up in DataFusion as well:
ParquetPushDecoderinParquetOpenerdatafusion#20839 (comment)It would be very nice to be able to read bloom filters from the ParquetMetadataDecoder
feat(parquet): Add async `Sbbf::read_from_column_chunk_async` Bloom f…