roapi
roapi copied to clipboard
Does ROAPI do row group pruning on parquet files stored on disk?
Hi,
We are trying to use ROAPI to efficiently run queries against Parquet files. These files are large, so storing them in-memory is not an option. We were hoping that ROAPI/Datafusion would use predicate pushdown/"pruning" on the row groups to speed up our queries, but creating more row groups and sorting our queried columns only seemed to slow queries down.
Just wondering if ROAPI is expected to prune queries on parquets when stored on disk? And if so, do we need to do something special to achieve this?
thank you!