The new Polars GPU streaming execution engine allows users to crunch much larger datasets and easily scale to multiple GPUs. Exciting work!
The Polars GPU engine now runs on a new streaming backend, built with NVIDIA on their RapidsMPF library.
• Queries are no longer bound by GPU memory. Data flows through the query graph in morsels that spill to host memory when device memory gets tight.
• The same query scales
