You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
bigquery: Multi-stream ReadSession acceleration for large query results #7039
Extend StorageReader to automatically upgrade large query result reads (above the multi-stream row threshold) to a parallel multi-stream ReadSession against the query job's destination table.
Fetch the query job's destination_table via JobService::get_job when total_rows / total_estimated_row_count exceeds the multi-stream threshold.
Create a ReadSession with DataFormat::Arrow and CompressionCodec::Lz4Frame on the destination table.
Spawn concurrent stream reader tasks across session.streams sending decoded RecordBatches over a bounded channel to StorageReader::next_batch() (ensuring proper backpressure and task cancellation on drop).
Unit and integration tests for multi-stream session creation and fallback behavior when no destination table or only a single stream is available.
Activity
impl(bigquery): return all created ReadSessions
impl(bigquery): use fluent RPC setters