Skip to content

bigquery: Accelerate query result reading with the BigQuery Storage Read API #7031

Open
2 of 3 issues completed
Open

Description

@alvarowolfx

Accelerate RowIterator for large query result sets by automatically streaming results in Apache Arrow format over gRPC via the BigQuery Storage Read API (client::Read), while keeping the user-facing RowIterator and Row API unchanged.

Currently, RowIterator fetches subsequent pages of query results sequentially via BigQuery v2 with JSON pagination (jobs.getQueryResults), which becomes a bottleneck on medium-to-large result sets.

  • bigquery: Expose Arrow C Stream interface (FFI_ArrowArrayStream) on CompleteQuery
    • To be discussed if we should do it now

Activity

  1. self-assigned this
    on Sep 29, 2026
  2. added
    type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.
    api: bigqueryIssues related to the BigQuery API.
    on Sep 29, 2026
  3. changed the title [-]bigquery Accelerate query result reading with the BigQuery Storage Read API[/-] [+]bigquery: Accelerate query result reading with the BigQuery Storage Read API[/+] on Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the BigQuery API.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions