Skip to content

Create Client.write_table and Client.sink_table methods for DataFrame and LazyFrame inputs, respectively #18

Open

Description

@tswast

See googleapis/google-cloud-python#15710 for a customer requesting this feature.

IMO, we should probably allow for different "engines":

from google.cloud import bigquery

client = bigquery.Client()

# Write DataFrame to stream as parquet file; does not hit disk
with io.BytesIO() as stream:
    df.write_parquet(stream)
    stream.seek(0)
    parquet_options = bigquery.ParquetOptions()
    parquet_options.enable_list_inference = True
    job = client.load_table_from_file(
        stream,
        destination='tablename',
        project='projectname',
        job_config=bigquery.LoadJobConfig(
            source_format=bigquery.SourceFormat.PARQUET,
            parquet_options=parquet_options,
        ),
    )
job.result()  # Waits for the job to complete
  • BQ Storage Write API
  • possibly: legacy streaming API

Activity

  1. changed the title [-]Create a `write_bigquery_table` and `sink_bigquery_table` methods for DataFrame and LazyFrame inputs, respectively[/-] [+]Create `Client.write_table` and `Client.sink_table` methods for DataFrame and LazyFrame inputs, respectively[/+] on Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions