Skip to content

perf(sql): Parquet export with functions in projection falls back to slow temp table path #6766

@bluestreak01

Description

@bluestreak01

When exporting SQL results to Parquet, if the projection contains functions (e.g. CAST, arithmetic expressions, or any other computed columns), the export falls back to using a temporary table, which is significantly slower than the direct export path.

Queries with simple column references export quickly, but adding any function to the SELECT list triggers the slow path.

Example

-- Fast (direct path)
COPY (SELECT col1, col2 FROM my_table) TO 'output.parquet';

-- Slow (temp table path)
COPY (SELECT CAST(col1 AS INT), col2 + 1 FROM my_table) TO 'output.parquet';

Expected behavior

The Parquet export should handle computed columns without falling back to a temporary table materialization step.

Metadata

Metadata

Assignees

Labels

easy winSmall, straightforward improvements

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions