You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 246a397
Browse filesBrowse the repository at this point in the historyBrowse files
fix(duckdb): handle table filters in aggregate pushdown (#11071)
`TryReplaceAggregate` rewrites the scan's `column_ids` to one column per
aggregate but keeps its table filters, which DuckDB keys by position in
`column_ids`.
Over files partitioned by `region`, `SELECT sum(v) ... WHERE region =
'US-East'` fails with `INTERNAL Error: Attempted to access index 1
within vector of size 1`. With two aggregates the filter lands on `v`:
over files partitioned by `y`, `SELECT count(*), max(v) ... WHERE y =
2024` returns `0 NULL` instead of `2 3`.
The rewrite now:
- declines table filters on hive, `filename` and `file_index` columns,
which DuckDB evaluates per file by `column_ids`;
- converts the other table filters to Vortex filters, declines if one
doesn't convert, and clears them;
- reads storage indexes from `column_ids` before the resize.
## Tests
On `develop`, the new hive, `filename`, `file_index` and
`struct_extract_at` queries fail or return wrong results. `EXPLAIN`
cases check that file-column and `file_row_number` filters keep the
aggregate in the scan. Only the `struct_extract_at` query reaches the
conversion, and declines; with `pushdown_complex_filter` off, `v > 1`
and `i > 1` convert and keep the aggregate. The `vortex-duckdb` tests,
clippy, clang-format and the other SLTs pass; as on the previous head,
the TPC-H and TPC-DS runs stop at the `q11` and `q41` plans, so later
plans didn't run. ClickBench and the spatial e2e tests didn't run.
## AI assistance
Written with agentic AI assistance.
---------
Signed-off-by: jackylee-ch <[email protected]>
0 commit comments