You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 8348047
Browse filesBrowse the repository at this point in the historyBrowse files
chore: Remove memory limits from property tests for now, and add spilling integration test. (#7439)
## What
Lift `work_mem` constraints in the `qgen` property tests and cover
DataFusion disk spilling in a dedicated parameterized `rstest` suite
(`tests/tests/spill.rs`).
## Why
Until `1c8c0bc9bf` (`#7108`), MPP spilling was covered by a `pg_regress`
test; `1c8c0bc9bf` replaced that with a single-shape `#[pg_test]` and
attempted to cover memory pressure more broadly via shape-dependent
`work_mem` bounds in the `qgen` property tests. In practice, upstream
DataFusion does not yet support spilling across enough operators
(notably `HashJoinExec`:
apache/datafusion#24768) for arbitrary
generated query shapes to run under tight `work_mem` limits.
## How
* Set `work_mem` to `1GB` unconditionally in `PgGucs::set_with`
(`tests/src/fixtures/querygen/mod.rs`), and drop the `work_mem`
overrides and `sketch_keys` workaround in `tests/tests/qgen.rs` and
`tests/src/fixtures/querygen/pdbagggen.rs`.
* Add `datafusion_spill_to_disk` in `tests/tests/spill.rs` parameterized
over `(Serial, Mpp)` and the spillable `AggregateScan` plan shapes.
0 commit comments