Repository navigation
feat(spatial): prune contains and within filters using AABB statistics - #11372
Merged
Merged
Conversation
Register a contains pruning rule for both operand directions, including within filters lowered by swapping operands. Support nested geometry fields in file statistics and keep rule and file/zone tests together in contains.rs. Signed-off-by: Nemo Yu <[email protected]>
HarukiMoriarty
force-pushed
the
nemo/spatial-contains-pruning
branch
from
November 9, 2026 21:43
6d898f8 to
93e8632
Compare
HarukiMoriarty
marked this pull request as ready for review
November 9, 2026 21:49
robert3005
approved these changes
Nov 10, 2026
Signed-off-by: Robert Kruszewski <[email protected]>
robert3005
enabled auto-merge (squash)
November 10, 2026 12:14
ChristopherWirt
added a commit
to quantcast/vortex
that referenced
this pull request
Nov 11, 2026
## Summary QC `develop` is missing 13 upstream commits. Merge `vortex-data/vortex:develop` at `cb613ac1c0adc56cbe63dfc79e4bbc431ad0bc52` into QC `develop` at `b2c429ac2830b883cb37c1ff9262dc9e3cb7e483`. Merge this PR with a merge commit to preserve upstream ancestry for future syncs. ## Changes - Import typed Arrow extension plugins and round-trip metadata preservation, run-end export fixes, and DuckDB dictionary reuse across chunks ([vortex-data#11360](vortex-data#11360), [vortex-data#11375](vortex-data#11375), [vortex-data#11378](vortex-data#11378)). - Bring in improved string dictionary selection, spatial contains/within pruning, and safe PCO decoding ([vortex-data#9762](vortex-data#9762), [vortex-data#11372](vortex-data#11372), [vortex-data#11363](vortex-data#11363)). - Include CUDA Widen execution, bounded dictionary gathers, encoded-child host copies, and string dictionaries in the CUDA compression preset. - Reconcile the upstream Java I/O ports with their existing QC implementations, preserving read metrics, Kryo-safe Hadoop broadcasts, lifecycle handling, session defaults, and regression coverage. Correct the coalescing default's Javadoc to 1 MiB. ## Testing - Pinned `nightly-2026-09-10` Rust formatting for all affected crates, root/subproject `./gradlew spotlessApply`, and clang-format for the changed CUDA kernels passed. Reviewed and included the formatter diff. - `git diff --check` passed; no unresolved merge entries or conflict markers remain. The worktree is clean and both remote tips are parents of the signed-off merge commit. - All 13 imported commits have sign-off trailers and retain their original upstream metadata. - Tests, builds, lint, and benchmarks were not run locally, following repository guidance. --------- Signed-off-by: Daniel King <[email protected]> Signed-off-by: Tanish Gudise <[email protected]> Signed-off-by: Alexander Droste <[email protected]> Signed-off-by: Christopher Wirt <[email protected]> Signed-off-by: Nemo Yu <[email protected]> Signed-off-by: Connor Tsui <[email protected]> Signed-off-by: Will Manning <[email protected]> Signed-off-by: Robert Kruszewski <[email protected]> Co-authored-by: Dan King <[email protected]> Co-authored-by: Tanish Gudise <[email protected]> Co-authored-by: Claude Opus 5.5 <[email protected]> Co-authored-by: Alexander Droste <[email protected]> Co-authored-by: Connor Tsui <[email protected]> Co-authored-by: Nemo Yu <[email protected]> Co-authored-by: Will Manning <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ST_Containsfilters currently scan even when stored geometry bounds prove that a file or zone cannot match. Register oneSpatialContainsPrunerule that consumes the existingGeometryAabbstatistic in both file and zone pruning.Containment is directional:
ST_Contains(column, constant)can skip a scope whose aggregate box cannot cover the constant's box.ST_Contains(constant, column)can skip a scope whose aggregate box is disjoint from the constant's box. Partial overlap must remain eligible because individual rows can still match.DuckDB already lowers
ST_Within(a, b)toST_Contains(b, a), so it uses the same rule. Missing/null bounds, null/empty constants, touching boxes in the constant-container case, and unsupported predicate shapes remain conservative. There is no containment proof for negated filters.The shared column matcher also accepts direct and nested field paths so file-level proofs can use footer statistics. Unit cases cover bounding-box relations and operand order; write/read tests exercise file pruning without zones and zone pruning without file statistics.
This draft targets
developindependently of #10837. Default-statistics selection changes remain in that PR.Validation
git diff --cached --checkpassed.