Skip to content

refactor(spatial): make bounding-box statistics explicit - #10837

Open
HarukiMoriarty wants to merge 2 commits into
developfrom
nemo/spatial-file-stats
Open

HarukiMoriarty wants to merge 2 commits into
developfrom
nemo/spatial-file-stats

Conversation

@HarukiMoriarty

@HarukiMoriarty HarukiMoriarty commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Make spatial bounding-box statistics an explicit writer choice. Initializing vortex-spatial still registers geometry types, GeometryAabb, and spatial pruning rules, but no longer adds AABB statistics to zone maps automatically. Writers select file bounds with include_nested_statistics().with_file_statistics(...) and zone bounds through ZonedLayoutOptions::aggregate_fns.

Also fix spatial pruning for table files: the existing rules only recognized geometry columns at the expression root, so predicates such as ST_Intersects(geom, constant) could not use footer bounds for a struct field. The shared matcher now accepts direct and nested field paths while retaining the column-versus-literal restriction.

Changes

  • Remove the unused zone_stat_default hooks and registry discovery from vortex-array and the zoned writer. Built-in zone defaults remain unchanged.
  • Reuse the existing aggregate APIs and file format; no new core extension hook is introduced.
  • Cover explicit and default statistics, cross-batch and nested file bounds, file pruning without zone maps, both spatial operand orders, and zone pruning with opt-in bounds.
  • Document writer configuration and pruning limits: only supported ST_Intersects and ordered ST_Distance filters use the spatial rules; registering another compute function does not automatically enable pruning for it.

Validation

  • Reviewed the full patch, comments, callers, and statistics/pruning paths.
  • git diff --cached --check passed.
  • Based on develop at 2e1b05b84.
  • Local builds, tests, formatting, and lint commands were not run, following the repository's opt-in verification policy. The new regression cases have not been executed locally.

@codspeed

codspeed Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Merging this PR will degrade performance by 10.95%

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

❌ 3 regressed benchmarks
✅ 1913 untouched benchmarks
⏩ 48 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Walltime add_constant_shapes_neon[(32768, ConstantPerRow)] 14 µs 15.8 µs -11.22%
❌ Walltime add_constant_shapes_neon[(32768, PerRowNullableConstant)] 14.7 µs 16.5 µs -10.9%
❌ Walltime add_constant_shapes_neon[(32768, PerRowConstant)] 14.1 µs 15.8 µs -10.72%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing nemo/spatial-file-stats (203fcd6) with develop (30446ea)

Open in CodSpeed

Footnotes

  1. 48 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@connortsui20 connortsui20 mentioned this pull request Nov 2, 2026
12 of 49 tasks
@HarukiMoriarty
HarukiMoriarty force-pushed the nemo/spatial-file-stats branch from a5e57e7 to 2b4dec5 Compare November 9, 2026 20:28
Comment thread vortex-array/src/aggregate_fn/vtable.rs Resolved
Remove automatic zone-stat discovery and document explicit file and zone configuration. Let spatial pruning rules match nested field paths so footer AABB statistics can prune table files, and cover opt-in statistics and pruning with integration tests.

Signed-off-by: Nemo Yu <[email protected]>
@HarukiMoriarty
HarukiMoriarty force-pushed the nemo/spatial-file-stats branch from 2b4dec5 to aaea379 Compare November 9, 2026 21:15
@HarukiMoriarty HarukiMoriarty changed the title feat(spatial): write geometry bounding boxes to file statistics refactor(spatial): make bounding-box statistics explicit Nov 9, 2026
@HarukiMoriarty HarukiMoriarty changed the title refactor(spatial): make bounding-box statistics explicit feat(spatial): make AABB statistics opt-in and prune containment filters Nov 9, 2026
@HarukiMoriarty
HarukiMoriarty force-pushed the nemo/spatial-file-stats branch from 595bf9a to aaea379 Compare November 9, 2026 21:27
@HarukiMoriarty HarukiMoriarty changed the title feat(spatial): make AABB statistics opt-in and prune containment filters refactor(spatial): make bounding-box statistics explicit Nov 9, 2026
@robert3005
robert3005 enabled auto-merge (squash) November 10, 2026 12:04

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants