Repository navigation
feat(layout): add a Filter plan over dense segment scans - #10389
joseph-isaacs wants to merge 5 commits into
Conversation
Add `Filter`, a plan operator with no predicate that keeps only the rows of the selection it is executed with. Its child produces every row of the same row domain, and values outside the selection are unspecified. Flat layouts now lower to a `Filter` over their `SegmentScan`, so a bare scan can stay dense and return every row of its range while the filter above it keeps the selected ones. Plan display snapshots are updated for the extra level. Signed-off-by: Joe Isaacs <[email protected]> Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01NWYmfHTFtaEdhdyd3u5hv3
Merging this PR will improve performance by 10.62%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | WallTime | words_gather_dispatch_neon[65536] |
2.3 µs | 2 µs | +10.62% |
| Simulation | bench_compare_primitive[(10000, 2)] |
64.4 µs | < 1 ns | N/A |
Tip
Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ji/exec-1-filter-plan (3670db8) with develop (8185e6a)2
Footnotes
-
534 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
No successful run was found on
develop(e0c00d5) during the generation of this report, so 8185e6a was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩
Lowering a flat layout returns the bare SegmentScan again; Filter stays an opt-in operator for the executor to insert. Filter carries no operator data, so its PlanData is () like Take's. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01DbnSuK9EpzvTSbrwpJT1sW Signed-off-by: Joe Isaacs <[email protected]>
Filter now has two children, [child, mask], where mask is a non-nullable bool plan over the same row domain. A predicate is Filter(x, Eval(pred, x)). The rows a plan is executed with become a Selection leaf that the executor binds per range, so the mask is a visible part of the plan rather than ambient execution state. Construction and with_children reject a mask of another dtype or row count, which catches domain mismatches under Take and ListPack. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01DbnSuK9EpzvTSbrwpJT1sW Signed-off-by: Joe Isaacs <[email protected]>
Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01DbnSuK9EpzvTSbrwpJT1sW Signed-off-by: Joe Isaacs <[email protected]>
|
This PR has been marked as stale because it has been open for 14 days with no activity. Please comment or remove the stale label if you wish to keep it active, otherwise it will be closed in 7 days |
|
Closing as superseded: Generated by Claude Code |
Summary
First piece pulled out of the V2 scan work in #10349. It prepares the plan layer for a push-based executor, which follows in a separate stack of five PRs starting with the
ExecNodetrait and exec graph.In that executor a bare
SegmentScanproduces every row of its range, taking the selection only as a hint, so that a predicate can be evaluated over a whole decoded segment and the selection applied afterwards. Something above the scan has to keep the selected rows. That is whatFilteris for.Changes
Filter, a plan operator with no predicate. The rows it keeps are the selection it is executed with. Its child produces every row of the same row domain, and values outside the selection are unspecified. Its dtype and row domain are its child's.lowernow lowers a flat layout toFilter(SegmentScan)instead of a bareSegmentScan.Filter.