Skip to content

Commit da8a7c8

Browse files
joseph-isaacsclaude
andcommitted
Call the zone-map plans pruning plans, not proofs
The scan code named the filter's zone-map plans proofs. They are the filter's pruning expressions evaluated over the file's zone map, so the layout, file and DataFusion code now says pruning, matching prune_zones, PrunedZones and the pruning field they already used. Behaviour is unchanged. Co-Authored-By: Claude Fable 5.1 <[email protected]> Signed-off-by: Joe Isaacs <[email protected]>
1 parent 4cad9a8 commit da8a7c8

12 files changed

Lines changed: 59 additions & 59 deletions

File tree

‎vortex-datafusion/src/persistent/opener.rs‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -121,7 +121,7 @@ pub(crate) struct VortexOpener {
121121
/// shared layout reader.
122122
pub segment_sources: Arc<DashMap<Path, Weak<dyn SegmentSource>>>,
123123
/// The V2 scan cache each open file's partitions share: its lowered layout, filter and
124-
/// projection plans, proved zones and dictionary values, built once per file.
124+
/// projection plans, pruned zones and dictionary values, built once per file.
125125
pub scan_caches: Arc<DashMap<Path, WeakScanCache>>,
126126
/// Shared full-file natural splits keyed by file path.
127127
pub natural_splits: Arc<DashMap<Path, Arc<NaturalSplits>>>,
@@ -553,7 +553,7 @@ impl FileOpener for VortexOpener {
553553
.map_err(|e| exec_datafusion_err!("Failed to plan Vortex scan: {e}"))?;
554554
let scan = Arc::new(scan);
555555

556-
// The planners, composed: with a filter, its zone proofs first over the whole
556+
// The planners, composed: with a filter, its pruning plans first over the whole
557557
// partition, continuing into the chunks of the rows they keep; without a
558558
// filter, the chunks directly. A chunk plans its conjuncts, then its
559559
// projection.

‎vortex-file/src/v2/scan.rs‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
//! Drives a V2 scan of a file for callers without an engine of their own.
55
//!
66
//! A planner says what it needs: compute, or reads. This driver decides where each runs. It
7-
//! composes the planners over a [`PreparedScan`], the zone proofs first when the scan has a
7+
//! composes the planners over a [`PreparedScan`], the pruning plans first when the scan has a
88
//! filter and the chunks of the rows they keep after, and runs them on planning runs it owns:
99
//! the root run stepped on `spawn_cpu` tasks as the stream is polled, each spawned chunk on a
1010
//! task of its own, up to the concurrency, or on the root run at a concurrency of one. Every
@@ -112,7 +112,7 @@ impl ScanDriver {
112112
}
113113
}
114114

115-
/// The planners of a scan, composed: with a filter, its proofs first over the whole scan,
115+
/// The planners of a scan, composed: with a filter, its pruning plans first over the whole scan,
116116
/// continuing into the chunks of the rows they keep; without, the chunks directly.
117117
fn compose(scan: &Arc<PreparedScan>) -> Box<dyn Planner> {
118118
let rows = scan.rows().clone();

‎vortex-layout/src/plan/physical/plans/legacy_stats.rs‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ use crate::plan::physical::optimizer::PlanParentReduceRule;
3030
use crate::plan::physical::optimizer::PlanReduceRule;
3131
use crate::plan::physical::pipeline::Chain;
3232
use crate::plan::physical::pipeline::Compiler;
33-
use crate::plan::physical::plans::zoned::is_zone_proof;
33+
use crate::plan::physical::plans::zoned::is_zone_pruning;
3434

3535
const DATA: usize = 0;
3636
const ZONES: usize = 1;
@@ -176,7 +176,7 @@ impl PlanReduceRule<LegacyStats> for LegacyStatsRule {
176176
}
177177
}
178178

179-
/// Converts the legacy stats beneath a zone proof in place, so the proof meets a [`Zoned`] plan
179+
/// Converts the legacy stats beneath a zone pruning plan in place, so the plan meets a [`Zoned`] plan
180180
/// before the data path optimizes the stats away.
181181
#[derive(Debug)]
182182
pub(crate) struct ExpressionLegacyStatsRule;
@@ -190,7 +190,7 @@ impl PlanParentReduceRule<LegacyStats> for ExpressionLegacyStatsRule {
190190
parent: &Plan<Eval>,
191191
_child_idx: usize,
192192
) -> VortexResult<Option<PlanRef>> {
193-
if !is_zone_proof(parent)? {
193+
if !is_zone_pruning(parent)? {
194194
return Ok(None);
195195
}
196196
let Some(converted) = LegacyStatsRule.reduce(child)? else {

‎vortex-layout/src/plan/physical/plans/zoned.rs‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -455,7 +455,7 @@ impl PlanParentReduceRule<Zoned> for ExpressionZonedRule {
455455
parent: &Plan<Eval>,
456456
_child_idx: usize,
457457
) -> VortexResult<Option<PlanRef>> {
458-
if !is_zone_proof(parent)? {
458+
if !is_zone_pruning(parent)? {
459459
return Ok(None);
460460
}
461461
Ok(child
@@ -465,8 +465,8 @@ impl PlanParentReduceRule<Zoned> for ExpressionZonedRule {
465465
}
466466

467467
/// Whether `eval` is a boolean expression over abstract statistics alone, which a zone table can
468-
/// prove without reading the data it summarises.
469-
pub(crate) fn is_zone_proof(eval: &Plan<Eval>) -> VortexResult<bool> {
468+
/// prune without reading the data it summarises.
469+
pub(crate) fn is_zone_pruning(eval: &Plan<Eval>) -> VortexResult<bool> {
470470
let mut contains_stat = false;
471471
let mut contains_root = false;
472472
eval.expression().clone().transform_down(|expression| {
@@ -488,8 +488,8 @@ pub(crate) fn is_zone_proof(eval: &Plan<Eval>) -> VortexResult<bool> {
488488
}
489489

490490
/// Replaces a zoned data plan by its data: a scan reads the data through it and never its zone
491-
/// table. Pruning proofs are derived from the filter over the lowered plan before this runs, so
492-
/// an optimized plan carries a zone table only where a proof reads it.
491+
/// table. Pruning plans are derived from the filter over the lowered plan before this runs, so
492+
/// an optimized plan carries a zone table only where a pruning plan reads it.
493493
#[derive(Debug)]
494494
pub(crate) struct ZonedDataRule;
495495

‎vortex-layout/src/plan/physical/tests.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1589,7 +1589,7 @@ fn zoned_over_legacy_stats_read_as_data_keeps_only_the_data() -> VortexResult<()
15891589
}
15901590

15911591
/// Legacy stats over a zoned plan are dropped: read as data only the data remains, and a zone
1592-
/// proof is proved over the aggregate zones, never the legacy table.
1592+
/// pruning plan is evaluated over the aggregate zones, never the legacy table.
15931593
#[test]
15941594
fn legacy_stats_over_zoned_are_dropped() -> VortexResult<()> {
15951595
let dtype = primitive(PType::I32, Nullability::NonNullable);

‎vortex-layout/src/scan/execution/chunk.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ pub struct ChunkPlanner {
6666
rows: Range<u64>,
6767
chunks: IntoIter<Range<u64>>,
6868
selection: Selection,
69-
/// The rows zone pruning kept, over `rows`, or `None` when nothing was proved.
69+
/// The rows zone pruning kept, over `rows`, or `None` when nothing was pruned.
7070
kept: Option<Mask>,
7171
}
7272

‎vortex-layout/src/scan/execution/filter.rs‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ use crate::segments::SegmentId;
4747
pub struct FilterPlans {
4848
expression: FilterExpr,
4949
predicates: Vec<PlanRef>,
50-
/// A metadata-only proof per conjunct, where the conjunct has one.
50+
/// A metadata-only pruning plan per conjunct, where the conjunct has one.
5151
pruning: Vec<Option<PlanRef>>,
5252
}
5353

@@ -71,7 +71,7 @@ impl FilterPlans {
7171
.iter()
7272
.map(|expr| {
7373
expr.falsify(session)?
74-
.map(|proof| optimize(plan_row_idx_expression(proof, root.clone())?))
74+
.map(|pruning| optimize(plan_row_idx_expression(pruning, root.clone())?))
7575
.transpose()
7676
})
7777
.map(|plan| {
@@ -89,13 +89,13 @@ impl FilterPlans {
8989
})
9090
}
9191

92-
/// The pruning proofs, run once over a scan's rows before its chunks are planned.
93-
pub(crate) fn proofs(&self) -> impl Iterator<Item = &PlanRef> {
92+
/// The pruning plans, run once over a scan's rows before its chunks are planned.
93+
pub(crate) fn pruning(&self) -> impl Iterator<Item = &PlanRef> {
9494
self.pruning.iter().flatten()
9595
}
9696
}
9797

98-
/// A pruning proof must not fetch data when its requested statistics are unavailable.
98+
/// A pruning plan must not fetch data when its requested statistics are unavailable.
9999
fn metadata_only(plan: &PlanRef) -> VortexResult<bool> {
100100
if plan
101101
.as_opt::<Zoned>()

‎vortex-layout/src/scan/execution/mod.rs‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
//! The planners of a V2 scan, and the prepared plans they run.
55
//!
66
//! A [`PreparedScan`] holds a scan's plans over one file. An engine composes the planners over it
7-
//! and drives them: a [`PrunePlanner`] proves the filter's zone maps over the scan's rows and
7+
//! and drives them: a [`PrunePlanner`] evaluates the filter's pruning plans over the scan's rows and
88
//! emits the planner it was built to continue into, with the rows they keep; a [`ChunkPlanner`]
99
//! cuts those rows into chunks and spawns a [`FilterPlanner`] or a [`ProjectionPlanner`] per
1010
//! chunk; a filter planner runs its chunk's conjuncts and emits a projection planner for the rows
@@ -91,7 +91,7 @@ pub struct PreparedScan {
9191
pub(super) locations: Arc<[SegmentLocation]>,
9292
/// The optimized projection plan, over the scan's root row domain.
9393
pub(super) projection: PlanRef,
94-
/// The filter's conjuncts, their plans and their proofs.
94+
/// The filter's conjuncts, their plans and their pruning plans.
9595
pub(super) filter: Option<Arc<FilterPlans>>,
9696
/// The global row index of the root row domain's first row.
9797
pub(super) row_offset: u64,

‎vortex-layout/src/scan/execution/prune.rs‎

Lines changed: 27 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
// SPDX-License-Identifier: Apache-2.0
22
// SPDX-FileCopyrightText: Copyright the Vortex contributors
33

4-
//! Proves a filter's zone maps once over a scan's rows, before any chunk is planned.
4+
//! Evaluates a filter's pruning plans once over a scan's rows, before any chunk is planned.
55
66
use std::ops::Range;
77
use std::sync::Arc;
@@ -31,24 +31,24 @@ use crate::plan::physical::PlanRef;
3131
use crate::plan::physical::pipeline::Scan;
3232
use crate::plan::physical::pipeline::Split;
3333

34-
/// The planner a prune planner continues into, given the rows the proofs kept, or `None` when
35-
/// no proof ran.
34+
/// The planner a prune planner continues into, given the rows pruning kept, or `None` when
35+
/// no pruning plan ran.
3636
pub type PruneNext = Box<dyn FnOnce(Option<Mask>) -> Box<dyn Planner> + Send>;
3737

38-
/// Runs the filter's zone-map proofs over the scan's rows and emits the planner it was built
39-
/// to continue into, with the rows no proof excludes. The zone maps are the only reads,
40-
/// started together. Rows every proof excludes end the scan here: nothing is emitted. A filter
41-
/// without proofs emits the next planner at once.
38+
/// Evaluates the filter's pruning plans over the scan's rows and emits the planner it was
39+
/// built to continue into, with the rows no plan pruned. The zone maps are the only reads,
40+
/// started together. Rows every plan prunes end the scan here: nothing is emitted. A filter
41+
/// without pruning plans emits the next planner at once.
4242
pub struct PrunePlanner {
4343
scan: Arc<PreparedScan>,
4444
rows: Range<u64>,
45-
/// What to emit once every proof has run, taken when it is.
45+
/// What to emit once every pruning plan has run, taken when it is.
4646
next: Option<PruneNext>,
47-
/// Whether any proof ran, so the kept rows mean something.
48-
proved: bool,
49-
/// The rows no proof has excluded yet, over `rows`.
47+
/// Whether any pruning plan ran, so the kept rows mean something.
48+
pruned: bool,
49+
/// The rows no pruning plan has excluded yet, over `rows`.
5050
mask: Mask,
51-
proofs: IntoIter<PlanRef>,
51+
pruning: IntoIter<PlanRef>,
5252
active: Option<ScanAdaptor>,
5353
pieces: Vec<Mask>,
5454
announced: bool,
@@ -58,21 +58,21 @@ pub struct PrunePlanner {
5858
}
5959

6060
impl PrunePlanner {
61-
/// The planner of the proofs of `scan`'s filter over `rows`, continuing into `next`.
61+
/// The planner of `scan`'s filter's pruning plans over `rows`, continuing into `next`.
6262
pub fn new(scan: Arc<PreparedScan>, rows: Range<u64>, next: PruneNext) -> Self {
63-
let proofs: Vec<_> = scan
63+
let pruning: Vec<_> = scan
6464
.filter
6565
.iter()
66-
.flat_map(|filter| filter.proofs().cloned())
66+
.flat_map(|filter| filter.pruning().cloned())
6767
.collect();
6868
let len = usize::try_from(rows.end - rows.start).unwrap_or(usize::MAX);
6969
Self {
7070
scan,
7171
rows,
7272
next: Some(next),
73-
proved: false,
73+
pruned: false,
7474
mask: Mask::new_true(len),
75-
proofs: proofs.into_iter(),
75+
pruning: pruning.into_iter(),
7676
active: None,
7777
pieces: Vec::new(),
7878
announced: false,
@@ -107,11 +107,11 @@ impl Planner for PrunePlanner {
107107
}
108108
if !self.announced {
109109
self.announced = true;
110-
// A proof whose zones an earlier scan of the file proved compiles to its kept
111-
// result and lists no segment, so only unproved proofs prefetch their zone maps.
110+
// A pruning plan whose zones an earlier scan of the file pruned compiles to its kept
111+
// result and lists no segment, so only unpruned plans prefetch their zone maps.
112112
let segments = segments_read(
113113
&self.scan.session,
114-
self.proofs.as_slice().iter(),
114+
self.pruning.as_slice().iter(),
115115
self.rows.clone(),
116116
&self.mask,
117117
)?;
@@ -149,20 +149,20 @@ impl Planner for PrunePlanner {
149149
row_end = self.rows.end,
150150
input_rows = input,
151151
output_rows = self.mask.true_count(),
152-
"v2 zone proof evaluated"
152+
"v2 pruning plan evaluated"
153153
);
154154
self.active = None;
155155
self.pieces.clear();
156-
self.proved = true;
156+
self.pruned = true;
157157
}
158158
if self.mask.all_false() {
159159
self.done = true;
160160
return Ok(PlannerOutput::Done);
161161
}
162-
if let Some(proof) = self.proofs.next() {
162+
if let Some(plan) = self.pruning.next() {
163163
let scan = Scan::try_new(
164164
self.scan.session.clone(),
165-
proof,
165+
plan,
166166
vec![Split {
167167
rows: self.rows.clone(),
168168
mask: self.mask.clone(),
@@ -173,20 +173,20 @@ impl Planner for PrunePlanner {
173173
self.active = Some(ScanAdaptor::new(scan, Arc::clone(&self.scan.locations)));
174174
return Ok(PlannerOutput::Continue);
175175
}
176-
// Every proof has run: the planner this one continues into gets the kept rows.
176+
// Every pruning plan has run: the planner this one continues into gets the kept rows.
177177
self.done = true;
178178
tracing::debug!(
179179
row_start = self.rows.start,
180180
row_end = self.rows.end,
181181
kept_rows = self.mask.true_count(),
182-
"v2 zone proofs finished"
182+
"v2 pruning finished"
183183
);
184184
let next = self
185185
.next
186186
.take()
187187
.ok_or_else(|| vortex_err!("the prune planner has nothing to continue into"))?;
188188
let kept = self
189-
.proved
189+
.pruned
190190
.then(|| std::mem::replace(&mut self.mask, Mask::new_true(0)));
191191
Ok(PlannerOutput::Planner(
192192
WorkScope {

‎vortex-layout/src/scan/v2/cache.rs‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,8 @@ use crate::scan::execution::FilterPlans;
2323
/// over it, indexes every segment's byte range, and decodes the dictionary values it needs. A scan
2424
/// that finds an entry here reuses it, and one that misses builds the entry and keeps it for later
2525
/// scans. The lowered plan keeps the children it has initialized, each distinct projection keeps
26-
/// its optimized plan, each distinct filter keeps its conjunct and zone-proof plans along with the
27-
/// zones those proofs have already excluded, and dictionary values stay keyed by the cached plan
26+
/// its optimized plan, each distinct filter keeps its conjunct and pruning plans along with the
27+
/// zones those plans have already excluded, and dictionary values stay keyed by the cached plan
2828
/// that produced them.
2929
///
3030
/// A cache describes one file's layout, segment locations, and segment source. Use a new cache
@@ -40,8 +40,8 @@ struct Inner {
4040
/// them, so a projection must reuse its plan for its values to be found again.
4141
projections: Mutex<FxHashMap<ExactBoundExpr, PlanRef>>,
4242
ranges: Mutex<Option<SegmentRanges>>,
43-
/// Plans of filters, keyed by the exact filter expression. A proof plan remembers the zones it
44-
/// excluded, so sharing the plans proves a file's zones once for every scan of that filter.
43+
/// Plans of filters, keyed by the exact filter expression. A pruning plan remembers the zones it
44+
/// excluded, so sharing the plans prunes a file's zones once for every scan of that filter.
4545
filters: Mutex<FxHashMap<ExactBoundExpr, Arc<FilterPlans>>>,
4646
values: ValueCache,
4747
}

0 commit comments

Comments
 (0)