Repository navigation
Commit 984f55e
contrib(delta) P7f: 4.1.0.diff hunk for DeltaSuite partition-pruning test
Patch `DeltaSuite "query with predicates should skip partitions"` to
also recognise `CometDeltaNativeScanExec` in its
`executedPlan.collect { case f: FileSourceScanExec => f }`. Comet's
planner replaces the underlying `FileSourceScanExec` with the contrib's
native exec; without this hunk the collect returns an empty list and
the test fails at `assert(fileScans.size == 1)`.
The contrib's exec exposes the same `numFiles` metric (aliased to
`total_files` over the post-prunePartitions task list), so once the
test sees the exec the existing assertion on `numFiles == 1` validates
that driver-side partition pruning is actually filtering AddFiles.
Closes tasks apache#124 (partition pushdown validated) and apache#125 (data-stats
pushdown validated via `DataSkippingDeltaV1Suite "loading data from
Delta to parquet should skip data"`).
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>1 parent abd9d95 commit 984f55e
1 file changed
Lines changed: 15 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
215 | 215 | | |
216 | 216 | | |
217 | 217 | | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
0 commit comments