[Spark] Distribute scans of a large Vortex file across Spark input partitions #2

A dataset containing only a few large Vortex files cannot distribute its scans across many Spark tasks/executors. Native work may parallelize inside an executor, but Spark does not split a file into independently scheduled input partitions.

Revalidated evidence

• Current connector: VortexBatchExec.planInputPartitions creates one input partition per path, with no row range or native split selection.
• Upstream draft PR #9658 replaces this planner with Spark's file-source machinery, which groups small files. It still explicitly returns false from VortexScan.isSplitable, as does its V1 file format. The large-file limitation remains.
• Source comparison against upstream develop at a1fdb64942a645ed1a84b34ccc72b05692a47154 found no Spark connector changes relative to the inspected checkout.

This is a performance enhancement supported by source inspection; no distributed speedup has been measured. No matching upstream issue or open PR for Spark large-file splitting was found when checking upstream Spark/split issues and all 187 open PRs on 2026-10-23. This issue tracks the residual large-file work, not the small-file grouping already addressed by vortex-data#9658.

Proposed scope

Investigate scheduling disjoint native partitions or row ranges from one file across Spark tasks. A byte-offset split alone is insufficient unless the reader can map it to valid Vortex scan work.

Completion criteria

• A sufficiently large single file can produce multiple independently scheduled Spark input partitions.
• Reads have no missing or duplicate rows, including with filters, projection, nulls and partition columns.
• Compare a few-large-files workload before/after, including planning overhead and executor utilization. Keep native parallelism and Spark task parallelism distinct in the results.