The `lance.partition.columns` write contract in lance-format#445 declares
`Distributions.clustered` plus a sort on the partition columns, intending
that each Lance fragment hold exactly one partition value (SPJ relies on
this). `ClusteredDistribution` only guarantees "same value → same task",
not "one value per task" — hash collisions (distinct values > shuffle
partitions) pack multiple partition values into one task, and
`Fragment.create` rolls only on `maxRowsPerFile`/`maxBytesPerFile`, so a
fragment could still span multiple values.

Thread the partition columns through `SparkWrite` → `LanceBatchWrite` →
`WriterFactory` → `LanceDataWriter`. In the sorted input, detect
partition-key transitions and roll a fresh `ArrowBatchWriteBuffer` +
`Fragment.create` task at each boundary. The existing streaming
producer/consumer buffer is reused per group — groups still stream
through rather than being materialized in memory.

Adds `BasePartitionedWriteTest.testHashCollisionsBreakSinglePartitionPerFragment`
which forces `spark.sql.shuffle.partitions=2` with 20 distinct region
values so collisions are guaranteed, and asserts one region per fragment.
Also adds the missing concrete `PartitionedWriteTest` subclass in
`lance-spark-3.5_2.12` — without it, the abstract base tests from lance-format#445
never actually executed.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

The remaining testHashCollisionsBreakSinglePartitionPerFragment strictly
subsumes it: if one-value-per-fragment holds under forced hash collisions
(2 shuffle partitions, 20 regions), it trivially holds when the default
shuffle cleanly separates values. The removed test never actually
exercised the fragment-rolling logic — it passed only because 200 default
buckets happened to spread 5 regions without collisions.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

Compare partition-column values against the stored key in-place; only clone
on fragment boundaries. Eliminates the per-row Object[] + UTF8String.clone
cost in the writer hot path.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>