Reject nullable nested levels in list-based spatial types - #11183

TS (#10953) merged on 2026-11-03, and SN1 (#11183) and V10 (#11187)
merged on 2026-11-04. The dependency graph marks the three merged and
moves its state date to 2026-11-04. It gives SD, DP, FD and U9 their
open PRs (#11184, #11185, #11186 and #11188), and it drops #10408,
which closed without a merge. The text of parts 4 and 5 and the Fix
lines of EXT-V9, EXT-V10, EXT-N2 and EXT-S3 now record the same state.

Also record that AX, DS and SN2 are implemented on branches without a
PR, and add the approved one-pass design for the masked nulls of SN2.

Part 2 and decision X31 also record the fixes: #10953 made every i64
legal, #11183 made the spatial validators reject nullable nested
levels, #11187 gave the Date display its fallback, and #10408 closed.

Signed-off-by: Connor Tsui <[email protected]>

## Summary

- Tracking Issue: #11114

Right now, an extension type validates whatever storage dtype the caller
hands it, so one type can have several storage forms, and its metadata
doesn't say which one a dtype uses. Each `ExtVTable` now declares its
storage from its metadata, so the metadata **deterministically fixes /
decides** the whole storage dtype except the top-level nullability.

Reads then require the stored storage to equal that declaration exactly,
**including nested nullability**. This has to be strict, since arrays
decode their storage child with the declared dtype, so a stored form
that differs would be misread instead of rejected. Every built-in type
accepts the same stored forms as before, so existing files still read.

_NB on nested nullability: this means that if we potentially need nulls
in the storage, an extension type MUST require its storage type to be
nullable, even if in some cases it might not be. I think this is a
worthwhile tradeoff for clarity of implementation._

## Changes

Some facts lived only in the storage dtype, like the element type of
`Vector` and `FixedShapeTensor` and the dimension of the native spatial
types, so they move into the in-memory `Metadata`.
`deserialize_metadata` decodes `StoredMetadata` (which is the old
`Metadata`, and `metadata_from_storage` combines it with storage to
rebuild complete metadata. The serialized bytes don't change.

Some other things:
- File reads and storage-based dtype, array, and scalar constructors
share the same checked path.
- `ForeignExtDType` can't derive its storage, since it doesn't know its
type, so it keeps the stored storage in its metadata and declares that.
- The nested-nullability checks that #11183 added to the spatial helpers
go away, since the read check now covers them.

Follow-up: a read error in a struct field still panics on first access
(SF in #11114).

## API Changes

- `ExtVTable::validate_dtype` becomes `storage_dtype` (plus an optional
`validate_metadata`), `deserialize_metadata` returns `StoredMetadata`,
and `metadata_from_storage` reconstructs complete metadata from stored
metadata and storage.
- `ExtDType::try_new` and `try_with_vtable` take a nullability in place
of the storage dtype.
- Reading an extension dtype whose stored storage differs from its
declaration fails, and so does building a scalar or array over the wrong
storage.
- `FixedShapeTensorMetadata::new` takes the element type, `Vector`'s
metadata is `VectorMetadata`, metadata-first construction of native
spatial dtypes needs `SpatialMetadata::with_dimension`, and
`SpatialMetadata` no longer implements `prost::Message`.
- `ExtDType` and `ExtensionArray` gain `try_from_storage` constructors,
and `Scalar` gains `try_extension_from_storage`, with explicit-vtable
variants. Existing metadata-first constructors remain available.
- `FixedShapeTensorStoredMetadata` supplies shape and layout, and
`SpatialStoredMetadata` supplies CRS when constructing from storage.
- Fallible constructors reject non-nullable null storage, including
extensions over null storage, with an error.

---------

Signed-off-by: Connor Tsui <[email protected]>