You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 31cb93f
Browse filesBrowse the repository at this point in the historyBrowse files
## Which issue does this PR close?
- Backport of #26310 to `branch-54`
- Part of #23805
## Rationale for this change
Hive writers use `__HIVE_DEFAULT_PARTITION__` for NULL partition values.
`ListingTable` currently parses this marker as the partition column's
type, causing cast errors for non-string columns and returning the
literal marker for string columns. `IS NULL` consequently misses those
rows.
## What changes are included in this PR?
- Read the Hive marker as a typed NULL and declare partition columns
nullable.
- Treat NULL partition-predicate results as non-matches and account for
NULL partition values in file statistics.
- Reject partitioned writes of the literal marker, which would otherwise
read back as NULL.
This backport uses a fixed marker, without the new execution option or
public constant. Ordering tests reflect branch-54's existing
explicit-sort fallback.
## Are these changes tested?
Yes
## Are there any user-facing changes?
Yes. Hive marker directories yield NULL values, and partition columns
are nullable. Writing the literal marker returns an error. The marker
cannot be configured or disabled in this backport.
AI assistance: Codex.
---------
Co-authored-by: Xander <[email protected]>
0 commit comments