-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Closed as not planned
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
After #256 we completely ignore NaNs in parquet statistics. However, there are good reasons to fully order floats and include NaN somewhere:
- Databases usually need it, see sql: NaN = NaN (in Postgres) cockroachdb/cockroach#18860 and if DataFusion wants to follow the Postgres dialect we will have the same issue there
- Other parquet users are interested as well: PARQUET-1222
- Just ignoring NaN is kinda having an implicit NULL for float while we already have a NULL-mask
Describe the solution you'd like
Put NaNs at the end of the float scale, so the order is:
- -inf
- "ordinary" numbers
- +inf
- NaN
Describe alternatives you've considered
Keeping the status quo (aka current master).
Additional context
.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels