[branch-56] IN LIST: treat signed zeros as equal (backport #25186) - #27204

## Which issue does this PR close?

- Part of apache#26138 (shared floating-point semantics).
- Covers the IN-list regression reported in apache#27121.
- Closes apache#27121
- Closes apache#27122

## Rationale for this change

DataFusion equality treats `+0.0` and `-0.0` as equal, but the optimized
`IN` filters compare or hash floating-point bit patterns. Because the
two zeros have different encodings, an `IN` predicate could therefore
return a different result depending on whether the optimizer retained it
or rewrote it into equality comparisons.

This signed-zero limitation predates the IN-list optimization epic
apache#19241: [apache#7378](apache#7378) already
used bit-pattern equality for floating-point membership in August 2023.
[apache#22835](apache#22835) corrected
ordinary comparisons in June 2026 but did not fix `IN` membership. This
PR addresses that remaining correctness gap.

This PR makes `Float16`, `Float32`, and `Float64` `IN` predicates,
including floats inside nested and encoded values, follow the same
signed-zero equality as ordinary comparisons. The shared static-filter
factory adds the missing zero encoding before choosing a primitive
filter. Strategy selection uses the expanded physical non-null list
length, so both encodings count toward the existing cutoffs. Generic
static filters and non-static lists use the shared `normalize_floats`
and recursive `ScalarValue::normalize` APIs before comparison. Distinct
NaN payloads remain distinct, preserving the existing bit-equality
behavior for NaNs.

## What changes are included in this PR?

- Add the missing signed-zero encoding once in the shared static-filter
factory, after flattening dictionary lists.
- Select branchless, bitmap, or hash-set lookup using the expanded
physical non-null key count.
- Extend `ScalarValue::normalize` to nested and encoded values, and use
`float_semantics::normalize_floats` for arrays in dynamic comparisons
and generic static filters.
- Prevent structural literal equality from driving IN-list set-operation
rewrites for types containing floating-point values.
- Preserve null behavior, dictionary inputs, `IN`, and `NOT IN`.

## What is the testing strategy for this PR?

SQL regressions in `negative_zero.slt` cover both signed-zero
directions, static filters, dynamic scalar and array operands, nested
values, IN-list intersection/difference, `IN`, `NOT IN`, and nulls.
Focused Rust tests cover dictionary inputs, distinct NaN payloads,
encoded scalar metadata, and sliced values. Factory tests exercise both
zero signs immediately below and at each floating-point branchless
cutoff, including dispatch after adding the missing physical key. SQL
tests also check constant nested scalar self-membership; an existing
Rust test covers nested dictionary scalar inputs.

Validation after rebasing onto `upstream/main` (`4bf32cb46d`):
formatting, CI Clippy with its standard feature set, all 111 focused
Rust tests, and all 546 sqllogictest files (including Parquet
encryption) passed.

## Are there any user-facing changes?

Yes. Floating-point `IN` and `NOT IN`, including nested and encoded
values, now treat `+0.0` and `-0.0` as equal, consistently with ordinary
DataFusion equality. `ScalarValue::normalize` now applies these rules
recursively; public API signatures are unchanged.

Prepared with AI assistance.

---------

Co-authored-by: Andrew Lamb <[email protected]>
(cherry picked from commit 94486b1)