fix(serialization): keep non-finite numpy and pandas floats JSON-compliant - #15229

Changes

serialize() replaces non-finite floats with None — that is the invariant its own tests
document (test_nan_float_serialized_to_none, test_inf_float_serialized_to_none,
test_nan_inside_dict_serialized_to_none) and the one src/backend/tests/unit/test_messages.py:647
relies on when it asserts json.dumps(..., allow_nan=False) on persisted output.

The check lives in _serialize_primitive, which only matches Python float, so the two paths
that convert without going back through it kept leaking:

• _serialize_numpy_type returned obj.item() / obj.tolist() as they are. np.float64
  happens to be a float subclass and is caught upstream, but np.float32 and np.float16
  are not, so the same value in a narrower dtype came out as nan / inf.
• _serialize_series reads values straight off the Series and only truncates them, so it
  never reaches the dispatcher — a plain float64 Series with one missing value leaked.

This adds _sanitize_non_finite() and applies it to those two returns in both mirrored copies
(langflow.serialization.serialization and lfx.serialization.serialization). It recurses so
array and list-valued entries are covered, and touches nothing that is not a float:

float('nan')                     -> None          (unchanged)
np.float64(nan)                  -> None          (unchanged)
np.float32(nan)                  -> nan     -> None
np.float16(inf)                  -> inf     -> None
np.array([1.0, nan], f32)        -> [1.0, nan]  -> [1.0, None]
pd.Series([1.0, nan])            -> {0: 1.0, 1: nan} -> {0: 1.0, 1: None}
json.dumps(payload)              -> {"score": NaN, "vec": [1.0, Infinity]} -> {"score": null, "vec": [1.0, null]}

_serialize_dataframe re-enters serialize(), so a float64 column was already clean — but a
narrower column reaches it as np.float32 and leaked the same way; both cases are pinned by the
new tests. Finite output is unchanged, including dtypes and truncation behaviour.

Left deliberately out of scope: _serialize_decimal has the same gap for Decimal("NaN") /
Decimal("Infinity"), but that is what the open #11648 proposes, so this PR stays on the
numpy/pandas paths that PR does not reach.

Fixes #15228

How was this tested?

Added TestNonFiniteNumpySerialization to
src/backend/tests/unit/serialization/test_serialization.py. It is parametrized over
[langflow, lfx] so both mirrored copies are pinned to the same behaviour — today neither copy
has any test coverage of its own for this, and the langflow copy is the only one the existing
tests exercise.

Against main @ 5621dcfd with only the new tests added:

14 failed, 41 passed in 14.79s

Nine cases × two implementations = 18 new tests. The four that already pass on main are
test_nan_in_float64_dataframe_serialized_to_none and test_finite_values_are_preserved, which
must pass before and after. With the serializer change in place:

55 passed, 1 warning in 9.83s

Regression checks on the consumers I could reach locally:

• src/backend/tests/unit/test_messages.py — 1 failed, 82 passed. The failure is
  test_aget_messages, and it fails identically on pristine main with the serializer
  untouched (verified by reverting both files, running it, and restoring), so it is not caused
  by this diff.
• src/backend/tests/unit/services/tracing/test_langfuse_v3_compatibility.py and
  src/backend/tests/unit/components/files_ingestion/test_file_content_retriever.py —
  80 passed, 4 skipped.
• ruff format --check and ruff check pass on the three changed files.

How can this PR be tested?

uv run --no-sync python -m pytest src/backend/tests/unit/serialization/test_serialization.py -q

Checklist

• Contributor: I am the agent account Lesereingrape; these changes were produced by an automated contributor and have not been read line-by-line by a human, so the reviewer box in the contributing guide is deliberately left unticked.
• I confirm that you have tested the changes and verified that they work and don't break anything (as well as I can manage)
• The title follows semantic commit rules

Summary by CodeRabbit

• Bug Fixes
  • Improved serialization of NumPy and pandas data containing NaN or infinite values.
  • Non-finite numeric values are now converted to null, ensuring serialized output remains valid JSON.
  • Applied consistently across scalar values, arrays, dictionaries, Series, and DataFrames.