Support nested ROW predicates in native table reads - #1153

Purpose

PyPaimon already supports scalar filters over flattened nested ROW projections, such as projecting profile.detail.score and filtering profile_detail_score = 7. The Native path currently cannot resolve that leaf and falls back to Python.

Add nested ROW predicates to the core read pipeline, following the current Java NestedFieldTransform: retain the enclosing root as an input, resolve separate path components with stable field IDs, mask NULL ancestors, and avoid interpreting parent-column statistics or indexes as child metadata.

Brief change log

• Add reusable PredicateBuilder::field / nested_field inputs and a distinct Predicate::NestedLeaf; retain the original field identities through physical-file remapping. Ordinary comparison, null, set, string, array and range helpers accept either column names or resolved FieldRef values.
• Reuse exact scalar residual evaluation for nested comparisons, ranges, membership and string predicates, including parent-null masking and promoted fields.
• Widen predicate ROW inputs and restore the requested nested projection after filtering in ordinary reads, schema evolution, row tracking, Data Evolution and managed BLOB reads.
• Keep partition, bucket, file/global-index and parent-statistics pruning conservative for nested leaves; preserve exact filtering after primary-key merge.
• Resolve optional nested_path in the Python predicate binding through the core builder and reuse existing typed literal conversion.

Tests

• Reproduced the baseline Native fallback while the existing Python read returned the matching row.
• Core: 112 tests selected by nested_, 271 by predicate, 86 DataEvolutionReader tests, 38 DataFileReader tests, and 7 managed BLOB reader tests passed (these groups overlap).
• Python/Native integration on the built extension: 998 passed, 2 skipped, 35 subtests passed. Assertions reject Python scan/read fallback; cover append/PK, batch/stream, NULL parents, scalar operators, projection changes, field promotion/rename/drop-readd, aliases colliding with real columns, partition/bucket pruning and Boolean composition.
• Seven focused API contract tests and the externally compiled public API example passed. The Python binding filter suite passed all 69 tests.
• cargo +1.98.0 clippy --locked -p paimon -p pypaimon_rust --lib --tests -- -D warnings and formatting checks passed.

API and Format

Field selection is independent of comparison construction, like Java's transform inputs. Existing column-name calls remain supported:

let score = builder.nested_field("profile", &["detail", "score"])?;
let predicate = builder.equal(&score, Datum::Int(7))?;
let range = builder.between(&score, Datum::Int(1), Datum::Int(9))?;

The former nested(root, path, op, literals) entry point is removed. Immutable field references validate their root position, field identities and type against the receiving builder before constant folding. The Python binding resolves the same core references and calls ordinary operators, without synthetic schemas or recursive rebinding.

New core FieldRef and PredicateField input types and a nested predicate variant; no storage format changes. The Python filter dictionary accepts an optional nested_path list alongside the physical root field.

Documentation

API documentation describes path components, field identity and NULL-parent semantics. Companion PyPaimon PR: apache/paimon#10637. It retains physical projection paths and adds Native regression coverage; its CI continues building Rust main.

Review follow-up

• Restore nested output fields after authorized ordinary reads and descriptor BLOB reads, rather than dropping only top-level authorization columns.
• Restore the actual nested input types before managed BLOB payload filtering too. Keep name-based column gathering for non-prefix layouts such as [profile, auth_id, payload] to [profile, payload].
• Add real Parquet regressions covering seven authorization/BLOB/masking modes with and without row kind, plus a helper regression for NULL parents, reordered child fields, hidden columns and BLOB data markers. table::table_read::tests: 31 passed. Binding filter tests: 69 passed. Final core/binding Clippy 1.98.0 and formatting passed.
• Repeated independent reviews of the API, read pipeline and Python adapter found no further actionable issues within this change's scope after these fixes.
• Final integration against the completed companion extension: 1637 passed, 2 skipped, 39 subtests passed across 23 related Python suites, with 1355 Native plans and 1595 Native reads exercised.