You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Filter manifest lists on pyiceberg-core Arrow output in plan_files #4299
Scan planning reads the manifest list into ManifestFile objects and then filters them by partition summaries (table/__init__.py#L2805-L2809). #4007 shows that pyiceberg-core parses a manifest list faster than Cython, but building the ManifestFile objects makes the end-to-end path slower.
Apply the approach from #4298 to read_manifest_list: read the list from pyiceberg-core as Arrow, and filter it on the Arrow columns so that only matching manifests become Python objects. This needs apache/iceberg-rust#3649. pyiceberg-core is an optional extra, so the Cython reader stays as the fallback.
Activity
mbutrovich commentedon Nov 6, 2026
take