You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Implement zero-copy FFI transfer of batches from JVM to Native #2171
Comet doesn't currently follow Arrow best practice when transferring batches from JVM to native code. Ownership is retained by the JVM after batches have been made available in FFI, requiring a copy to be made on the native side.
It would be better to pass ownership of batches to native rather than make copies.
This design was necessary originally due to the use of mutable Parquet buffers in the native_comet Parquet scan.
Describe the potential solution
Once #2152 is implemented, and once we have removed the native_comet scan (#2177) and all uses of mutable Parquet buffers, we should be able to implement zero-copy FFI.
Activity
jatin510 commentedon Aug 17, 2025
i would love to work on this issue @andygrove
andygrove commentedon Aug 17, 2025
Thanks @jatin510. I think the next steps are:
native_comet#2172 (in progress)native_cometscan implementation #2177native_iceberg_compat#2152andygrove commentedon May 26, 2026
this is implemented now