Repository navigation
Enable udf hashing across FFI boundary #17087
Description
Is your feature request related to a problem or challenge?
This is a follow on to #17067
Right now the comparison for UDFs to see if they are equal is relying on checking pointers instead of using the hash function. For UDF, UDAF, and UDWF we should add a hash function to the FFI interface and call across it.
Describe the solution you'd like
Add hash() to FFI definitions for UDF, UDAF, UDWF.
Use this hash call to check for equality.
Add user tests to validate.
Describe alternatives you've considered
None
Additional context
No response
Reacted by Andrew Lamb
Activity
crystalxyz commentedon Aug 29, 2025
Hi @timsaucer, I'm returning to the community and refreshing my memory on FFI! I've just implemented a quick fix for this issue, and please let me know if this is what you had in mind.
timsaucer commentedon Aug 30, 2025
Thank you! I'll take a look right now, and welcome back!
findepi commentedon Sep 9, 2025
I agree it's better to implement value-based equality also for FFI-wrapped functions rather than have pointer-equality-only based one.
However, without equivalent of Rust's
Any, it's not clear how an FFI function would implement equals correctly.Is comparing the pointers only really a big problem?
Maybe let's focus on lifecycles of those pointers. The
FFI_ScalarUDFis not instantiated in DataFusion code.These might be registered once with the session context, are effectively singletons. If this is the case, pointer-based eq is sufficient for most cases.
If this is not the case, maybe let's start from describing the scenario, i.e. what problem are we truing to solve.