-
Notifications
You must be signed in to change notification settings - Fork 2k
Description
Is your feature request related to a problem or challenge?
Given the importantance of spark functions in general, we are consolidating / adding spark compatible functions in the datafusion-spark crate in DataFusion: https://github.com/apache/datafusion/tree/main/datafusion/spark
This ticket is part of a series of ticket to fill out the set of spark compatible functions in DataFusion
Describe the solution you'd like
Implement the function listed above in the datafusion-spark crate
Describe alternatives you've considered
- Add the relevant function here: https://github.com/apache/datafusion/tree/main/datafusion/spark/src/function/math
- Add tests. Instructions are here: https://github.com/apache/datafusion/blob/main/datafusion/spark/README.md
Examples to follow:
- The existing
expm1implementation : https://github.com/apache/datafusion/blob/main/datafusion/spark/src/function/math/expm1.rs - COmet implementation here: https://github.com/apache/datafusion-comet/blob/main/native/spark-expr/src/math_funcs
- Sail implementation here: https://github.com/lakehq/sail/blob/4144d9a53d3f1ef447b3d2ce9afa9030906de640/crates/sail-plan/src/function/scalar/math.rs#L404
You can also find the corresponding datafusion implementations in the corresponding file here: https://github.com/apache/datafusion/tree/main/datafusion/functions/src (it would be ok to call into these implementations if they are the same as the spark implementation)
Additional context
No response