Feat: Support array_join function#1290
Conversation
erenavsarogullari
force-pushed
the
array_join
branch
from
b98415e to
5d3c94d
Compare
native/proto/src/proto/expr.proto
Outdated
| ArrayInsert array_insert = 59; | ||
| BinaryExpr array_contains = 60; | ||
| BinaryExpr array_remove = 61; | ||
| BinaryExpr array_join = 63; |
There was a problem hiding this comment.
I have another PR for array_intersect and it uses 62. I think array_intersect can go first.
|
I think the handling of the third argument (nullReplacement) is missing see: https://docs.databricks.com/en/sql/language-manual/functions/array_join.html#arguments. I think this will also require a new expression type in expr.proto |
Thanks - That is right both Spark and DataFusion support optional |
erenavsarogullari
force-pushed
the
array_join
branch
2 times, most recently
from
a5e54c3 to
c5f8e7e
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1290 +/- ##
============================================
+ Coverage 34.69% 34.71% +0.01%
- Complexity 991 992 +1
============================================
Files 116 117 +1
Lines 44891 45148 +257
Branches 9864 9956 +92
============================================
+ Hits 15574 15671 +97
- Misses 26168 26310 +142
- Partials 3149 3167 +18 ☔ View full report in Codecov by Sentry. |
erenavsarogullari
force-pushed
the
array_join
branch
from
c5f8e7e to
895e701
Compare
|
lgtm 👍 |
erenavsarogullari
changed the title
|
@erenavsarogullari could you fix the conflicts on this PR? |
|
Thanks @andygrove. Sure, i will have a look in today. |
andygrove
left a comment
andygrove
left a comment
There was a problem hiding this comment.
LGTM. Thanks @erenavsarogullari.
erenavsarogullari
force-pushed
the
array_join
branch
from
895e701 to
7824059
Compare
6 participants
Which issue does this PR close?
Related to Epic: #1042
array_join:select array_join(array('hello', '-', 'world'), ' ')=>hello - worldDataFusion' s
array_joinfunction is alias of array_to_string function:https://datafusion.apache.org/user-guide/sql/scalar_functions.html#array-join
Rationale for this change
Defined under Epic: #1042
What changes are included in this PR?
planner.rs:Created DataFusionarray_joinphysical expression from Spark physical expression with return type:DataType::Utf8,expr.proto:array_joinmessage has been added (array_intersect is 62),QueryPlanSerde.scala:array_joinpattern matching case has been added,CometExpressionSuite.scala:A new UT has been added forarray_joinfunction.How are these changes tested?
A new UT has been added and first query result is as follows:
