[SPARK-2235][SQL]Spark SQL basicOperator add Intersect operator #1150
Conversation
Hi all, I want to submit a basic operator Intersect For example , in sql case select * from table1 intersect select * from table2 So ,i want use this operator support this function in Spark SQL This operator will return the the intersection of SparkPlan child table RDD .
|
Can one of the admins verify this patch? |
YanjieGao
changed the title
|
hi all ,I have finished the INTERSECT operator ,and has update the 6 files this operator needed . |
|
Hi all, I have resolved the conflict . |
|
This is conflicting now that we merged the other PR. test this please |
There was a problem hiding this comment.
Does hive support this? if so we should add a test to HiveQuerySuite too.
There was a problem hiding this comment.
Hive doesn't support ,I will remove this
There was a problem hiding this comment.
I don't know why this line always red
There was a problem hiding this comment.
Hi, it says you deleted some whitespace after Nil.
|
Hi Michael,I have resolve the conflict .And modify as your suggestion |
|
Thanks ,I add three blank after that line ,Now there aren't red lines. |
If there is trailing whitespace we should remove it. It is okay if it is not code you added in the PR. |
|
Thanks a lot, I have reformat the code style. |
|
Thanks, I've merged this into master. Did not apply cleanly to 1.0 so we'll backport it later. |
|
Hi, Michael I also have a Skew Join pr need to be reviewed and need some suggestions .Could you help me review it ?I have test it can pass the testsuite .Thanks a lot |
Hi all, I want to submit a basic operator Intersect For example , in sql case select * from table1 intersect select * from table2 So ,i want use this operator support this function in Spark SQL This operator will return the the intersection of SparkPlan child table RDD . JIRA:https://issues.apache.org/jira/browse/SPARK-2235 Author: Yanjie Gao <[email protected]> Author: YanjieGao <[email protected]> Closes apache#1150 from YanjieGao/patch-5 and squashes the following commits: 4629afe [YanjieGao] reformat the code bdc2ac0 [YanjieGao] reformat the code as Michael's suggestion 3b29ad6 [YanjieGao] Merge remote branch 'upstream/master' into patch-5 1cfbfe6 [YanjieGao] refomat some files ea78f33 [YanjieGao] resolve conflict and add annotation on basicOperator and remove HiveQl 0c7cca5 [YanjieGao] modify format problem a802ca8 [YanjieGao] Merge remote branch 'upstream/master' into patch-5 5e374c7 [YanjieGao] resolve conflict in SparkStrategies and basicOperator f7961f6 [Yanjie Gao] update the line less than bdc4a05 [Yanjie Gao] Update basicOperators.scala 0b49837 [Yanjie Gao] delete the annotation f1288b4 [Yanjie Gao] delete annotation e2b64be [Yanjie Gao] Update basicOperators.scala 4dd453e [Yanjie Gao] Update SQLQuerySuite.scala 790765d [Yanjie Gao] Update SparkStrategies.scala ac73e60 [Yanjie Gao] Update basicOperators.scala d4ac5e5 [Yanjie Gao] Update HiveQl.scala 61e88e7 [Yanjie Gao] Update SqlParser.scala 469f099 [Yanjie Gao] Update basicOperators.scala e5bff61 [Yanjie Gao] Spark SQL basicOperator add Intersect operator
…s" message appears when spark session is starting (apache#1150)
…s" message appears when spark session is starting (apache#1150)
4 participants
Hi all,
I want to submit a basic operator Intersect
For example , in sql case
select * from table1
intersect
select * from table2
So ,i want use this operator support this function in Spark SQL
This operator will return the the intersection of SparkPlan child table RDD .
JIRA:https://issues.apache.org/jira/browse/SPARK-2235