Skip to content

[MINOR][SQL][PYTHON] Rename Python plan node names - #44373

Closed
HyukjinKwon wants to merge 3 commits into
apache:masterfrom
HyukjinKwon:minor-arrow-rename
Closed

[MINOR][SQL][PYTHON] Rename Python plan node names#44373
HyukjinKwon wants to merge 3 commits into
apache:masterfrom
HyukjinKwon:minor-arrow-rename

Conversation

@HyukjinKwon

@HyukjinKwon HyukjinKwon commented Dec 15, 2023 •
edited
Loading

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

This PR is a sort of a followup of #38624 that proposes to rename the plan nodes for Python as below:

From:

package org.apache.spark.sql.execution.python

MapInBatchExec
├── MapInPandasExec
└── *PythonMapInArrowExec* (and *PythonMapInArrow*)

*FlatMapCoGroupsInPythonExec*
├── FlatMapCoGroupsInArrowExec
└── FlatMapCoGroupsInPandasExec

*FlatMapGroupsInPythonExec*
├── FlatMapGroupsInArrowExec
└── FlatMapGroupsInPandasExec

To:

package org.apache.spark.sql.execution.python

MapInBatchExec
├── MapInPandasExec
└── *MapInArrowExec* (and *MapInArrow*)

*FlatMapCoGroupsInBatchExec*
├── FlatMapCoGroupsInArrowExec
└── FlatMapCoGroupsInPandasExec

*FlatMapGroupsInBatchExec*
├── FlatMapGroupsInArrowExec
└── FlatMapGroupsInPandasExec

Why are the changes needed?

To have the consistent names for Python related execution nodes.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing CI should pass.

Was this patch authored or co-authored using generative AI tooling?

No.

@HyukjinKwon HyukjinKwon changed the title [MINOR][SQL][PYTHON] Rename Python physical plan node names [MINOR][SQL][PYTHON] Rename Python plan node names Dec 15, 2023
ueshin
ueshin approved these changes Dec 15, 2023
View reviewed changes

@ueshin ueshin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise, LGTM.

Comment thread sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala Resolved
allisonwang-db
allisonwang-db approved these changes Dec 16, 2023
View reviewed changes

@allisonwang-db allisonwang-db left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now the plan name makes more sense

zhengruifeng
zhengruifeng approved these changes Dec 16, 2023
View reviewed changes
Comment thread sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala Resolved
@HyukjinKwon

HyukjinKwon commented Dec 16, 2023

Copy link
Copy Markdown
Member Author

Merged to master.

@HyukjinKwon
HyukjinKwon deleted the minor-arrow-rename branch January 15, 2024 00:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ueshin ueshin ueshin approved these changes
@zhengruifeng zhengruifeng zhengruifeng approved these changes
@allisonwang-db allisonwang-db allisonwang-db approved these changes

Assignees

No one assigned

Labels

CONNECT PYTHON SQL

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@HyukjinKwon @ueshin @zhengruifeng @allisonwang-db