GH-36928: [Java] Make it run well with the netty newest version 4.1.96 by panbingkun · Pull Request #36926 · apache/arrow

panbingkun · 2023-07-28T11:43:16Z

When I used netty arrow memory 13.0.0 and netty 4.1.96.Final in Spark, the following error occurred,
Because netty 4.1.96.Final version has revert some modifications, in order to ensure that netty arrow memory 13.0.0 works well with netty 4.1.96.Final version, I suggest making similar modifications here.
1.Compilation errors are as follows:
https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/builds/47657403

2.Some modifications have been reverted in netty 4.1.96.Final as follows:

https://netty.io/news/2023/07/27/4-1-96-Final.html
netty/netty#13510

Closes: [Java] Make it run well with the netty newest version 4.1.96 #36928

github-actions · 2023-07-28T11:43:42Z

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

In the case of PARQUET issues on JIRA the title also supports:

PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

See also:

panbingkun · 2023-07-28T11:52:50Z

When I used netty arrow memory 13.0.0 and netty 4.1.96.Final in Spark, the following error occurred,
Because netty 4.1.96.Final version has revert some modifications, in order to ensure that netty arrow memory 13.0.0 works well with netty 4.1.96.Final version, I suggest making similar modifications here.
1.Compilation errors are as follows:
https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/builds/47657403

2.Some modifications have been reverted in netty 4.1.96.Final as follows:

https://netty.io/news/2023/07/27/4-1-96-Final.html
netty/netty#13510

github-actions · 2023-07-28T11:56:24Z

⚠️ GitHub issue #36928 has been automatically assigned in GitHub to PR creator.

lidavidm

Thank you for catching this!

@raulcd since we don't have RC2 out yet, would it be possible to cherry-pick this too? It also apparently avoids a potential performance regression in Flight

raulcd · 2023-07-28T14:29:11Z

Sure! that sounds good to me!

lidavidm · 2023-07-28T14:48:14Z

Thanks Raúl! I added this to the 13.0.0 milestone, I'll go update the blog post too

raulcd · 2023-07-28T14:49:43Z

I added this to the 13.0.0 milestone

cherry-picked to maint-13.0.0

#36926) When I used `netty arrow memory 13.0.0` and `netty 4.1.96.Final` in Spark, the following error occurred, Because `netty 4.1.96.Final` version has revert some modifications, in order to ensure that `netty arrow memory 13.0.0` works well with ``netty 4.1.96.Final`` version, I suggest making similar modifications here. 1.Compilation errors are as follows: https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/builds/47657403 <img width="955" alt="image" src="https://github.com/apache/arrow/assets/15246973/e7ee2da9-97c0-474c-a62d-5821858e361f"> 2.Some modifications have been reverted in `netty 4.1.96.Final` as follows: <img width="884" alt="image" src="https://github.com/apache/arrow/assets/15246973/0226685a-cfa3-4b8b-b114-23ad8d027c05"> <img width="907" alt="image" src="https://github.com/apache/arrow/assets/15246973/a6ea21a0-8531-42b6-ab9d-25eaab1c7fde"> https://netty.io/news/2023/07/27/4-1-96-Final.html netty/netty#13510 * Closes: #36928 Authored-by: panbingkun <[email protected]> Signed-off-by: David Li <[email protected]>

lidavidm · 2023-07-28T15:08:37Z

@github-actions crossbow submit spark

github-actions · 2023-07-28T15:12:03Z

Revision: 3006cfc

Submitted crossbow builds: ursacomputing/crossbow @ actions-1f081194d6

Task	Status
test-conda-python-3.10-spark-master
test-conda-python-3.8-spark-v3.1.2
test-conda-python-3.9-spark-v3.2.0

… 4.1.96 (apache#36926) (#40) When I used `netty arrow memory 13.0.0` and `netty 4.1.96.Final` in Spark, the following error occurred, Because `netty 4.1.96.Final` version has revert some modifications, in order to ensure that `netty arrow memory 13.0.0` works well with ``netty 4.1.96.Final`` version, I suggest making similar modifications here. 1.Compilation errors are as follows: https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/builds/47657403 <img width="955" alt="image" src="https://github.com/apache/arrow/assets/15246973/e7ee2da9-97c0-474c-a62d-5821858e361f"> 2.Some modifications have been reverted in `netty 4.1.96.Final` as follows: <img width="884" alt="image" src="https://github.com/apache/arrow/assets/15246973/0226685a-cfa3-4b8b-b114-23ad8d027c05"> <img width="907" alt="image" src="https://github.com/apache/arrow/assets/15246973/a6ea21a0-8531-42b6-ab9d-25eaab1c7fde"> https://netty.io/news/2023/07/27/4-1-96-Final.html netty/netty#13510 * Closes: apache#36928 Authored-by: panbingkun <[email protected]> Signed-off-by: David Li <[email protected]> Co-authored-by: panbingkun <[email protected]>

conbench-apache-arrow · 2023-08-09T02:47:33Z

After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit 6b1c723.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

… 4.1.96 (apache#36926) When I used `netty arrow memory 13.0.0` and `netty 4.1.96.Final` in Spark, the following error occurred, Because `netty 4.1.96.Final` version has revert some modifications, in order to ensure that `netty arrow memory 13.0.0` works well with ``netty 4.1.96.Final`` version, I suggest making similar modifications here. 1.Compilation errors are as follows: https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/builds/47657403 <img width="955" alt="image" src="https://github.com/apache/arrow/assets/15246973/e7ee2da9-97c0-474c-a62d-5821858e361f"> 2.Some modifications have been reverted in `netty 4.1.96.Final` as follows: <img width="884" alt="image" src="https://github.com/apache/arrow/assets/15246973/0226685a-cfa3-4b8b-b114-23ad8d027c05"> <img width="907" alt="image" src="https://github.com/apache/arrow/assets/15246973/a6ea21a0-8531-42b6-ab9d-25eaab1c7fde"> https://netty.io/news/2023/07/27/4-1-96-Final.html netty/netty#13510 * Closes: apache#36928 Authored-by: panbingkun <[email protected]> Signed-off-by: David Li <[email protected]>

Make it run well with the netty newest version 4.1.96

3006cfc

panbingkun requested a review from lidavidm as a code owner July 28, 2023 11:43

github-actions bot added Component: Java awaiting review Awaiting review labels Jul 28, 2023

panbingkun changed the title ~~Make it run well with the netty newest version 4.1.96~~ GH-36928: [Java] Make it run well with the netty newest version 4.1.96 Jul 28, 2023

lidavidm approved these changes Jul 28, 2023

View reviewed changes

github-actions bot added awaiting merge Awaiting merge and removed awaiting review Awaiting review labels Jul 28, 2023

This was referenced Jul 28, 2023

[Abandoned] [Java] Spark integration failure due to Netty version #36493

Closed

[SPARK-44128][BUILD] Upgrade netty to 4.1.93 apache/spark#41681

Closed

lidavidm merged commit 6b1c723 into apache:main Jul 28, 2023

lidavidm removed the awaiting merge Awaiting merge label Jul 28, 2023

xxlaykxx mentioned this pull request Jul 30, 2023

GH-36928: [Java] Make it run well with the netty newest version 4.1.9… dremio/arrow#40

Merged

panbingkun mentioned this pull request Jul 31, 2023

[SPARK-44247][BUILD] Upgrade Arrow to 13.0.0 apache/spark#42181

Closed

danepitkin mentioned this pull request Aug 2, 2023

[Java] Make arrow-memory-netty work with latest netty-handler.jar (4.1.94) #36713

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GH-36928: [Java] Make it run well with the netty newest version 4.1.96#36926

GH-36928: [Java] Make it run well with the netty newest version 4.1.96#36926
lidavidm merged 1 commit intoapache:mainfrom
panbingkun:upgrade_netty_4_1_96

panbingkun commented Jul 28, 2023 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Jul 28, 2023

Uh oh!

panbingkun commented Jul 28, 2023

Uh oh!

github-actions bot commented Jul 28, 2023

Uh oh!

lidavidm left a comment

Uh oh!

raulcd commented Jul 28, 2023

Uh oh!

lidavidm commented Jul 28, 2023

Uh oh!

raulcd commented Jul 28, 2023

Uh oh!

lidavidm commented Jul 28, 2023

Uh oh!

github-actions bot commented Jul 28, 2023

Uh oh!

conbench-apache-arrow bot commented Aug 9, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

panbingkun commented Jul 28, 2023 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 28, 2023

Uh oh!

panbingkun commented Jul 28, 2023

Uh oh!

github-actions bot commented Jul 28, 2023

Uh oh!

lidavidm left a comment

Choose a reason for hiding this comment

Uh oh!

raulcd commented Jul 28, 2023

Uh oh!

lidavidm commented Jul 28, 2023

Uh oh!

raulcd commented Jul 28, 2023

Uh oh!

lidavidm commented Jul 28, 2023

Uh oh!

github-actions bot commented Jul 28, 2023

Uh oh!

conbench-apache-arrow bot commented Aug 9, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

panbingkun commented Jul 28, 2023 •

edited by github-actions bot

Loading