Add Polaris Apprunner Gradle/Maven plugins by snazy · Pull Request #18 · apache/polaris-tools

snazy · 2025-04-24T12:41:44Z

Gradle and Maven plugins to run a Polaris process and "properly" terminate it for integration testing.

More information in the README.md in the apprunner/ directory.

dimas-b · 2025-04-24T14:05:48Z

apprunner/codestyle/copyright-header-hash.txt

Could we share ASF stuff across all sub-modules?

flyrain

Thanks @snazy for the PR. This seems a huge one. I'd suggest a dev mailing discussion first. This plugin seems to be used for integration tests. Other than adding a whole plugin in to Polaris repo, is there any alternative? For example, is there any plugin can be imported directly instead of having all source code within the project.

flyrain · 2025-04-26T21:11:29Z

apprunner/README.md

+}
+```
+
+### Groovy DSL


Do we need groovy as Polaris is using Kotlin with Gradle?

We don't need Groovy as, indeed, Polaris is using Kotlin. However, I believe this project is a runner to facilitate Polaris users' life. Users can run the integration tests of their Enterprise projects in isolation without having to manually deploy a Polaris server themselves. And we cannot make any assumption as to whether they are using Groovy or Kotlin in their Gradle build.

flyrain · 2025-04-26T21:12:40Z

apprunner/README.md

+}
+```
+
+## Maven


Do we need maven doc as Polaris doesn't use maven?

Same comment as my previous answer: we don't need Maven as, indeed, Polaris is using Gradle.

However, I believe this project is a runner to facilitate Polaris users' life. Users can run the integration tests of their Enterprise projects in isolation without having to manually deploy a Polaris server themselves. Maven is the other mainstream Java build system. So it only makes sense that the Apprunner comes with instructions and code to support Maven too.

pingtimeout

No explicit approval yet as there are some questions I think would be worth answering. But generally speaking, the code looks good.

apprunner/README.md

pingtimeout · 2025-05-09T15:01:32Z

apprunner/README.md

+
+## FAQ
+
+### Does it have to be a Polaris Quarkus server?


Q: This FAQ entry makes me think that, eventually, the code from this module could be extracted and donated to Quarkus. Then this module would just become a specialization of a "Quarkus Gradle Runner for IT" project. Is this correct?

I would probably leave this question/answer out unless we want to support this code for other projects.

pingtimeout · 2025-05-09T15:05:06Z

apprunner/README.md

+}
+```
+
+### Groovy DSL


We don't need Groovy as, indeed, Polaris is using Kotlin. However, I believe this project is a runner to facilitate Polaris users' life. Users can run the integration tests of their Enterprise projects in isolation without having to manually deploy a Polaris server themselves. And we cannot make any assumption as to whether they are using Groovy or Kotlin in their Gradle build.

pingtimeout · 2025-05-09T15:06:45Z

apprunner/README.md

+}
+```
+
+## Maven


Same comment as my previous answer: we don't need Maven as, indeed, Polaris is using Gradle.

However, I believe this project is a runner to facilitate Polaris users' life. Users can run the integration tests of their Enterprise projects in isolation without having to manually deploy a Polaris server themselves. Maven is the other mainstream Java build system. So it only makes sense that the Apprunner comes with instructions and code to support Maven too.

pingtimeout · 2025-05-09T15:14:55Z

apprunner/common/src/main/java/org/apache/polaris/apprunner/common/InputBuffer.java

+
+      var any = false;
+      while (input.ready()) {
+        var c = input.read();


Reading form an input stream byte by byte could result in a lot of syscalls. It seems to me that using a BufferedInputStream would be a lot better. Am I missing something ? Or, is it acceptable because we only use the InputBuffer for a small amount of data?

It's only there to capture the startup phase output from Quarkus via its stdout.

pingtimeout · 2025-05-09T16:04:33Z

apprunner/common/src/main/java/org/apache/polaris/apprunner/common/ProcessHandler.java

+         * out as fast as possible. If there's no data available, the loop will "yield" via a
+         * Thread.sleep(1L), which is good enough.
+         *
+         * Note: we cannot do blocking-I/O here, because we have to read from both stdout+stderr.


Is this still true? Given that ProcessBuilder::redirectErrorStream was called, both stdout and stderr are merged into a single stream. So it seems to me that we could do blocking I/O.

Nope. It would block forever when the process doesn't yield any output.

Ah ok so then it is the comment that is misleading

pingtimeout · 2025-05-09T16:14:24Z

apprunner/common/src/main/java/org/apache/polaris/apprunner/common/ListenUrlWaiter.java

+  public void accept(String line) {
+    if (!listenUrl.isDone()) {
+      synchronized (capturedLog) {
+        capturedLog.add(line);


Is it possible that capturing all Quarkus output and keeping it in the heap could become a problem if, say, Polaris was started with TRACE logging? It seems to me that capturing all logs was mostly used for debugging purposes?

It's not a problem in practice. But you want to see all output in case the startup fails. So yes, it's for debugging when things don't work right w/ "your" server.

pingtimeout · 2025-05-09T16:51:52Z

...src/test/resources/org/apache/polaris/apprunner/plugin/TestSimulatingTestUsingThePlugin.java

+ */
+class TestSimulatingTestUsingThePlugin {
+  @Test
+  void pingNessie() throws Exception {


Suggested change

void pingNessie() throws Exception {

void pingPolaris() throws Exception {

Well, it's Nessie in this test.

pingtimeout · 2025-05-09T16:54:14Z

...src/test/resources/org/apache/polaris/apprunner/plugin/TestSimulatingTestUsingThePlugin.java

+
+    var client = NessieClientBuilder.createClientBuilderFromSystemSettings().withUri(uri).build(NessieApiV2.class);
+    // Just some simple REST request to verify that Polaris is started - no fancy interactions w/ Nessie
+    var config = client.getConfig();


There are references to Nessie that can most likely be simply renamed (e.g. pingNessie()) and others that I am less sure about (Nessie client usage). Is using the Nessie Client intended? There are mentions of Nessie's REST API in code that targets the embedded Polaris server, and that is confusing. What is the rationale for using the Nessie Client here?

At the time of pushing this, there was no Polaris binary.

However, it should eventually test both Polaris and Nessie. The principle (code wise) however is the same.

pingtimeout · 2025-05-09T16:56:44Z

.../src/test/java/org/apache/polaris/appruner/maven/mavenit/ITSimulatingTestUsingThePlugin.java

+    String uri = String.format("http://127.0.0.1:%s/api/v1", port);
+
+    NessieApiV1 client = HttpClientBuilder.builder().withUri(uri).build(NessieApiV1.class);
+    // Just some simple REST request to verify that Nessie is started - no fancy interactions w/ Nessie


More mentions of Nessie here

snazy · 2025-05-12T07:34:01Z

Heads up: this is the successor of apache/polaris#785 - it's been open for quite a while.

RussellSpitzer · 2025-06-17T16:10:42Z

apprunner/README.md

+
+### Using the plugin in projects in the polaris-tools repository
+
+1. include the apprunner build in your project by adding the following snippet at the beginning of your


Suggested change

1. include the apprunner build in your project by adding the following snippet at the beginning of your

1. Include the apprunner build in your project by adding the following snippet at the beginning of your

RussellSpitzer · 2025-06-17T16:10:58Z

apprunner/README.md

+
+`build.gradle.kts`
+
+1. add the plugin


Suggested change

1. add the plugin

1. Add the plugin

snazy · 2025-11-11T18:10:53Z

This PR has been open for quite a while.
Questions have been answered, so I assume it's okay to merge this one in the next few days.

dimas-b · 2025-11-11T18:23:50Z

We still have a "requested change" ... @flyrain: Does this concern still exist in the latest state of this PR?

snazy · 2025-11-18T14:14:40Z

Ping: @flyrain: Does this concern still exist in the latest state of this PR?

If no objections, I'm going to merge tomorrow.

snazy · 2025-11-18T14:20:56Z

Ping: @flyrain: Does this concern still exist in the latest state of this PR?

If no objections, I'm going to merge tomorrow.

Oh, seems you're okay with merging

See https://lists.apache.org/thread/5ov54gbvfcmqpmydffqkynj4pdo1jo1c

snazy requested review from MonkeyCanCode, RussellSpitzer, adutra, ashvina, collado-mike, dennishuo, dimas-b, ebyhr, eric-maynard, flyrain, jackye1995, jbonofre, takidau and vvcephei as code owners April 24, 2025 12:41

snazy force-pushed the apprunner branch 5 times, most recently from ae71aa5 to 6e7cdc1 Compare April 24, 2025 13:03

dimas-b approved these changes Apr 24, 2025

View reviewed changes

apprunner/codestyle/copyright-header-hash.txt

Copy link

dimas-b Apr 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we share ASF stuff across all sub-modules?

snazy force-pushed the apprunner branch from 6e7cdc1 to f3d888b Compare April 26, 2025 18:36

flyrain previously requested changes Apr 26, 2025

View reviewed changes

snazy force-pushed the apprunner branch from f3d888b to 0eb0296 Compare May 6, 2025 12:23

snazy requested a review from HonahX as a code owner May 6, 2025 12:23

pingtimeout reviewed May 9, 2025

View reviewed changes

RussellSpitzer reviewed Jun 17, 2025

View reviewed changes

apprunner/README.md

`build.gradle.kts`

1. add the plugin

Copy link

Member

RussellSpitzer Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

1. add the plugin

1. Add the plugin

snazy force-pushed the apprunner branch from 537ece1 to cc6543e Compare June 23, 2025 08:30

snazy requested a review from ajantha-bhat as a code owner June 23, 2025 08:30

snazy requested a review from singhpk234 as a code owner June 23, 2025 08:30

snazy mentioned this pull request Aug 12, 2025

Add a regression test for Catalog Federation apache/polaris#2286

Merged

snazy force-pushed the apprunner branch from cc6543e to 617b1b9 Compare October 22, 2025 07:57

snazy mentioned this pull request Nov 11, 2025

feat: Create Spark 4 Client apache/polaris#3026

Closed

5 tasks

snazy force-pushed the apprunner branch from 617b1b9 to 77910be Compare November 11, 2025 13:52

Add Polaris Apprunner Gradle/Maven plugins

447bfa7

snazy force-pushed the apprunner branch from 77910be to 447bfa7 Compare November 11, 2025 14:17

dimas-b approved these changes Nov 11, 2025

View reviewed changes

snazy merged commit e122607 into apache:main Nov 18, 2025
6 checks passed

snazy deleted the apprunner branch November 18, 2025 14:21

	void pingNessie() throws Exception {
	void pingPolaris() throws Exception {


		### Using the plugin in projects in the polaris-tools repository

		1. include the apprunner build in your project by adding the following snippet at the beginning of your

Conversation

snazy commented Apr 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flyrain left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pingtimeout left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

snazy May 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

snazy commented May 12, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

snazy commented Nov 11, 2025

Uh oh!

dimas-b commented Nov 11, 2025

Uh oh!

snazy commented Nov 18, 2025

Uh oh!

snazy commented Nov 18, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

snazy May 12, 2025 •

edited

Loading