Remove "giant" constructors from Handlers and Adapters#3669
Remove "giant" constructors from Handlers and Adapters#3669adutra merged 3 commits intoapache:mainfrom
Conversation
| // Initialized in the authorize methods. | ||
| protected PolarisResolutionManifest resolutionManifest = null; | ||
|
|
||
| protected final ResolutionManifestFactory resolutionManifestFactory; |
There was a problem hiding this comment.
Just an idea: If we convert fields to abstract getters and make implementations of this class @Immutable, the CatalogHandlerRuntime object will probably become unnecessary and impl. instances will have direct builders... WDYT?
There was a problem hiding this comment.
I had this approach initially but changed for the Parameter Object pattern, for a few reasons:
CatalogHandlerand subclasses are inherently mutable. Fields likeresolutionManifestare lazily evaluated, but are hard to migrate to a@Value.Lazyequivalent because they may be evaluated more than once.- Getters in
CatalogHandlerwould have to be public, because it's in a different package from its subclasses, and thus the generated builders only compile if the getters are public.
That said, I don't mind showing my initial version, maybe it's acceptable indeed. The advantage being, that we could look into removing the mutable state little by little until the class hierarchy becomes fully immutable.
I will add a commit on top of this branch to show the approach.
99a05f3 to
63c3abd
Compare
| // Initialized in the authorize methods. | ||
| @SuppressWarnings("immutables:incompat") | ||
| protected PolarisResolutionManifest resolutionManifest = null; |
There was a problem hiding this comment.
This might be fix/addressed if we go ahead with what @sungwy proposed for AuthZ refactoring plus my additional proposal for handling the "manifest" per API call: https://lists.apache.org/thread/srqp2jtts5438drzcvky47mdt8zs80wt
dimas-b
left a comment
There was a problem hiding this comment.
Thanks for considering my proposal, @adutra !
I tend to like using direct builders (despite having to suppress build warnings from the Immutables library) because it looks less verbose than using Object Parameters.
However, I'm not attached to this approach 😉 I wonder what other maintainers think 🤔
snazy
left a comment
There was a problem hiding this comment.
Nice work!
The constructors became bigger and bigger, I hope that we'll eventually, in the future, be able not not change a lot of sources to add a reference.
This PR attempts to reduce the "giant constructor" symptoms in catalog adapters and handlers. It combines two techniques: * The injectable runtime dependencies were grouped together in a "runtime" object, inspired by the "Parameter Object" pattern. These objects are `@PolarisImmutable` objects, and thus can leverage the generated builder to create instances without the need for big constructors. * The runtime object is then produced by a new factory, which is a CDI request-scoped bean with direct field injection to avoid constructor injection.
1e581c0 to
d8acf4e
Compare
This PR attempts to reduce the "giant constructor" symptoms in catalog adapters and handlers. * The `CatalogHandler` subtypes are now `@PolarisImmutable` objects, and thus can leverage the generated builder to create instances without the need for big constructors. * The handlers are now produced by a new factory, which is a CDI request-scoped bean with direct field injection to avoid constructor injection.
This PR does some additional cleanup after apache#3669.
This PR does some additional cleanup after apache#3669.
* Remove "giant" constructors from Handlers and Adapters (apache#3669) This PR attempts to reduce the "giant constructor" symptoms in catalog adapters and handlers. * The `CatalogHandler` subtypes are now `@PolarisImmutable` objects, and thus can leverage the generated builder to create instances without the need for big constructors. * The handlers are now produced by a new factory, which is a CDI request-scoped bean with direct field injection to avoid constructor injection. * Fix FileIO usage in PolarisRestCatalogIntegrationBase (apache#3658) Currently, all tests inheriting from `PolarisRestCatalogIntegrationBase` are forcibly using `InMemoryFileIO` for all `RESTCatalog` instances. Furthermore, this class wasn't overriding `CatalogTests.baseTableLocation()`, so tests calling this method were actually operating on `file://` paths all the time. Because of that, some child test classes, and in particular, the MiniIO-based ones, weren't actually testing anything useful. This PR changes `PolarisRestCatalogIntegrationBase` to use a configurable `FileIO` impl, defaulting to `ResolvingFileIO`. This change uncovered a few latent issues with current tests, and notably: 1) Tests using MinIO should request credential vending consistently, otherwise most tests would fail because the S3 client attempts to load credentials from the environment. 2) Tests calling `registerTable` need special handling since this endpoint does NOT support access delegation at all. For these tests, the `RESTCatalog` clients need to be given direct credentials for S3 – since the server won't give them anything – even with vended credentials requested. Incidentally, this PR simplifies `IntegrationTestsHelper.mergeFromAnnotatedElements()` – but its functionality stays the same. * Automatically adjust gradlew during Gradle Wrapper updates (apache#3620) * CI for feature branches (apache#3686) This change enables CI on branches matching the pattern `feature/*`. This change does _not_ require PRs against `feature/*` branches to have "green" CI. * fix: Correct schema version number in schema-v4.sql files (apache#3690) - Update DatabaseType.java to allow schemaVersion up to 4 - Fix h2/schema-v4.sql to set version to 4 instead of 3 - Fix postgres/schema-v4.sql to set version to 4 instead of 3 Co-authored-by: Anand Kumar Sankaran <[email protected]> * Add doc for behavior change configuration (apache#3677) * Disable python update from GH action (apache#3678) * Add support for rustfs test-containers (apache#3679) * Fix CI (apache#3698) * Fix status badge for README.md (apache#3696) * Fix README.md CI URL * Fix README.md CI URL * Update dependency io.opentelemetry:opentelemetry-bom to v1.59.0 (apache#3704) * Update dependency com.puppycrawl.tools:checkstyle to v13.2.0 (apache#3703) * Update dependency com.google.errorprone:error_prone_core to v2.47.0 (apache#3702) * Update registry.access.redhat.com/ubi9/openjdk-21-runtime Docker tag to v1.24-2.1770236038 (apache#3701) * Update dependency ch.qos.logback:logback-classic to v1.5.28 (apache#3700) * chore(deps): update dependency ipykernel to v7.2.0 (apache#3705) * chore(deps): update docker.io/jaegertracing/jaeger docker tag to v2.15.0 (apache#3706) * fix(deps): update dependency io.smallrye.common:smallrye-common-annotation to v2.16.0 (apache#3707) * Last merged commit de43d1d * Fix integration-tests after apache#3704 Short story: This change fixes the int-tests in `:polaris-runtime-service`. Long story is this: In `:polaris-runtime-service` we intentionally declare "just" a `platform(libs.quarkus.bom)` dependency, because of the `:polaris-spark-integration-*` module. It is however generally recommended by Quarkus to _only_ use `enforcedPlatform(libs.quarkus.bom)` to effectively prevent breaking changes coming from transitive dependencies. Which is exactly what happend after apache#3704. Why did CI not catch this issue? The answer is pretty simple: The effective Gradle task inputs, including the `intTestRuntimeClasspath` did not change. So the previously cached test outcomes could be reused, and the int-tests did not run. Just adding the `runtimeClasspath` as another task-input of the `intTest` may _not_ work as intended. Why does _removing_ `implementation(platform(libs.opentelemetry.bom))` help? Simply because that lets the dependencies fall back to to the declared transitive dependencies. We do not have direct dependencies to OTel. The correct fix here _would_ be to use `implementation(enforcedPlatform(libs.quarkus.bom))`, but that breaks the Spark plugin integration tests. There is a better alternative: Let the Spark plugin tests leverage polaris-apprunner, which is meant for exactly the use case of effectively decoupling some module from the build requirements of a Quarkus application. --------- Co-authored-by: Alexandre Dutra <[email protected]> Co-authored-by: Anand K Sankaran <[email protected]> Co-authored-by: Anand Kumar Sankaran <[email protected]> Co-authored-by: Yong Zheng <[email protected]> Co-authored-by: Mend Renovate <[email protected]>
This PR does some additional cleanup after #3669.
More cleanup after apache#3669.
This PR attempts to reduce the "giant constructor" symptoms in catalog adapters and handlers.
It combines two techniques:
@PolarisImmutableobjects, and thus can leverage the generated builder to create instances without the need for big constructors.UPDATE
CatalogHandlerand subtypes directly annotated with@PolarisImmutable.Checklist
CHANGELOG.md(if needed)site/content/in-dev/unreleased(if needed)