Refactor: Use per-request STS credentials#1629
Merged
dimas-b merged 4 commits intoapache:mainfrom May 23, 2025
Merged
Conversation
dimas-b
requested review from
HonahX,
MonkeyCanCode,
RussellSpitzer,
adutra,
ashvina,
collado-mike,
dennishuo,
ebyhr,
eric-maynard,
flyrain,
jackye1995,
jbonofre,
snazy,
takidau and
vvcephei
as code owners
singhpk234
reviewed
May 21, 2025
Contributor
There was a problem hiding this comment.
can we call this credentialProvider instead ?
Contributor
There was a problem hiding this comment.
any reason to keep Logger getter in this method rather outside ?
Contributor
Author
There was a problem hiding this comment.
oops, it was just a copy-paste.... will fix.
Contributor
Author
There was a problem hiding this comment.
Actually, refactoring this in current PR is awkward - I would not like to add a logger field to this interface. Adding a class looks like an overkill.
Would you mind if I moved this to a production readiness check in a follow-up PR?
Contributor
There was a problem hiding this comment.
[orthogonal] Thoughts on supportingdynamically loading credential provider implementation ?
Contributor
Author
There was a problem hiding this comment.
The idea (for now) is for custom builds to manage cred. provided via a custom impl. of PolarisStorageIntegrationProvider.
I suppose this will evolve as we move forward with @XJDKC 's proposal: https://lists.apache.org/thread/ph1tvn3lzvn8kh8fnhc6k585qmw2m12r
Member
There was a problem hiding this comment.
Yes, I have a branch that's based on the proposal, and it's working. I'm trying to clean it and add more tests, then I will open a PR. Welcome to review the PR and leave your comments!
https://github.com/XJDKC/polaris/tree/rxing-catalog-federation-sigv4-poc
snazy
reviewed
May 21, 2025
No functional changes. This is mostly to allow more storage integration flexibility in downstream build. This might also be useful for non-AWS storage.
snazy
reviewed
May 23, 2025
| import java.util.List; | ||
| import java.util.Map; | ||
| import java.util.Set; | ||
| import java.util.*; |
snazy
approved these changes
May 23, 2025
Member
snazy
left a comment
snazy
left a comment
There was a problem hiding this comment.
no more wildcard imports -> +1 ;)
github-project-automation
bot
moved this from PRs In Progress
to Ready to merge
in Basic Kanban Board
github-project-automation
bot
moved this from Ready to merge
to Done
in Basic Kanban Board
snazy
added a commit
to snazy/polaris
that referenced
this pull request
Jun 13, 2025
* fix(nightly-CI): Do not publish snapshots from forks (apache#1635) Adopt the `Nightly Build` workflow to not (try to) publish every night from forks. * main: Update dependency io.smallrye.config:smallrye-config-core to v3.13.0 (apache#1637) * Use echo to print script errors (apache#1648) * [HOTFIX] QUICKSTART (apache#1646) The change adds the following to fix Quick start experience : [1] ENV variables required by common assets after apache#1522 [2] New configs required to enable FILE based sources apache#1649 Co-authored-by: singhpk234 <[email protected]> Co-authored-by: pjanuario <[email protected]> * main: Update dependency gradle to v8.14.1 (apache#1652) * main: Update dependency gradle to v8.14.1 * Re-adopt PR to the project's needs --------- Co-authored-by: Robert Stupp <[email protected]> * [Policy Store] Add policyTypeCode to Slice/Index for Future Filtering Support and Update Policy Persistence Method (apache#1628) This PR adds policyTypeCode to the in-memory tree map store's slice and the SQL index on policy_mapping_records (already done in JDBC in apache#1468). This prepares for future features that need to filter efficiently by policy type, like listing all entities with a data compaction policy. It also updates the loadAllTargetsOnPolicy method to accept policyTypeCode, enabling it to use the new index for better performance. * fix(test): Do not let some more tests spam `/tmp` (apache#1651) * fix(test): Do not let some more tests not spam `/tmp` * `PolarisRestCatalogViewFileIntegrationTest` * `FileIOExceptionsTest` * `PolarisRestCatalogViewFileIntegrationTest` Changes the tests to leverage JUnit's `@TempDir`. Simplifies `PolarisEclipseLinkMetaStoreManagerTest` * review: rename the (now) abstract class * fix(testing): Do not let PolarisOverlappingTableTest spam `/tmp` (apache#1641) Changes the test to leverage JUnit's `@TempDir`. * Add CATALOG_MANAGE_METADATA to super privilege set of policy attachment privileges (apache#1643) * Fix quickstart doc with docker compose (apache#1610) * main: Update dependency boto3 to v1.38.22 (apache#1657) * Refactor IcebergCatalog to isolate internal state (apache#1659) Following up on apache#1694 * Restore `private` scope on internal fields in `IcebergCatalog` * Use a test-only setter instead of sub-classing to manage injecting test FileIO implementations * Refactor: Use per-request STS credentials (apache#1629) * Refactor: Use per-request STS credentials No functional changes. This is mostly to allow more storage integration flexibility in downstream build. This might also be useful for non-AWS storage. * fix and enforce more errorprone checks (apache#1663) enforces the following checks: https://errorprone.info/bugpattern/ObjectsHashCodePrimitive https://errorprone.info/bugpattern/OptionalMapToOptional https://errorprone.info/bugpattern/StringCharset https://errorprone.info/bugpattern/VariableNameSameAsType * Create a wrapper script to generate python client; regenerate the python client (apache#1347) As noted in apache#755 and elsewhere, the generated types in client/python are currently out of date. This introduces a script to regenerate them and a gradle task to run that script. I've also run the script, which necessitated several things to get tests passing: 1. There were small nonfunctional spec changes needed in order to keep the Python client working 2. The CLI and its tests required a few fixes to work with the updated Python client 3. Many of the regtests required fixes to work with the updated Python client * [Python Client] CI for Python client (Continue PR#1096) (apache#1639) Adds CI for python client. It does not include caching poetry step for now since we do not have poetry.lock (it is in .gitignore), see relevant discussion in: apache#1102 (comment), apache#1096 (comment), we can add that later * main: Update actions/setup-python action to v5 (apache#1671) * main: Update actions/checkout action to v4 (apache#1670) * main: Update python Docker tag to v3.13 (apache#1669) * main: Update dependency pytest to ~=7.4.4 (apache#1668) * main: Update dependency software.amazon.awssdk:bom to v2.31.50 (apache#1677) * main: Update dependency boto3 to v1.38.23 (apache#1667) * feat(build): make archive builds reproducible (apache#1664) See https://docs.gradle.org/current/userguide/working_with_files.html#sec:reproducible_archives * main: Update dependency io.prometheus:prometheus-metrics-exporter-servlet-jakarta to v1.3.8 (apache#1679) * NoSQL: adapt to change on oss/main * INFO: Last merged commit: 6ef8b3e --------- Co-authored-by: Mend Renovate <[email protected]> Co-authored-by: ModEtchFill <[email protected]> Co-authored-by: Prashant Singh <[email protected]> Co-authored-by: singhpk234 <[email protected]> Co-authored-by: pjanuario <[email protected]> Co-authored-by: Honah (Jonas) J. <[email protected]> Co-authored-by: MonkeyCanCode <[email protected]> Co-authored-by: Dmitri Bourlatchkov <[email protected]> Co-authored-by: Dmitri Bourlatchkov <[email protected]> Co-authored-by: Christopher Lambert <[email protected]> Co-authored-by: Eric Maynard <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No functional changes.
This is mostly to allow more storage integration
flexibility in downstream build.
This might also be useful for non-AWS storage.