Skip to content

AWS: Prevent token refresh scheduling on every sign request#7270

Merged
rdblue merged 1 commit intoapache:masterfrom
nastra:signer-token-refreshes
Apr 3, 2023
Merged

AWS: Prevent token refresh scheduling on every sign request#7270
rdblue merged 1 commit intoapache:masterfrom
nastra:signer-token-refreshes

Conversation

@nastra
Copy link
Contributor

@nastra nastra commented Apr 3, 2023

One of the main issues here was that we were previously scheduling a token for refresh on every single sign request.
This PR caches the AuthSession based on the given token/credential to prevent this issue.
The existing TestS3RestSigner was slightly restructured and additional checks were added that immediately showed the original problem.

@nastra nastra force-pushed the signer-token-refreshes branch from e3d7c2b to d06f01c Compare April 3, 2023 06:34
@nastra nastra added this to the Iceberg 1.2.1 milestone Apr 3, 2023
PropertyUtil.propertyAsLong(
properties(),
CatalogProperties.AUTH_SESSION_TIMEOUT_MS,
CatalogProperties.AUTH_SESSION_TIMEOUT_MS_DEFAULT);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is 1 hour by default. Are we sure that's a reasonable timeout?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a 24 hour idle timeout would more closely align with patterns where a table is accessed and left idle. Beyond that, it's very unlikely the reference will be used again.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, after some discussion to clarify Dan, Eduard, and I think 1h is pretty reasonable. The cache is using expireAfterAccess so reusing the token keeps it alive. And the removal listener stops refreshing the token when it is expired from the cache. The downside of too short of an interval is starting new sessions, but only if they are already mostly inactive (more than 1 hour between uses). The downside of too long of an interval is potentially needlessly refreshing a token for 24 hours when it's unused. I think it is better to have a short interval.

@rdblue
Copy link
Contributor

rdblue commented Apr 3, 2023

This looks correct to me. My only question is what would be a reasonable default timeout.

@rdblue rdblue merged commit f7166be into apache:master Apr 3, 2023
@nastra nastra deleted the signer-token-refreshes branch April 3, 2023 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants