Repository navigation
REST catalog: support AWS sigV4 #1236
Description
Is your feature request related to a problem or challenge?
For AWS glue/s3 tables
rest.sigv4-enabled, rest.signing-name, rest.signing-region
rest_catalog = load_catalog(
catalog_name,
**{
"type": "rest",
"warehouse":"arn:aws:s3tables:<Region>:<accountID>:bucket/<bucketname>",
"uri": "https://s3tables.<Region>.amazonaws.com/iceberg",
"rest.sigv4-enabled": "true",
"rest.signing-name": "s3tables",
"rest.signing-region": "<Region>"
}
)
Describe the solution you'd like
No response
Willingness to contribute
None
Reacted by Anantha Krishnan, Gokul Soundararajan, Renjie Liu, Andrew Kane, Shawn Chang, Mike Williams, Josef, Roberto Trani, Francois Stephany, Haotian Chen and 1 more
Activity
xxchan commentedon Apr 23, 2025
Found a problem in
reqsign:canonical request: GET /iceberg/v1/arn%3Aaws%3As3tables%3Aap-southeast-1..., but should have been sth like/iceberg/v1/arn%253Aaws%253As3tables%253Aap-southeast-1%..., wherereqsignwrongly decoded%3.x-amz-content-sha256seems to be required for this endpoint, which is not natively supported by reqsign.ananthaksr commentedon Apr 23, 2025
@xxchan I'd like to work on this.
xxchan commentedon Apr 24, 2025
@ananthaksr Hi, thanks for offering help, but I've already finished a working draft. #1241
phillipleblanc commentedon May 1, 2025
I've also implemented this in #917
gsoundar commentedon May 7, 2025
@xxchan How is your change coming along ?
github-actions commentedon Nov 6, 2025
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
liurenjie1024 commentedon Nov 6, 2025
Missing feature.
xxchan commentedon Nov 6, 2025
pschugt commentedon Jan 14, 2026
Is there any update on this that AWS sigV4 support will be implemented in the near future? Or can i help somewhere to speed up things :)
Br!
feat(rest-catalog): add request signing support with AWS SigV4
ublubu commentedon Jun 25, 2026
Hey, I had a look at the prior art catalogued in #2311, and I really like the
Authenticatortrait idea from #2088.Over at MaterializeInc/materialize, we've already been using
iceberg-rustwith AWS SigV4 for S3 Tables (andgcp_authfor Google's BigLake) using that technique.The main difference is that, while the trait lives in our
iceberg-rustfork, the AWS/GCP-specific authenticators are implemented in our home repo,materialize.SigV4 Authenticator:
GCP Authenticator (Bearer Token Authenticator wrapper + GCP Token Provider):
Proposal for an authenticator trait to upstream into
apache/iceberg-rustRequestAuthenticator/AuthenticatorAlmost exactly the same as #2088. Most importantly,
mutaccess to the entire request (needed for SigV4):iceberg-rust/crates/catalog/rest/src/token.rs
Lines 48 to 49 in a9ad7e2
Helper types and traits:
BearerTokenAuthenticator+trait TokenProviderOAuth2 and
gcp_auth(and the static token config option) all provide bearer tokens the authenticator can put in a request header.A
BearerTokenAuthenticatorholds aimpl TokenProviderso it can act asimpl RequestAuthenticator.Similar to #2088's
OAuth2Authenticator, but split into two parts.Draft diff to add just these new traits without breaking anything:
Here's a full diff illustrating what I have in mind. I did my best to preserve the existing auth behavior and config APIs.
main...MaterializeInc:iceberg-rust:to-upstream/requestauthenticator
Next steps
I'm actively working on Materialize's support for Iceberg. I'd like to get our
iceberg-rustfork closer to upstream, so I'm happy to drive this if it needs a driver.