S3: fix aws-global validation in CreateBucket
#13250
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bentsku
added
aws:s3
Test Results (amd64) - Integration, Bootstrap 5 files 5 suites 2h 39m 33s ⏱️ Results for commit 27fa8d4. ♻️ This comment has been updated with latest results. |
bentsku
force-pushed
the
fix-aws-global
branch
from
d302d10 to
27fa8d4
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Following up on errors report we have where some SDK are directly using the
aws-globalregion to sign the request, leading to exception as our internal stores are validating the regions used.localstack.aws.api.core.CommonServiceException: exception while calling s3.CreateBucket: 'aws-global' is not a valid AWS region name for s3This is because some SDK/IaC tools are falling back to the
aws-globalregion for S3 when no region is configured. This is a pseudo-region used only for endpoint resolving by SDK, like in Botocore.But those SDK will have a mapping to use a certain region for the credentials, meaning they need to use a different region than the pseudo-region for signing the request. The region is
us-east-1is all cases I could find.But it seems a bug in the SDK themselves might be there, as if you manually override the endpoint by setting
AWS_ENDPOINT_URLor something like that, it skips the properly mapping between endpoint/credentials to sign the request and signs the request withaws-globalwhich also fails again AWS.This PR raises a proper exception in S3, as it's the most touched service, so that at least the proper exception in parity with AWS is returned and we don't return
500exception when the problem is client side.The exception can happen globally in any service, but the exception type/object per service and per protocol, so we can scope it down to S3 only for now and generalize if we encounter the issue on other services later. This could also be a specific service request handler for all S3 operations, I'm choosing to scope it down on purpose for now to see if it solves most issues.
Note: failures unrelated, due to update in DynamoDB
Changes
aws-globalis not workings3.CreateBucketandHeadBucket, even if this exception can happen generally globally in LocalStack. We could add better handling of invalid region all together, but this is scoped down to S3 to help users debug their issues and avoid IaC provider breaking on500status code.