Repository navigation
[DynamoDB] - Allow passing ddb endpoint_url to enable feast local testing #2358
Description
Is your feature request related to a problem? Please describe.
Currently in feature_store.yaml, we can only specify a region for DynamoDB provider. As a result, it requires an actual DynamoDB to be available when we want to do local development/testing or integration testing in a sandbox environment.
Describe the solution you'd like
A way to solve this is to let user pass an endpoint_url. More information can be found here.
This way, users can install and run a local dynamoDB, and use it as an online store locally.
The is especially useful when:
- accessing real DynamoDB requires a tedious and time-consuming steps (e.g.: IAM role set up, permissions, etc.) and these steps can be deferred later,
- integration testing locally, in docker, in Jenkins
Describe alternatives you've considered
N/A
Additional context
Not much but the initial slack thread can be found here, which was followed by a slack message from our team member
Activity
TremaMiguel commentedon Mar 24, 2022
@vlin-lgtm I think this one is kind of easy _initialize_dynamodb_client and _initialize_dynamodb_resource would now accept
endpoint_urlwhich can be passed through the DynamoDBOnlineStoreConfig.
But, I'm not sure how will this be for local integration tests, as far as I know just need to change the DYNAMODB_CONFIG with an additional parameter
endpoint_urland set an IntegrationTestConfig with DynamoDB as Online Store.What are your thoughts?
P.D. I'm happy to contribute with this
vlin-lgtm commentedon Mar 30, 2022
Thanks so much, @TremaMiguel!
My understanding is one can run a local dynamoDB on their development machine or in a docker container, and by specifying an endpoint_url to for example
localhost:8080, all DynamoDB invocations will go to the localhost instance instead of an actual one in AWS.This is useful for integration testing as everything is still "local".
Would love it if you can help contribute to this 🙏
TremaMiguel commentedon Mar 31, 2022
@vlin-lgtm Thanks for replying
I agree with the local integration tests, additionaly I think this development could help with #2400
I'll start working on this.