feat(iam auth): allow iam roles anywhere auth profile by sicoyle · Pull Request #3591 · dapr/components-contrib

sicoyle · 2024-11-04T19:41:08Z

Description

The overall logic for this auth profile + it's proper rotation is completed. I just need to update to add a mock for the authprovider clients, update the rest of the existing tests, and wrap up adding new ones.

Summary: Add an auth profile for using AWS IAM Roles Anywhere. This leverages x.509 certificates to establish trust between an AWS account and a trusted certificate authority. Here, we can use the Dapr SVID very nicely.

I had to do an overhaul on the existing AWS IAM setup because now it is the case that there is the existing static IAM auth with things like accesskeys, secretkeys, etc. However, now, I'm adding temporary auth that Dapr is responsible for rotating. Default expiration time with AWS for IAM Roles Anywhere session durations is 1hour, so that is our default with a rotation when we've hit half of that time in a goroutine if that auth profile is picked up. If a user sets a session duration for timeout, then we will not rotate the credentials.

This applies to all existing AWS components.

I plan to add for Postgres + Kafka in a secondary PR since this one is already so beefy.

Issue reference

We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.

Please reference the issue this PR will close: #[issue number]

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

Code compiles correctly
Created/updated tests - TODO
Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number] TODO

Signed-off-by: Samantha Coyle <[email protected]>

Signed-off-by: Sam <[email protected]>

Signed-off-by: Samantha Coyle <[email protected]>

sicoyle · 2024-11-12T23:46:51Z

I'm trying to inject mocked creds to make conformance tests pass for some of the aws components, but not too familiar with their setup. Clearly I need to adjust this further bc they're all failing... maybe add an IAM role on the terraform for IAM roles anywhere? or inject mock creds somewhere else? I had to do that for the unit tests for some of them... 🫠 I will come back to this tomorrow morning.

=== RUN   TestSecretStoreConformance/aws.secretsmanager.docker/get/get
    secretstores.go:84: 
        	Error Trace:	/home/runner/work/components-contrib/components-contrib/tests/conformance/secretstores/secretstores.go:84
        	Error:      	Received unexpected error:
        	            	couldn't get secret: UnrecognizedClientException: The security token included in the request is invalid.
        	            		status code: 400, request id: fbf08edb-02d5-47d2-b[60](https://github.com/dapr/components-contrib/actions/runs/11807458135/job/32894166961?pr=3591#step:24:61)a-4dcb432b6e8f
        	Test:       	TestSecretStoreConformance/aws.secretsmanager.docker/get/get
        	Messages:   	expected no error on getting secret {conftestsecret map[]}
--- FAIL: TestSecretStoreConformance/aws.secretsmanager.docker/get/get (0.06s)
--- FAIL: TestSecretStoreConformance/aws.secretsmanager.docker/get (0.06s)
=== RUN   TestSecretStoreConformance/aws.secretsmanager.docker/bulkGet
=== RUN   TestSecretStoreConformance/aws.secretsmanager.docker/bulkGet/bulkget
    secretstores.go:105: 
        	Error Trace:	/home/runner/work/components-contrib/components-contrib/tests/conformance/secretstores/secretstores.go:105
        	Error:      	Received unexpected error:
        	            	couldn't list secrets: UnrecognizedClientException: The security token included in the request is invalid.
        	            		status code: 400, request id: 4[70](https://github.com/dapr/components-contrib/actions/runs/11807458135/job/32894166961?pr=3591#step:24:71)14dae-011e-41ee-8420-d71d91fcb48d
        	Test:       	TestSecretStoreConformance/aws.secretsmanager.docker/bulkGet/bulkget
        	Messages:   	expected no error on getting secret {map[]}
--- FAIL: TestSecretStoreConformance/aws.secretsmanager.docker/bulkGet/bulkget (0.02s)
--- FAIL: TestSecretStoreConformance/aws.secretsmanager.docker/bulkGet (0.02s)
--- FAIL: TestSecretStoreConformance/aws.secretsmanager.docker (0.08s)
--- FAIL: TestSecretStoreConformance (0.08s)

Signed-off-by: Samantha Coyle <[email protected]>

dapr-bot · 2024-11-13T21:07:52Z

Complete Build Matrix

The build status is currently not updated here. Please visit the action run below directly.

🔗 Link to Action run

Commit ref: d8af3c0

dapr-bot · 2024-11-13T21:07:54Z

Components certification test

🔗 Link to Action run

Commit ref: d8af3c0

❌ Some certification tests failed

These tests failed:

pubsub.aws.snssqs
pubsub.gcp.pubsub
state.azure.cosmosdb
state.sqlserver

dapr-bot · 2024-11-13T21:07:58Z

Components conformance test

🔗 Link to Action run

Commit ref: d8af3c0

❌ Some conformance tests failed

These tests failed:

bindings.azure.eventgrid
state.postgresql.v1.azure
state.sqlserver

Signed-off-by: Samantha Coyle <[email protected]>

sicoyle · 2024-11-13T21:32:00Z

/ok-to-test

dapr-bot · 2024-11-13T21:32:29Z

Components conformance test

🔗 Link to Action run

Commit ref: 37021c5

❌ Some conformance tests failed

These tests failed:

bindings.azure.eventgrid
state.sqlserver

dapr-bot · 2024-11-13T21:32:30Z

Components certification test

🔗 Link to Action run

Commit ref: 37021c5

❌ Some certification tests failed

These tests failed:

bindings.azure.servicebusqueues
pubsub.gcp.pubsub
state.azure.cosmosdb
state.sqlserver

dapr-bot · 2024-11-13T21:32:30Z

Complete Build Matrix

The build status is currently not updated here. Please visit the action run below directly.

🔗 Link to Action run

Commit ref: 37021c5

Signed-off-by: Samantha Coyle <[email protected]>

Signed-off-by: Sam <[email protected]>

sicoyle · 2024-11-13T22:54:54Z

this is ready for review please. All certification/conformance tests that are failing are unrelated, and flaky / failing in other PRs.

Signed-off-by: Samantha Coyle <[email protected]>

sicoyle · 2024-11-14T15:49:52Z

/ok-to-test

dapr-bot · 2024-11-14T15:50:18Z

Complete Build Matrix

The build status is currently not updated here. Please visit the action run below directly.

🔗 Link to Action run

Commit ref: 3d4da39

dapr-bot · 2024-11-14T15:50:19Z

Components certification test

🔗 Link to Action run

Commit ref: 3d4da39

❌ Some certification tests failed

These tests failed:

pubsub.aws.snssqs
pubsub.gcp.pubsub
pubsub.mqtt3
state.azure.cosmosdb
state.sqlserver

dapr-bot · 2024-11-14T15:50:21Z

Components conformance test

🔗 Link to Action run

Commit ref: 3d4da39

❌ Some conformance tests failed

These tests failed:

bindings.azure.eventgrid
state.sqlserver

Signed-off-by: Samantha Coyle <[email protected]>

sicoyle · 2024-11-14T17:44:18Z

/ok-to-test

dapr-bot · 2024-11-14T17:44:41Z

Complete Build Matrix

The build status is currently not updated here. Please visit the action run below directly.

🔗 Link to Action run

Commit ref: cde5a10

dapr-bot · 2024-11-14T17:44:43Z

Components certification test

🔗 Link to Action run

Commit ref: cde5a10

❌ Some certification tests failed

These tests failed:

pubsub.gcp.pubsub
pubsub.azure.servicebus.topics
pubsub.mqtt3
state.azure.cosmosdb
state.sqlserver

dapr-bot · 2024-11-14T17:44:43Z

Components conformance test

🔗 Link to Action run

Commit ref: cde5a10

❌ Some conformance tests failed

These tests failed:

bindings.azure.eventgrid

feat(iam auth): allow iam roles anywhere auth profile

5bd981c

Signed-off-by: Samantha Coyle <[email protected]>

sicoyle requested review from a team as code owners November 4, 2024 19:41

JoshVanL requested changes Nov 4, 2024

View reviewed changes

Comment thread .build-tools/builtin-authentication-profiles.yaml Outdated

Comment thread .build-tools/builtin-authentication-profiles.yaml Outdated

sicoyle added 10 commits November 4, 2024 13:54

fix(build): update more aws components

45db3a1

Signed-off-by: Samantha Coyle <[email protected]>

fix(metadata): add endpoint field to options

58bbeaa

Signed-off-by: Samantha Coyle <[email protected]>

style: update descriptions on new fields

e9493ae

Signed-off-by: Samantha Coyle <[email protected]>

fix: acct for nil check

bff9b16

Signed-off-by: Samantha Coyle <[email protected]>

style: make linter happy

fd30089

Signed-off-by: Samantha Coyle <[email protected]>

style: more linter fixes

f712b69

Signed-off-by: Samantha Coyle <[email protected]>

style: final linter tweaks

78cf670

Signed-off-by: Samantha Coyle <[email protected]>

fix(session): apply auto refresh to s3

8aec6a5

Signed-off-by: Samantha Coyle <[email protected]>

style: mv x509 auth around

3a5c8bf

Signed-off-by: Samantha Coyle <[email protected]>

refactor: overhaul + interfaces for cleanliness + tests + update comps

43ba1e3

Signed-off-by: Samantha Coyle <[email protected]>

JoshVanL requested changes Nov 11, 2024

View reviewed changes

lrascao reviewed Nov 11, 2024

View reviewed changes

Comment thread pubsub/aws/snssqs/snssqs.go Outdated

sicoyle added 10 commits November 12, 2024 12:16

fix: address initial feedback and fix tests

423e993

Signed-off-by: Samantha Coyle <[email protected]>

test: add tests and make things more testable

b228e39

Signed-off-by: Samantha Coyle <[email protected]>

Merge branch 'main' into feat-iam-rolesanywhere-auth

b69e83c

Signed-off-by: Sam <[email protected]>

style: make linter happy

4b1ad20

Signed-off-by: Samantha Coyle <[email protected]>

style: clean up logs

ce94dfe

Signed-off-by: Samantha Coyle <[email protected]>

style: more linter things and adjust for mocking client

30de3db

Signed-off-by: Samantha Coyle <[email protected]>

fix: make 1 hr default timeout

f8e3567

Signed-off-by: Samantha Coyle <[email protected]>

fix: update more tests

3e6a471

Signed-off-by: Samantha Coyle <[email protected]>

fix: address final feedback

bb22450

Signed-off-by: Samantha Coyle <[email protected]>

fix(conformance): try to inject mocked creds for session

5a17558

Signed-off-by: Samantha Coyle <[email protected]>

sicoyle added 3 commits November 12, 2024 18:01

style: make linter happy

72fe803

Signed-off-by: Samantha Coyle <[email protected]>

fix: go back on conformance test changes

d4f23ba

Signed-off-by: Samantha Coyle <[email protected]>

fix: try this for conformance

67ed5ec

Signed-off-by: Samantha Coyle <[email protected]>

fix: update default to be one hour with timeout

37021c5

Signed-off-by: Samantha Coyle <[email protected]>

sicoyle added 2 commits November 13, 2024 15:37

style: session duration can default to 1h so not required

b081bfe

Signed-off-by: Samantha Coyle <[email protected]>

Update builtin-authentication-profiles.yaml

6263e19

Signed-off-by: Sam <[email protected]>

JoshVanL requested changes Nov 14, 2024

View reviewed changes

sicoyle added 2 commits November 14, 2024 09:47

fix: address final feedback

e6f7699

Signed-off-by: Samantha Coyle <[email protected]>

fix: address final feedback

3d4da39

Signed-off-by: Samantha Coyle <[email protected]>

sicoyle requested a review from JoshVanL November 14, 2024 15:49

sicoyle added 3 commits November 14, 2024 10:19

style: make linter happy

a300a8c

Signed-off-by: Samantha Coyle <[email protected]>

fix: allow for mocked clients without exported field

0b80a39

Signed-off-by: Samantha Coyle <[email protected]>

fix: add one last closer

cde5a10

Signed-off-by: Samantha Coyle <[email protected]>

JoshVanL approved these changes Nov 14, 2024

View reviewed changes

yaron2 approved these changes Nov 14, 2024

View reviewed changes

yaron2 merged commit a00a853 into dapr:main Nov 14, 2024

sicoyle mentioned this pull request Dec 4, 2024

docs(aws): revamp aws docs + iam role anywhere profile dapr/docs#4456

Merged

7 tasks

Uh oh!

Conversation

sicoyle commented Nov 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issue reference

Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sicoyle commented Nov 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dapr-bot commented Nov 13, 2024

Complete Build Matrix

Uh oh!

dapr-bot commented Nov 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Components certification test

❌ Some certification tests failed

Uh oh!

dapr-bot commented Nov 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Components conformance test

❌ Some conformance tests failed

Uh oh!

sicoyle commented Nov 13, 2024

Uh oh!

dapr-bot commented Nov 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Components conformance test

❌ Some conformance tests failed

Uh oh!

dapr-bot commented Nov 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Components certification test

❌ Some certification tests failed

Uh oh!

dapr-bot commented Nov 13, 2024

Complete Build Matrix

Uh oh!

sicoyle commented Nov 13, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sicoyle commented Nov 14, 2024

Uh oh!

dapr-bot commented Nov 14, 2024

Complete Build Matrix

Uh oh!

dapr-bot commented Nov 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Components certification test

❌ Some certification tests failed

Uh oh!

dapr-bot commented Nov 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Components conformance test

❌ Some conformance tests failed

Uh oh!

sicoyle commented Nov 14, 2024

sicoyle commented Nov 4, 2024 •

edited

Loading

sicoyle commented Nov 12, 2024 •

edited

Loading

dapr-bot commented Nov 13, 2024 •

edited

Loading

dapr-bot commented Nov 13, 2024 •

edited

Loading

dapr-bot commented Nov 13, 2024 •

edited

Loading

dapr-bot commented Nov 13, 2024 •

edited

Loading

dapr-bot commented Nov 14, 2024 •

edited

Loading

dapr-bot commented Nov 14, 2024 •

edited

Loading

dapr-bot commented Nov 14, 2024 •

edited

Loading

dapr-bot commented Nov 14, 2024 •

edited

Loading