feat(postgres): add iam roles anywhere auth profile by sicoyle · Pull Request #3604 · dapr/components-contrib

sicoyle · 2024-11-15T23:25:08Z

Description

Add IAM Roles Anywhere auth profile to PostgreSQL components.

I saw where we've had fields like awsAccessKey and awsSecretKey fields specific to postgres. However, in the AWS built in profiles for authentication we have accessKey and secretKey. I've moved things to where we have some TODO comments to remove those aws prefixed fields in Dapr 1.17; however, to not break existing users, we cannot just remove these additional fields. Due to this, I had to manually manipulate the metadata bundle even further unfortunately. This is something we should never do. However, again in the case of the auth profile titled "AWS: Access Key ID and Secret Access Key" I am forced to update the manipulations further to make the normally required accessKey and secretKey fields as non-required as we make a transition in the next two releases to the normalized AWS auth profile fields without the aws prefix.

AWS Auth profiles with these changes:


{
          "title": "AWS: Access Key ID and Secret Access Key",
          "description": "Authenticate using an Access Key ID and Secret Access Key included in the metadata",
          "metadata": [
            {
              "name": "useAWSIAM",
              "description": "Must be set to `true` to enable the component to retrieve access tokens from AWS IAM.\nThis authentication method only works with AWS Relational Database Service for PostgreSQL databases.",
              "required": true,
              "type": "bool",
              "example": "\"true\""
            },
            {
              "name": "connectionString",
              "description": "The connection string for the PostgreSQL database\nThis must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the AWS IAM policy. This connection string should not contain any password. Note that the database name field is denoted by dbname with AWS.",
              "required": true,
              "sensitive": true,
              "type": "string",
              "example": "\"host=mydb.postgres.database.aws.com user=myapplication port=5432 dbname=dapr_test sslmode=require\"\n"
            },
            {
              "name": "awsAccessKey",
              "description": "Deprecated as of Dapr 1.17. Use 'accessKey' instead if using AWS IAM.\nIf both fields are set, then 'accessKey' value will be used.\nAWS access key associated with an IAM account.",
              "type": "string",
              "example": "\"AKIAIOSFODNN7EXAMPLE\""
            },
            {
              "name": "awsSecretKey",
              "description": "Deprecated as of Dapr 1.17. Use 'secretKey' instead if using AWS IAM.\nIf both fields are set, then 'secretKey' value will be used.\nThe secret key associated with the access key.",
              "sensitive": true,
              "type": "string",
              "example": "\"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\""
            },
            {
              "name": "awsRegion",
              "description": "The AWS Region where the AWS resource is deployed to.",
              "required": true,
              "type": "string",
              "example": "\"us-east-1\""
            },
            {
              "name": "accessKey",
              "description": "AWS access key associated with an IAM account",
              "sensitive": true,
              "example": "\"AKIAIOSFODNN7EXAMPLE\""
            },
            {
              "name": "secretKey",
              "description": "The secret key associated with the access key",
              "sensitive": true,
              "example": "\"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\""
            },
            {
              "name": "sessionToken",
              "description": "AWS session token to use. A session token is only required if you are using\ntemporary security credentials.",
              "sensitive": true,
              "type": "string",
              "example": "\"TOKEN\""
            }
          ]
        },
        {
          "title": "AWS: Credentials from Environment Variables",
          "description": "Use AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY from the environment",
          "metadata": [
            {
              "name": "useAWSIAM",
              "description": "Must be set to `true` to enable the component to retrieve access tokens from AWS IAM.\nThis authentication method only works with AWS Relational Database Service for PostgreSQL databases.",
              "required": true,
              "type": "bool",
              "example": "\"true\""
            },
            {
              "name": "connectionString",
              "description": "The connection string for the PostgreSQL database\nThis must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the AWS IAM policy. This connection string should not contain any password. Note that the database name field is denoted by dbname with AWS.",
              "required": true,
              "sensitive": true,
              "type": "string",
              "example": "\"host=mydb.postgres.database.aws.com user=myapplication port=5432 dbname=dapr_test sslmode=require\"\n"
            }
          ]
        },
        {
          "title": "AWS: IAM Roles Anywhere",
          "description": "Use X.509 certificates to establish trust between AWS and your AWS account and the Dapr cluster using AWS IAM Roles Anywhere.",
          "metadata": [
            {
              "name": "useAWSIAM",
              "description": "Must be set to `true` to enable the component to retrieve access tokens from AWS IAM.\nThis authentication method only works with AWS Relational Database Service for PostgreSQL databases.",
              "required": true,
              "type": "bool",
              "example": "\"true\""
            },
            {
              "name": "connectionString",
              "description": "The connection string for the PostgreSQL database\nThis must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the AWS IAM policy. This connection string should not contain any password. Note that the database name field is denoted by dbname with AWS.",
              "required": true,
              "sensitive": true,
              "type": "string",
              "example": "\"host=mydb.postgres.database.aws.com user=myapplication port=5432 dbname=dapr_test sslmode=require\"\n"
            },
            {
              "name": "trustAnchorArn",
              "description": "ARN of the AWS Trust Anchor in the AWS account granting trust to the Dapr Certificate Authority.",
              "required": true,
              "example": "arn:aws:rolesanywhere:us-west-1:012345678910:trust-anchor/01234568-0123-0123-0123-012345678901"
            },
            {
              "name": "trustProfileArn",
              "description": "ARN of the AWS IAM Profile in the trusting AWS account.",
              "required": true,
              "example": "arn:aws:rolesanywhere:us-west-1:012345678910:profile/01234568-0123-0123-0123-012345678901"
            },
            {
              "name": "assumeRoleArn",
              "description": "ARN of the AWS IAM role to assume in the trusting AWS account.",
              "required": true,
              "example": "arn:aws:iam:012345678910:role/exampleIAMRoleName"
            }
          ]
        }

Issue reference

We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.

Please reference the issue this PR will close: #[issue number]

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

Code compiles correctly
Created/updated tests
Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

Signed-off-by: Samantha Coyle <[email protected]>

sicoyle · 2024-11-19T01:57:05Z

/ok-to-test

dapr-bot · 2024-11-19T01:57:36Z

Complete Build Matrix

The build status is currently not updated here. Please visit the action run below directly.

🔗 Link to Action run

Commit ref: 0e4401c

dapr-bot · 2024-11-19T01:57:38Z

Components conformance test

🔗 Link to Action run

Commit ref: 0e4401c

❌ Some conformance tests failed

These tests failed:

bindings.azure.eventgrid
state.sqlserver

dapr-bot · 2024-11-19T01:57:40Z

Components certification test

🔗 Link to Action run

Commit ref: 0e4401c

❌ Some certification tests failed

These tests failed:

bindings.azure.eventhubs
pubsub.gcp.pubsub
state.azure.cosmosdb

famarting · 2024-11-19T13:57:13Z

+func (a *StaticAuth) getDatabaseToken(ctx context.Context, poolConfig *pgxpool.Config) (string, error) {
+	dbEndpoint := poolConfig.ConnConfig.Host + ":" + strconv.Itoa(int(poolConfig.ConnConfig.Port))
+	switch {
+	case a.accessKey != nil && a.secretKey != nil:


rather than this I would do a simple if statement, much simpler to read

famarting · 2024-11-19T13:59:31Z

so this PR is missing the assume role auth profile, right?

sicoyle · 2024-11-20T16:31:03Z

https://github.com/dapr/components-contrib/pull/3607/files
yes, didn't want to bring in a bunch into this 1 PR, but after I get the kafka PR merged then this will be an easy add here.

Pls give me a sec on this PR bc I do need to make tweaks, but doing the kafka PR first.

Signed-off-by: Samantha Coyle <[email protected]>

sicoyle · 2024-12-03T17:06:21Z

so this PR is missing the assume role auth profile, right?

This PR includes the ability to assume a role in the auth profile

Signed-off-by: Samantha Coyle <[email protected]>

sicoyle · 2024-12-03T17:25:29Z

latest metadata for postgres components aws auth profiles


 {
          "title": "AWS: Access Key ID and Secret Access Key",
          "description": "Authenticate using an Access Key ID and Secret Access Key included in the metadata",
          "metadata": [
            {
              "name": "useAWSIAM",
              "description": "Must be set to `true` to enable the component to retrieve access tokens from AWS IAM.\nThis authentication method only works with AWS Relational Database Service for PostgreSQL databases.",
              "required": true,
              "type": "bool",
              "example": "\"true\""
            },
            {
              "name": "connectionString",
              "description": "The connection string for the PostgreSQL database\nThis must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the AWS IAM policy. This connection string should not contain any password. Note that the database name field is denoted by dbname with AWS.",
              "required": true,
              "sensitive": true,
              "type": "string",
              "example": "\"host=mydb.postgres.database.aws.com user=myapplication port=5432 dbname=dapr_test sslmode=require\"\n"
            },
            {
              "name": "awsAccessKey",
              "description": "Deprecated as of Dapr 1.17. Use 'accessKey' instead if using AWS IAM.\nIf both fields are set, then 'accessKey' value will be used.\nAWS access key associated with an IAM account.",
              "type": "string",
              "example": "\"AKIAIOSFODNN7EXAMPLE\""
            },
            {
              "name": "awsSecretKey",
              "description": "Deprecated as of Dapr 1.17. Use 'secretKey' instead if using AWS IAM.\nIf both fields are set, then 'secretKey' value will be used.\nThe secret key associated with the access key.",
              "sensitive": true,
              "type": "string",
              "example": "\"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\""
            },
            {
              "name": "awsRegion",
              "description": "This maintains backwards compatibility with existing fields. \nIt will be deprecated as of Dapr 1.17. Use 'region' instead.\nThe AWS Region where the AWS service is deployed to.",
              "type": "string",
              "example": "\"us-east-1\""
            },
            {
              "name": "region",
              "description": "The AWS Region where the AWS resource is deployed to.",
              "type": "string",
              "example": "\"us-east-1\""
            },
            {
              "name": "accessKey",
              "description": "AWS access key associated with an IAM account",
              "sensitive": true,
              "example": "\"AKIAIOSFODNN7EXAMPLE\""
            },
            {
              "name": "secretKey",
              "description": "The secret key associated with the access key",
              "sensitive": true,
              "example": "\"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\""
            },
            {
              "name": "sessionToken",
              "description": "AWS session token to use. A session token is only required if you are using\ntemporary security credentials.",
              "sensitive": true,
              "example": ""
            }
          ]
        },
        {
          "title": "AWS: Assume IAM Role",
          "description": "Assume a specific IAM role. Note: This is only supported for Kafka and PostgreSQL.",
          "metadata": [
            {
              "name": "useAWSIAM",
              "description": "Must be set to `true` to enable the component to retrieve access tokens from AWS IAM.\nThis authentication method only works with AWS Relational Database Service for PostgreSQL databases.",
              "required": true,
              "type": "bool",
              "example": "\"true\""
            },
            {
              "name": "connectionString",
              "description": "The connection string for the PostgreSQL database\nThis must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the AWS IAM policy. This connection string should not contain any password. Note that the database name field is denoted by dbname with AWS.",
              "required": true,
              "sensitive": true,
              "type": "string",
              "example": "\"host=mydb.postgres.database.aws.com user=myapplication port=5432 dbname=dapr_test sslmode=require\"\n"
            },
            {
              "name": "region",
              "description": "The AWS Region where the AWS resource is deployed to.",
              "type": "string",
              "example": "\"us-east-1\""
            },
            {
              "name": "assumeRoleArn",
              "description": "IAM role that has access to AWS resource.\nThis is another option to authenticate with MSK and RDS Aurora aside from the AWS Credentials.",
              "type": "string",
              "example": "\"arn:aws:iam::123456789:role/mskRole\""
            },
            {
              "name": "sessionName",
              "description": "The session name for assuming a role.",
              "type": "string",
              "default": "\"DaprDefaultSession\"",
              "example": "\"MyAppSession\""
            }
          ]
        },
        {
          "title": "AWS: Credentials from Environment Variables",
          "description": "Use AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY from the environment",
          "metadata": [
            {
              "name": "useAWSIAM",
              "description": "Must be set to `true` to enable the component to retrieve access tokens from AWS IAM.\nThis authentication method only works with AWS Relational Database Service for PostgreSQL databases.",
              "required": true,
              "type": "bool",
              "example": "\"true\""
            },
            {
              "name": "connectionString",
              "description": "The connection string for the PostgreSQL database\nThis must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the AWS IAM policy. This connection string should not contain any password. Note that the database name field is denoted by dbname with AWS.",
              "required": true,
              "sensitive": true,
              "type": "string",
              "example": "\"host=mydb.postgres.database.aws.com user=myapplication port=5432 dbname=dapr_test sslmode=require\"\n"
            }
          ]
        },
        {
          "title": "AWS: IAM Roles Anywhere",
          "description": "Use X.509 certificates to establish trust between AWS and your AWS account and the Dapr cluster using AWS IAM Roles Anywhere.",
          "metadata": [
            {
              "name": "useAWSIAM",
              "description": "Must be set to `true` to enable the component to retrieve access tokens from AWS IAM.\nThis authentication method only works with AWS Relational Database Service for PostgreSQL databases.",
              "required": true,
              "type": "bool",
              "example": "\"true\""
            },
            {
              "name": "connectionString",
              "description": "The connection string for the PostgreSQL database\nThis must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the AWS IAM policy. This connection string should not contain any password. Note that the database name field is denoted by dbname with AWS.",
              "required": true,
              "sensitive": true,
              "type": "string",
              "example": "\"host=mydb.postgres.database.aws.com user=myapplication port=5432 dbname=dapr_test sslmode=require\"\n"
            },
            {
              "name": "trustAnchorArn",
              "description": "ARN of the AWS Trust Anchor in the AWS account granting trust to the Dapr Certificate Authority.",
              "required": true,
              "example": "arn:aws:rolesanywhere:us-west-1:012345678910:trust-anchor/01234568-0123-0123-0123-012345678901"
            },
            {
              "name": "trustProfileArn",
              "description": "ARN of the AWS IAM Profile in the trusting AWS account.",
              "required": true,
              "example": "arn:aws:rolesanywhere:us-west-1:012345678910:profile/01234568-0123-0123-0123-012345678901"
            },
            {
              "name": "assumeRoleArn",
              "description": "ARN of the AWS IAM role to assume in the trusting AWS account.",
              "required": true,
              "example": "arn:aws:iam:012345678910:role/exampleIAMRoleName"
            }
          ]
        }

Signed-off-by: Samantha Coyle <[email protected]>

sicoyle · 2024-12-03T17:37:20Z

/ok-to-test

dapr-bot · 2024-12-03T17:38:13Z

Components conformance test

🔗 Link to Action run

Commit ref: faf743b

❌ Some conformance tests failed

These tests failed:

bindings.azure.eventgrid
state.azure.cosmosdb

dapr-bot · 2024-12-03T17:38:15Z

Components certification test

🔗 Link to Action run

Commit ref: faf743b

❌ Some certification tests failed

These tests failed:

pubsub.aws.snssqs
pubsub.gcp.pubsub
state.azure.cosmosdb
state.sqlserver

dapr-bot · 2024-12-03T17:38:43Z

Complete Build Matrix

The build status is currently not updated here. Please visit the action run below directly.

🔗 Link to Action run

Commit ref: faf743b

sicoyle · 2024-12-03T18:36:42Z

this is ready pls. Conformance/certification test failures are unrelated

sicoyle · 2024-12-03T18:40:37Z

/ok-to-test

dapr-bot · 2024-12-03T18:41:00Z

Complete Build Matrix

The build status is currently not updated here. Please visit the action run below directly.

🔗 Link to Action run

Commit ref: faf743b

dapr-bot · 2024-12-03T18:41:02Z

Components certification test

🔗 Link to Action run

Commit ref: faf743b

❌ Some certification tests failed

These tests failed:

bindings.azure.eventhubs
pubsub.aws.snssqs
pubsub.gcp.pubsub
pubsub.mqtt3
pubsub.rabbitmq
state.azure.cosmosdb
state.memcached
state.sqlserver

dapr-bot · 2024-12-03T18:41:02Z

Components conformance test

🔗 Link to Action run

Commit ref: faf743b

❌ Some conformance tests failed

These tests failed:

bindings.azure.eventgrid
bindings.azure.eventhubs
state.azure.cosmosdb

Signed-off-by: Samantha Coyle <[email protected]> Co-authored-by: Yaron Schneider <[email protected]>

Signed-off-by: Samantha Coyle <[email protected]> Co-authored-by: Yaron Schneider <[email protected]> # Conflicts: # .build-tools/pkg/metadataschema/builtin-authentication-profiles.go # bindings/kafka/metadata.yaml # common/authentication/aws/static.go # common/authentication/aws/x509.go # common/authentication/postgresql/metadata.go # pubsub/kafka/metadata.yaml

Signed-off-by: Samantha Coyle <[email protected]> Co-authored-by: Yaron Schneider <[email protected]>

sicoyle added 2 commits November 15, 2024 16:39

feat(iam roles anywhere): add support for postgres components

fa08cbc

Signed-off-by: Samantha Coyle <[email protected]>

fix(metadata): acct for manipulated fields properly

3f52321

Signed-off-by: Samantha Coyle <[email protected]>

sicoyle requested review from a team as code owners November 15, 2024 23:25

sicoyle changed the title ~~Feat iam roles anywhere postgres~~ feat(postgres): add iam roles anywhere auth profile Nov 15, 2024

sicoyle added 4 commits November 15, 2024 17:45

fix: rm duplicate field on regions in the metadata

6c4f42d

Signed-off-by: Samantha Coyle <[email protected]>

style: make linter happy

5cffcf1

Signed-off-by: Samantha Coyle <[email protected]>

fix: update close funcs

d867910

Signed-off-by: Samantha Coyle <[email protected]>

Merge branch 'main' into feat-iam-roles-anywhere-postgres

0e4401c

famarting reviewed Nov 19, 2024

View reviewed changes

yaron2 and others added 6 commits November 26, 2024 08:45

Merge branch 'main' into feat-iam-roles-anywhere-postgres

b6c85f4

fix: address merge conflicts

ad47876

Signed-off-by: Samantha Coyle <[email protected]>

fix: address merge conflicts

5056e3b

Signed-off-by: Samantha Coyle <[email protected]>

Merge branch 'main' into feat-iam-roles-anywhere-postgres

fb1ee5f

fix: update from changes in main

8bc140b

Signed-off-by: Samantha Coyle <[email protected]>

fix: clean up and add assume role capability

a68577f

Signed-off-by: Samantha Coyle <[email protected]>

fix: make sure metadata bundle is good

97493ea

Signed-off-by: Samantha Coyle <[email protected]>

sicoyle added 3 commits December 3, 2024 11:28

style: make linter happy

309efba

Signed-off-by: Samantha Coyle <[email protected]>

style: update default values

5b32347

Signed-off-by: Samantha Coyle <[email protected]>

fix: go mod tidy

faf743b

Signed-off-by: Samantha Coyle <[email protected]>

sicoyle requested a review from famarting December 3, 2024 20:53

famarting approved these changes Dec 3, 2024

View reviewed changes

yaron2 approved these changes Dec 3, 2024

View reviewed changes

yaron2 merged commit 72c92fb into dapr:main Dec 3, 2024

sicoyle mentioned this pull request Dec 4, 2024

docs(aws): revamp aws docs + iam role anywhere profile dapr/docs#4456

Merged

7 tasks

mikeee pushed a commit to mikeee/dapr_components-contrib that referenced this pull request Feb 13, 2025

feat(postgres): add iam roles anywhere auth profile (dapr#3604)

355921a

Signed-off-by: Samantha Coyle <[email protected]> Co-authored-by: Yaron Schneider <[email protected]>

passuied pushed a commit to passuied/components-contrib that referenced this pull request Feb 14, 2026

feat(postgres): add iam roles anywhere auth profile (dapr#3604)

a8fc83f

Signed-off-by: Samantha Coyle <[email protected]> Co-authored-by: Yaron Schneider <[email protected]>

Uh oh!

Conversation

sicoyle commented Nov 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issue reference

Checklist

Uh oh!

sicoyle commented Nov 19, 2024

Uh oh!

dapr-bot commented Nov 19, 2024

Complete Build Matrix

Uh oh!

dapr-bot commented Nov 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Components conformance test

❌ Some conformance tests failed

Uh oh!

dapr-bot commented Nov 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Components certification test

❌ Some certification tests failed

Uh oh!

famarting Nov 19, 2024

Choose a reason for hiding this comment

Uh oh!

famarting commented Nov 19, 2024

Uh oh!

sicoyle commented Nov 20, 2024

Uh oh!

sicoyle commented Dec 3, 2024

Uh oh!

sicoyle commented Dec 3, 2024

Uh oh!

sicoyle commented Dec 3, 2024

Uh oh!

dapr-bot commented Dec 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Components conformance test

❌ Some conformance tests failed

Uh oh!

dapr-bot commented Dec 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Components certification test

❌ Some certification tests failed

Uh oh!

dapr-bot commented Dec 3, 2024

Complete Build Matrix

Uh oh!

sicoyle commented Dec 3, 2024

Uh oh!

sicoyle commented Dec 3, 2024

Uh oh!

dapr-bot commented Dec 3, 2024

Complete Build Matrix

Uh oh!

dapr-bot commented Dec 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Components certification test

❌ Some certification tests failed

Uh oh!

dapr-bot commented Dec 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Components conformance test

❌ Some conformance tests failed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sicoyle commented Nov 15, 2024 •

edited

Loading

dapr-bot commented Nov 19, 2024 •

edited

Loading

dapr-bot commented Nov 19, 2024 •

edited

Loading

dapr-bot commented Dec 3, 2024 •

edited

Loading

dapr-bot commented Dec 3, 2024 •

edited

Loading

dapr-bot commented Dec 3, 2024 •

edited

Loading

dapr-bot commented Dec 3, 2024 •

edited

Loading