Skip to content
This repository was archived by the owner on Nov 5, 2024. It is now read-only.

AWS Targets Auth update#1243

Merged
tzununbekov merged 7 commits into
mainfrom
targets-iamrole
Dec 20, 2022
Merged

AWS Targets Auth update#1243
tzununbekov merged 7 commits into
mainfrom
targets-iamrole

Conversation

@tzununbekov

@tzununbekov tzununbekov commented Dec 14, 2022

Copy link
Copy Markdown
Member
  • IAM Role auth support in AWS targets:

    • Comprehend
    • DynamoDB
    • EventBridge
    • Kinesis
    • Lambda
    • S3
    • SNS
    • SQS

    Resolves Support IAM role auth in AWS targets #1238

  • BREAKING CHANGE: All AWS target CRDs are updated to match AWS Sources' auth structure.
    Should be mentioned in the release note when merged (cc @sameersbn).

    Before:

    apiVersion: targets.triggermesh.io/v1alpha1
    kind: AWS*
    spec:
      ...
      awsApiKey:
        secretKeyRef:
          name: aws
          key: AWS_ACCESS_KEY_ID
      awsApiSecret:
        secretKeyRef:
          name: aws
          key: AWS_SECRET_ACCESS_KEY
    

    After:

    apiVersion: targets.triggermesh.io/v1alpha1
    kind: AWS*
    spec:
      ...
      auth:
        credentials:
          accessKeyID:
            valueFromSecret:
              name: aws
              key: AWS_ACCESS_KEY_ID
          secretAccessKey:
            valueFromSecret:
              name: aws
              key: AWS_SECRET_ACCESS_KEY
    

    Hence, IAM role auth is now available as:

    apiVersion: targets.triggermesh.io/v1alpha1
    kind: AWS*
    spec:
      ...
      auth:
        iamRole: arn:aws:iam::<redacted>:role/<role-name>
    

    Resolves AWS Spec Credentials Should Match in Targets and Sources #1114


Note for Docs (cc @jmcx)

After this PR is merged and released, all AWS Target docs and samples should be updated according to the "Before/After" example above.

Here is a draft "EKS IAM Role auth" instruction:

TriggerMesh AWS Sources and Targets on EKS support authentication based on IAM Roles. Authentication configuration consists of two simple steps: enable OIDC provider for EKS cluster and create IAM Role. There are several different ways to make these configuration steps, but in this instruction, we'll go with the CLI tools: aws and eksctl (installation).

Enabling OIDC provider:

eksctl utils associate-iam-oidc-provider --cluster my-cluster --approve

We'll use AWS S3 target as an example for this instruction. All other targets are configured in the same manner with corresponding resource updates in IAM policies.

Create IAM policy file with full access to S3 bucket test-bucket:

cat >aws-s3-target-policy.json <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "*",
            "Resource": "arn:aws:s3:::test-bucket"
        }
    ]
}
EOF

Create the policy from this file:

aws iam create-policy --policy-name aws-s3-target-policy --policy-document file://aws-s3-target-policy.json

Set account variables:

account_id=$(aws sts get-caller-identity --query "Account" --output text)
oidc_provider=$(aws eks describe-cluster --name <my-cluster> --region <aws-region> --query "cluster.identity.oidc.issuer" --output text | sed -e "s/^https:\/\///")

-replace <my-cluster> and <aws-region> with your EKS cluster values.

Create role's trust relationship file:

cat >trust-relationship.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::$account_id:oidc-provider/$oidc_provider"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "$oidc_provider:aud": "sts.amazonaws.com"
        }
      }
    }
  ]
}
EOF

Create role:

aws iam create-role --role-name aws-s3-target-role --assume-role-policy-document file://trust-relationship.json --description "TriggerMesh AWS S3 Target role"

This command should return JSON with the role details; Arn field value is what we'll use in our target later on.

Attach the policy to the role:

aws iam attach-role-policy --role-name aws-s3-target-role --policy-arn=arn:aws:iam::$account_id:policy/aws-s3-target-policy

At this point, IAM configuration is done, and we can create AWS S3 Target using our new Role:

kubectl apply -f - <<EOF
apiVersion: targets.triggermesh.io/v1alpha1
kind: AWSS3Target
metadata:
  name: triggermesh-aws-s3-test
spec:
  adapterOverrides:
    public: true
  arn: arn:aws:s3:::test-bucket
  auth:
    iamRole: <role arn>
EOF

After S3 target is created, it can be used as a Sink for other event sources or accessed directly via a public URL if the network's external endpoints are configured.

The official document used for this instruction is here.
The configuration process (IAM policies, roles, resources) may vary depending on the AWS target used and cluster security requirements.

@tzununbekov tzununbekov self-assigned this Dec 14, 2022
@tzununbekov tzununbekov changed the title AWS targets EKS IAM support AWS Targets Auth update Dec 16, 2022
@tzununbekov tzununbekov marked this pull request as ready for review December 20, 2022 09:40
@tzununbekov tzununbekov requested a review from a team December 20, 2022 09:40
Comment thread config/samples/targets/aws/200-aws-dynamodb-target.yaml
Comment thread pkg/targets/reconciler/aws.go Outdated

@odacremolbap odacremolbap left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is mergeable.
PTAL at the 2 comments, none of them blockers.

@tzununbekov tzununbekov merged commit bc2a3aa into main Dec 20, 2022
@tzununbekov tzununbekov deleted the targets-iamrole branch December 20, 2022 13:43
@jmcx

jmcx commented Dec 21, 2022

Copy link
Copy Markdown
Contributor

Thanks @tzununbekov for the clear explanations and input for the docs.

@jmcx

jmcx commented Apr 27, 2023

Copy link
Copy Markdown
Contributor

@FranBarrera as discussed with the team, can you share the missing piece of information about adding an annotation to the TriggerMesh service account? (for components that require reconciliation on the controller side, when using IAM role). That way we can mention it in the docs.

@FranBarrera

Copy link
Copy Markdown
Contributor

@jmcx Yes, the missing piece is to add an annotation to our triggermesh-controller service account and restart the triggermesh-controller pod:

To do that users could use these commands:

kubectl -n triggermesh patch sa triggermesh-controller \
  --type merge \
  --patch '{"metadata":{"annotations":{"eks.amazonaws.com/role-arn": "<IAM_ROLE_ARN>"}}}'
kubectl -n triggermesh rollout restart deployment triggermesh-controller

@FranBarrera

Copy link
Copy Markdown
Contributor

@jmcx Note for docs:
There is an error on this command:

cat >trust-relationship.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::$account_id:oidc-provider/$oidc_provider"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "$oidc_provider:aud": "sts.amazonaws.com"
        }
      }
    }
  ]
}
EOF

It should be like this:

cat >trust-relationship.json <<EOF
{
 "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
       "Federated": "arn:aws:iam::$account_id:oidc-provider/$oidc_provider"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "${oidc_provider}:aud": "sts.amazonaws.com"
        }
      }
    }
  ]
}
EOF

@tzununbekov

Copy link
Copy Markdown
Member Author

Curly braces in shell scripts are optional unless parameter expansion is used. If you prefer it this way, then for the consistency update the script to make all variables look the same.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support IAM role auth in AWS targets AWS Spec Credentials Should Match in Targets and Sources

4 participants