auth: support the AWS container credentials endpoint (ECS/Fargate) in the external_account AWS source #7287

Is your feature request related to a problem? Please describe.

google-cloud-auth cannot use Workload Identity Federation from an Amazon ECS task, in particular on AWS Fargate. An external_account credential with an AWS credential_source never obtains the task's AWS credentials, so it cannot sign the GetCallerIdentity request that is exchanged for a Google token.

ECS and Fargate do not give containers credentials through the EC2 instance metadata service. They set the environment variable AWS_CONTAINER_CREDENTIALS_RELATIVE_URI and serve credentials from a link-local endpoint, http://169.254.170.2<relative uri>. The response is JSON with AccessKeyId, SecretAccessKey, Token and Expiration. This is documented in the ECS task IAM role documentation, and it is the mechanism the AWS SDKs use for ECS tasks. On Fargate there is no instance role endpoint to fall back to.

In src/auth/src/credentials/external_account_sources/aws_sourced.rs, resolve_credentials currently tries:

1. AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (and AWS_SESSION_TOKEN) from the environment.
2. The IMDS role endpoint (url in the credential source): request the role name, then request the credentials for that role.

Nothing reads AWS_CONTAINER_CREDENTIALS_RELATIVE_URI. I checked the source of google-cloud-auth 1.16.0, 1.17.0 and 1.18.0: none of them mention AWS_CONTAINER_CREDENTIALS or 169.254.170.2, and aws_sourced.rs is identical in 1.17.0 and 1.18.0. In an ECS task the credential source has no usable url, so resolution ends with "unable to determine the AWS metadata server security credentials endpoint".

The url field cannot be pointed at the ECS endpoint instead, for two reasons. The AWS source requests a role name first and then <url>/<role name>, while the ECS endpoint returns the credentials directly. And the relative URI contains a per-task identifier, so it cannot be written into a static credential configuration file.

Describe the solution you'd like

When no static credentials are set in the environment and AWS_CONTAINER_CREDENTIALS_RELATIVE_URI is set, read the credentials from http://169.254.170.2 plus that relative URI, before falling back to IMDS. This is the order the AWS SDKs use: environment, then the container credentials endpoint, then IMDS.

The response uses the same field names the AWS source already deserializes (AccessKeyId, SecretAccessKey, Token), so AwsSecurityCredentials can be reused. Unknown fields such as Expiration and RoleArn are ignored.

Details I would propose, open to your preferences:

• Make the container credentials host a field of AwsSourcedCredentials, defaulting to http://169.254.170.2, the same way the region, role and token URLs are fields today. Tests can then point it at a local httptest server, as test_resolve_credentials_imds does for IMDS.
• Optionally also support AWS_CONTAINER_CREDENTIALS_FULL_URI with AWS_CONTAINER_AUTHORIZATION_TOKEN (and AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE), which the AWS SDKs use for other container environments such as EKS Pod Identity. If you would rather keep the first change small, I can limit it to the relative URI and leave the full URI for a follow-up.
• Tests: the container endpoint is used when the variable is set; it takes priority over IMDS; static environment credentials still take priority over it; a non-success response returns an error.

I am happy to send the pull request. I would like to agree the design here first, as CONTRIBUTING.md asks.

Describe alternatives you've considered

• Exporting AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN before the process starts. The crate reads these, but ECS task credentials are short-lived and rotate, and an environment variable set at start-up is never refreshed, so the process fails once they expire.
• Pointing the AWS credential source url at the container endpoint. This does not work for the reasons above.
• Using a different credential source (file, url, executable). These return a ready-made subject token and do not use AWS credentials, so each needs a separate component to mint the token. That moves the AWS signing outside the library.
• Using a service account key. That removes the reason for using Workload Identity Federation.
• Carrying a local patch to the crate. This works, but every application that embeds the crate has to maintain its own fork.

Additional context

• An earlier issue, auth: support AWS container credentials endpoint (ECS/Fargate) in external_account AWS source #6029, asked for this and was closed on the day it was opened. It had no description and no linked change, and I could not find the behavior in any released version, so I am filing it again with the details.
• Applications that embed the crate hit this directly. In my case, a Rust gateway that calls Vertex AI from ECS Fargate through Workload Identity Federation needs the patch above to authenticate. It builds credentials through credentials::Builder::default() (Application Default Credentials) with an external_account file whose credential_source has environment_id: "aws1" and a regional_cred_verification_url.
• Version checked: google-cloud-auth 1.17.0 (and 1.18.0 for the source comparison). Platform: Linux container on AWS Fargate.