Skip to content

fix(bedrock): surface web identity token aud/iss on InvalidIdentityToken#31412

Merged
yassin-berriai merged 1 commit into
litellm_internal_stagingfrom
litellm_bedrock_web_identity_audience_error
Jun 26, 2026
Merged

fix(bedrock): surface web identity token aud/iss on InvalidIdentityToken#31412
yassin-berriai merged 1 commit into
litellm_internal_stagingfrom
litellm_bedrock_web_identity_audience_error

Conversation

@yassin-berriai

Copy link
Copy Markdown
Contributor

Relevant issues

Resolves LIT-4026

Linear ticket

LIT-4026

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

A customer connecting Bedrock through an AWS IAM role with GCP OIDC (AssumeRoleWithWebIdentity) hit InvalidIdentityToken: Incorrect token audience. The whole investigation stalled on one question: what aud is LiteLLM actually putting in the token? The only way to find out was LITELLM_LOG=DEBUG on the prod instance, which degrades performance, so the answer stayed unknown for weeks.

This surfaces the token's aud and iss in the error itself, no debug logging required. The proof below runs a live proxy that makes a real AssumeRoleWithWebIdentity call against AWS STS (the call needs no AWS credentials; STS validates the token itself), so the same InvalidIdentityTokenException class and the same base_aws_llm.py code path as the customer's traceback are exercised end to end. The token here carries aud=https://guidepoint.litellm-prod.ai, the value the customer configured.

Config used (aws_web_identity_token resolves a JWT whose aud is https://guidepoint.litellm-prod.ai):

model_list:
  - model_name: bedrock-claude
    litellm_params:
      model: bedrock/us.anthropic.claude-sonnet-4-6-20250101-v1:0
      aws_web_identity_token: oidc/env/GUIDEPOINT_WEB_IDENTITY_TOKEN
      aws_role_name: arn:aws:iam::123456789012:role/litellm-bedrock-role
      aws_session_name: litellm-guidepoint
      aws_region_name: us-east-1
general_settings:
  master_key: sk-1234

Command (identical before and after):

curl -s -X POST http://127.0.0.1:4026/v1/chat/completions \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"model":"bedrock-claude","messages":[{"role":"user","content":"hi"}]}'

Before (opaque; raw botocore error, no audience anywhere, returned as a 500 APIConnectionError):

litellm.APIConnectionError: An error occurred (InvalidIdentityToken) when calling the
AssumeRoleWithWebIdentity operation: Couldn't retrieve verification key from your identity
provider,  please reference AssumeRoleWithWebIdentity documentation for requirements
  ...
  File ".../litellm/llms/bedrock/base_aws_llm.py", line 928, in _auth_with_web_identity_token
    sts_response = sts_client.assume_role_with_web_identity(**assume_role_params)
botocore.errorfactory.InvalidIdentityTokenException: ... Incorrect token audience

After (the STS reason is preserved and the token now names its own audience and issuer, as a 401 AuthenticationError):

litellm.AuthenticationError: BedrockException - AWS STS rejected the web identity token:
An error occurred (InvalidIdentityToken) when calling the AssumeRoleWithWebIdentity operation:
Couldn't retrieve verification key from your identity provider,  please reference
AssumeRoleWithWebIdentity documentation for requirements.
Token aud='https://guidepoint.litellm-prod.ai', iss='https://accounts.google.com'.
Received Model Group=bedrock-claude

The crafted token here is not validly Google-signed, so AWS reports the verification-key subreason rather than the audience subreason; the exception class, the code path and the audience surfacing are exactly what the customer's "Incorrect token audience" case exercises. An operator now reads the aud straight off the failure and compares it to the IAM OIDC provider's client_id_list and the role trust policy condition, with no debug logging on prod.

Type

🐛 Bug Fix

Changes

_auth_with_web_identity_token wraps the assume_role_with_web_identity call and catches InvalidIdentityTokenException. On rejection it decodes the public aud/iss claims of the already-resolved JWT (base64url of the payload segment only; the signature is never read, so no secret is exposed) and raises an AwsAuthError that keeps the original STS reason and appends the token's audience and issuer. The decode is failure-path only, so there is no cost on the happy path; opaque or non-JWT tokens degrade gracefully to the original reason with no audience appended.

This is the single funnel for the bedrock OIDC auth flow: chat (converse and invoke), embeddings, image and rerank all reach it through get_credentials, so they are all covered.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@yassin-berriai

Copy link
Copy Markdown
Contributor Author

@greptileai

@greptile-apps

greptile-apps Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR improves the debugging experience for Bedrock OIDC auth failures by surfacing the JWT's aud and iss claims directly in the error message when AWS STS rejects a web identity token with InvalidIdentityTokenException.

  • Adds _unverified_web_identity_audience — a static helper that base64url-decodes the JWT payload segment (no signature read) and extracts aud/iss via a Pydantic model, returning None gracefully on any parse error so non-JWT tokens degrade silently.
  • Wraps sts_client.assume_role_with_web_identity in a try/except for InvalidIdentityTokenException, re-raising as AwsAuthError(401) with the STS reason preserved and the token's audience/issuer appended.
  • Adds four targeted unit tests that mock boto3 and get_secret, covering the 401 status code, aud, iss, and original STS message all appearing in the raised error.

Confidence Score: 5/5

Safe to merge. The change is purely additive on the failure path and cannot affect the happy path.

The new code only runs when STS has already rejected the token, so it cannot change credentials-retrieval behavior on success. The JWT decode is guarded by a broad except (ValueError, ValidationError) with a None fallback, meaning any malformed or non-JWT token degrades silently to the original error message. aud and iss are public JWT claims; no signature material or secret is exposed. The four new unit tests are fully mocked and directly exercise the claims-surfacing logic end-to-end.

No files require special attention.

Important Files Changed

Filename Overview
litellm/llms/bedrock/base_aws_llm.py Adds _unverified_web_identity_audience to extract aud/iss claims from the JWT payload (no signature read), then wraps the assume_role_with_web_identity call to catch InvalidIdentityTokenException and raise an AwsAuthError(401) that includes those claims. Implementation is correct: base64url decoding with padding fix, Pydantic validation, graceful None fallback on any parse error.
tests/test_litellm/llms/bedrock/test_web_identity_session_policy.py Adds TestInvalidIdentityTokenSurfacesAudience with four focused unit tests that mock boto3 and get_secret. Tests verify the 401 status, presence of aud, iss, and the original STS reason in the error. No real network calls; fully mock-based.

Reviews (1): Last reviewed commit: "fix(bedrock): surface web identity token..." | Re-trigger Greptile

@greptile-apps

greptile-apps Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

When AssumeRoleWithWebIdentity fails with InvalidIdentityTokenException, LiteLLM previously surfaced an opaque botocore error with no information about the token's audience, forcing operators to enable debug logging on production to diagnose the mismatch. This PR wraps that STS call and, on failure, decodes the public aud/iss claims from the JWT payload (signature never read) to include them in a structured AwsAuthError(401).

  • _unverified_web_identity_audience: new static helper that base64url-decodes the JWT payload segment, validates it with a tight _WebIdentityTokenClaims Pydantic model, and returns the aud/iss formatted for the error message; returns None gracefully for non-JWT or malformed tokens.
  • The STS call is now wrapped in a try/except InvalidIdentityTokenException that raises the new structured error; all other exceptions propagate unchanged, and the happy path has no overhead.
  • Four mock-only unit tests cover the audience string, issuer string, original STS reason preservation, and 401 status code.

Confidence Score: 5/5

Safe to merge — the change only affects the error path; the happy path is untouched and carries no new overhead.

The new code is entirely on the failure branch of _auth_with_web_identity_token. _unverified_web_identity_audience decodes only the public payload segment, catches all expected decode errors (ValueError, ValidationError), and returns None rather than throwing when the token isn't a well-formed JWT. The exception handler is narrowly scoped to InvalidIdentityTokenException; all other STS errors propagate as before. Four targeted unit tests with full mocking confirm the aud/iss surfacing, reason preservation, and status code.

No files require special attention.

Important Files Changed

Filename Overview
litellm/llms/bedrock/base_aws_llm.py Adds _unverified_web_identity_audience helper to decode public JWT claims for diagnostics, and wraps the STS call to catch InvalidIdentityTokenException and raise a structured AwsAuthError(401) that includes the token's aud/iss. Logic is correct, gracefully degrades for non-JWT tokens, and is failure-path only (zero overhead on the happy path).
tests/test_litellm/llms/bedrock/test_web_identity_session_policy.py Adds TestInvalidIdentityTokenSurfacesAudience with four focused mock-only unit tests covering the aud/iss surfacing, original STS reason preservation, and 401 status code. Uses patch for boto3.client and get_secret — no real network calls, consistent with the existing test pattern in this file.

Reviews (2): Last reviewed commit: "fix(bedrock): surface web identity token..." | Re-trigger Greptile

@codecov

codecov Bot commented Jun 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.00000% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
litellm/llms/bedrock/base_aws_llm.py 84.00% 4 Missing ⚠️

📢 Thoughts on this report? Let us know!

When STS rejects a web identity token with InvalidIdentityToken (the
"Incorrect token audience" case), litellm propagated the raw botocore
error, which never names the aud LiteLLM actually sent. Diagnosing an
audience mismatch then required enabling LITELLM_LOG=DEBUG on the prod
instance, which degrades performance.

_auth_with_web_identity_token now catches InvalidIdentityTokenException,
decodes the public aud/iss claims of the resolved JWT without verifying
its signature (no secret is read), and raises an AwsAuthError that
preserves the STS reason and names the token audience and issuer, so the
mismatch is visible from the error alone.

Resolves LIT-4026
@yassin-berriai yassin-berriai force-pushed the litellm_bedrock_web_identity_audience_error branch from 7efa354 to dc2e2fd Compare June 26, 2026 06:46
@yassin-berriai yassin-berriai enabled auto-merge (squash) June 26, 2026 11:43
@yassin-berriai yassin-berriai merged commit bd5e046 into litellm_internal_staging Jun 26, 2026
122 checks passed
@yassin-berriai yassin-berriai deleted the litellm_bedrock_web_identity_audience_error branch June 26, 2026 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants