feat(credentials): vend AWS credentials via AssumeRoleWithWebIdentity to avoid role chaining by rpgreen · Pull Request #7757 · lance-format/lance

rpgreen · 2026-07-13T14:56:00Z

Problem

The credential vendor's api_key and static flows assume the target role with AssumeRole, signed by the process's ambient credentials. When the vendor runs with temporary credentials — e.g. an IRSA / EKS pod role — that is role chaining, which STS hard-caps to 1 hour regardless of the role's MaxSessionDuration.

In that setup the returned Expiration can also exceed the token's real validity, so downstream credential caches (StorageOptionsAccessor, CachingCredentialVendor) keep serving a token S3 already rejects with ExpiredToken and never refresh — their expiry checks trust the inflated expires_at_millis.

Change

Add an optional pod web-identity path. When pod_web_identity_token_file is configured — via property credential_vendor.aws_assume_via_pod_web_identity=true (resolving the EKS-injected AWS_WEB_IDENTITY_TOKEN_FILE), or an explicit credential_vendor.aws_pod_web_identity_token_file — the scoped assume uses AssumeRoleWithWebIdentity with the pod's projected service-account OIDC token.

AssumeRoleWithWebIdentity is a direct (non-chained) assumption, so STS honors the role's MaxSessionDuration (up to 12h) and reports an accurate expiration.

Per-table scoped session policy, permission handling, and the default chained AssumeRole behavior are unchanged.
The token file is re-read on every vend (kubelet rotates it).
Requires the target role's trust policy to federate the cluster OIDC provider for the service account.

This change also improves error handling to propagate the object store error message in namespace error messages.

Tests

test_config_builder extended for the new field.
test_pod_web_identity_path_reads_token_file — verifies the web-identity branch is selected and reads the token file.

🤖 Co-authored with Claude Opus 4.8

Summary by CodeRabbit

New Features
- Added AWS pod-based web identity authentication for credential vending.
- New configuration option to set the projected service-account token file path (with optional opt-in fallback to the standard AWS env var).
- When configured, scoped role vending uses web identity instead of the legacy chained assume-role flow.
Bug Fixes
- Preserves existing ambient-credential behavior when web identity is not configured.
Tests
- Added coverage to ensure vending fails with a clear token-file read error when the configured token file path is missing.

coderabbitai · 2026-07-13T14:56:50Z

📝 Walkthrough

Walkthrough

AWS credential vending now supports pod web-identity configuration. It resolves token files from properties or environment variables, routes scoped assumptions through AssumeRoleWithWebIdentity when configured, preserves legacy assumptions otherwise, and adds coverage for defaults and token-file errors.

Changes

AWS web-identity credential vending

Layer / File(s)	Summary
Configuration and token-file resolution `rust/lance-namespace-impls/src/credentials.rs`, `rust/lance-namespace-impls/src/credentials/aws.rs`	Adds AWS property keys, an optional token-file configuration field and builder, and resolution from explicit properties or `AWS_WEB_IDENTITY_TOKEN_FILE`.
Scoped STS assumption routing `rust/lance-namespace-impls/src/credentials/aws.rs`	Reads the token file per vend for `AssumeRoleWithWebIdentity`; otherwise retains the chained `AssumeRole` path and its `external_id` handling, with expanded error-chain formatting.
Vending integration and validation `rust/lance-namespace-impls/src/credentials/aws.rs`	Routes API-key and static-permission vending through shared scoped assumptions, simplifies identity-less vending, and tests defaults and token-file read failures.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CredentialVendor
  participant PodTokenFile
  participant STS
  CredentialVendor->>PodTokenFile: Read projected OIDC token
  CredentialVendor->>STS: AssumeRoleWithWebIdentity
  STS-->>CredentialVendor: Return scoped credentials

Suggested reviewers: geserdugarov

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: adding AWS credential vending via AssumeRoleWithWebIdentity to avoid role chaining.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@rust/lance-namespace-impls/src/credentials.rs`:
- Around line 514-533: Update the pod web-identity selection around
ASSUME_VIA_POD_WEB_IDENTITY so that when the opt-in is true but
AWS_WEB_IDENTITY_TOKEN_FILE is unset, emit an operator-visible warning before
continuing with the existing chained AssumeRole fallback. Keep explicit
POD_WEB_IDENTITY_TOKEN_FILE behavior unchanged and do not warn when the opt-in
is disabled or when a later error will already provide the relevant signal.

In `@rust/lance-namespace-impls/src/credentials/aws.rs`:
- Around line 813-836: The test test_pod_web_identity_path_reads_token_file must
assert the concrete NamespaceError/lance_core::Error variant as well as the
existing message text; destructure or match the returned error before validating
its message, while preserving the missing-token-file scenario. Run cargo fmt to
normalize formatting in the affected file.
- Line 443: Remove the stale doc comment above assume_scoped. Keep the shared
routine’s documentation aligned with its generic role across API-key and
static-config credential vending, without describing it as specifically
performing API-key-validated AssumeRole.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: e27d14be-4337-4dec-aef0-c0d94f02300c

📥 Commits

Reviewing files that changed from the base of the PR and between f0fcb81 and 523d66e.

📒 Files selected for processing (2)

rust/lance-namespace-impls/src/credentials.rs
rust/lance-namespace-impls/src/credentials/aws.rs

… to avoid role chaining The api_key/static credential-vendor flows assume the target role with `AssumeRole` signed by the process's ambient credentials. When the process runs with temporary credentials (e.g. an IRSA / EKS pod role), that is *role chaining*, which STS hard-caps to 1 hour regardless of the role's MaxSessionDuration -- and the returned `Expiration` can exceed the token's real validity, so downstream credential caches keep serving a token S3 already considers expired. Add an optional pod web-identity path: when `pod_web_identity_token_file` is configured (property `credential_vendor.aws_assume_via_pod_web_identity=true`, resolving `AWS_WEB_IDENTITY_TOKEN_FILE`, or an explicit `credential_vendor.aws_pod_web_identity_token_file`), the scoped assume uses `AssumeRoleWithWebIdentity` with the pod's projected service-account OIDC token. This is a direct (non-chained) assumption that honors MaxSessionDuration (up to 12h) and reports an accurate expiration. The per-table scoped session policy, permission handling, and the default chained `AssumeRole` behavior are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

coderabbitai

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@rust/lance-namespace-impls/src/credentials/aws.rs`:
- Around line 467-500: Extract the shared STS request construction from
vend_with_web_identity and the pod-token branch into a private helper such as
call_assume_role_with_web_identity, accepting the token, session name, policy,
and duration. Have both call sites use the helper while preserving their token
acquisition and surrounding behavior, and centralize the common error mapping
and credentials extraction there.
- Around line 470-477: Update the token_file read error mapping in
AwsCredentialVendor::new to use lance_core::Error::io(...) instead of
NamespaceError::Internal, preserving the existing contextual message and
propagation behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 5357d3d4-f385-4d50-b95d-e316a2048527

📥 Commits

Reviewing files that changed from the base of the PR and between 523d66e and b073b77.

📒 Files selected for processing (2)

rust/lance-namespace-impls/src/credentials.rs
rust/lance-namespace-impls/src/credentials/aws.rs

codecov · 2026-07-13T15:52:48Z

Codecov Report

❌ Patch coverage is 45.94595% with 60 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance-namespace-impls/src/credentials/aws.rs	53.12%	43 Missing and 2 partials ⚠️
rust/lance-namespace-impls/src/credentials.rs	0.00%	15 Missing ⚠️

📢 Thoughts on this report? Let us know!

…error chain Two observability improvements for the AWS credential vendor: - Apply full_error_chain() to the scoped AssumeRoleWithWebIdentity and AssumeRole failure sites so a vend failure reports the actual STS code and message (walked from the SDK error's source() chain) instead of a bare "service error". - Log once at vendor init which assume path was resolved: direct AssumeRoleWithWebIdentity via the pod token file, or the chained AssumeRole fallback. Warns when aws_assume_via_pod_web_identity=true but no token file is resolvable, which otherwise silently falls back to chaining. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

coderabbitai

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

rust/lance-namespace-impls/src/credentials.rs (1)
229-233: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Document the complete public web-identity configuration surface with examples.

The new public configuration APIs explain their semantics but lack the required usage examples and cross-links.

rust/lance-namespace-impls/src/credentials.rs#L229-L233: document both property-key constants with accepted values, precedence, and a configuration example.

rust/lance-namespace-impls/src/credentials/aws.rs#L76-L85: add an example for the token-file field and link to the builder.

rust/lance-namespace-impls/src/credentials/aws.rs#L128-L135: add an example using AwsCredentialVendorConfig::new(...).with_pod_web_identity_token_file(...).

As per coding guidelines, “All public APIs must have documentation with examples, and documentation should link to relevant structs and methods.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@rust/lance-namespace-impls/src/credentials.rs` around lines 229 - 233, Expand
documentation at rust/lance-namespace-impls/src/credentials.rs:229-233 for both
public web-identity property-key constants, including accepted values,
precedence rules, and a configuration example. In
rust/lance-namespace-impls/src/credentials/aws.rs:76-85, document the token-file
field with an example and a link to its builder method; in
rust/lance-namespace-impls/src/credentials/aws.rs:128-135, add an example using
AwsCredentialVendorConfig::new(...) with with_pod_web_identity_token_file(...).
Ensure all examples and cross-links accurately reference the relevant constants,
struct, and methods.
Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@rust/lance-namespace-impls/src/credentials.rs`:
- Around line 229-233: Expand documentation at
rust/lance-namespace-impls/src/credentials.rs:229-233 for both public
web-identity property-key constants, including accepted values, precedence
rules, and a configuration example. In
rust/lance-namespace-impls/src/credentials/aws.rs:76-85, document the token-file
field with an example and a link to its builder method; in
rust/lance-namespace-impls/src/credentials/aws.rs:128-135, add an example using
AwsCredentialVendorConfig::new(...) with with_pod_web_identity_token_file(...).
Ensure all examples and cross-links accurately reference the relevant constants,
struct, and methods.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 91c65094-66e0-4c5b-af1f-bb5b9009fec8

📥 Commits

Reviewing files that changed from the base of the PR and between b073b77 and 5fbfdb0.

📒 Files selected for processing (2)

rust/lance-namespace-impls/src/credentials.rs
rust/lance-namespace-impls/src/credentials/aws.rs

github-actions Bot added A-namespace Namespace impls enhancement New feature or request labels Jul 13, 2026

coderabbitai Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread rust/lance-namespace-impls/src/credentials.rs

Comment thread rust/lance-namespace-impls/src/credentials/aws.rs

Comment thread rust/lance-namespace-impls/src/credentials/aws.rs

rpgreen force-pushed the credential-vendor-pod-web-identity branch from 523d66e to b073b77 Compare July 13, 2026 15:07

coderabbitai Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread rust/lance-namespace-impls/src/credentials/aws.rs

Comment thread rust/lance-namespace-impls/src/credentials/aws.rs

jackye1995 approved these changes Jul 13, 2026

View reviewed changes

coderabbitai Bot reviewed Jul 13, 2026

View reviewed changes

rpgreen requested a review from jackye1995 July 13, 2026 18:08

jackye1995 merged commit a3d4719 into lance-format:main Jul 13, 2026
37 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(credentials): vend AWS credentials via AssumeRoleWithWebIdentity to avoid role chaining#7757

feat(credentials): vend AWS credentials via AssumeRoleWithWebIdentity to avoid role chaining#7757
jackye1995 merged 2 commits into
lance-format:mainfrom
rpgreen:credential-vendor-pod-web-identity

rpgreen commented Jul 13, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jul 13, 2026 •

edited

Loading

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Jul 13, 2026

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

rpgreen commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Change

Tests

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Jul 13, 2026

Codecov Report

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rpgreen commented Jul 13, 2026 •

edited

Loading

coderabbitai Bot commented Jul 13, 2026 •

edited

Loading