Skip to content

fix(media): allow Bedrock SDK auth for image and PDF tools#72092

Merged
steipete merged 5 commits into
openclaw:mainfrom
truffle-dev:fix/image-tool-allow-aws-sdk-auth-mode
Jul 7, 2026
Merged

fix(media): allow Bedrock SDK auth for image and PDF tools#72092
steipete merged 5 commits into
openclaw:mainfrom
truffle-dev:fix/image-tool-allow-aws-sdk-auth-mode

Conversation

@truffle-dev

@truffle-dev truffle-dev commented Apr 26, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

Amazon Bedrock chat already accepts auth.mode: "aws-sdk" without a literal API key, allowing the AWS SDK credential chain to resolve profile, role, SSO, environment, or instance credentials at request time. The image and PDF tools instead called requireApiKey first, so both failed before the Bedrock runtime could use those credentials.

Closes #72031.

Why This Change Was Made

  • Pass the empty auth sentinel only when the resolved mode is aws-sdk and the selected runtime is bedrock-converse-stream.
  • Do not persist the empty sentinel in runtime auth storage.
  • Treat configured aws-sdk runtime auth as available during automatic image/PDF model selection, including default-chain setups without AWS environment markers.
  • Register plugin-owned provider streams before non-native PDF completion, including PDF-only Bedrock model configurations.
  • Cover both independently confirmed surfaces: image description and PDF analysis.
  • Preserve strict missing-key errors for other auth modes and runtimes.
  • Remove the earlier audio/video expansion: Amazon Bedrock has no mediaUnderstandingProviders audio/video contract, so those branches were unreachable speculative surface.
  • Keep release-note context in this PR body; preserve contributor credit through commit authorship and the public history.

The Bedrock stream constructs BedrockRuntimeClient without injected credentials in SDK-chain mode; AWS SDK 3.1078.0 supplies its default credential provider in that case.

User Impact

Bedrock users can use the image and PDF tools with the same AWS profile, role, SSO, environment, or instance credentials that already work for normal chat. No static Bedrock bearer token or stored API-key surrogate is required.

Evidence

  • Focused Testbox proof: 84/84 tests passed across image execution, config-only image/PDF factory selection, shared tool-auth selection, and PDF execution/registration on lease tbx_01kwxgk2yze3acn2bnnwn8kr0f. The initial 59-test execution run is linked in Actions.
  • Full changed gate: corepack pnpm check:changed — passed on the same Testbox lease, including core/core-test typechecks, changed-file lint, changelog attribution, dependency guards, import cycles, and policy checks.
  • Formatting: targeted oxfmt --check passed for all four changed TypeScript files on Testbox.
  • Fresh structured autoreview: no accepted/actionable findings on the runtime and regression-test patch. The later release-note-only correction removed the root changelog hunk required by the repository prepare guard.
  • Live AWS image proof from the reported environment: before the change, the tool failed with missing API-key auth; after the same empty-sentinel handoff, Bedrock returned a normal orange-tabby description using profile credentials and no static key (proof comment).
  • PDF production evidence: separate reporters reproduced the same guard failure with an EC2 credential source and with static AWS environment credentials while normal Bedrock chat remained healthy (issue).
  • Maintainer live retry: the local environment had the exact profile-only shape, but both configured profiles currently failed STS with InvalidClientTokenId; no independent paid Bedrock request was possible from this checkout.

AI-assisted

  • Tested and reviewed.
  • No sanitized agent transcript attached; none was requested.

@greptile-apps

greptile-apps Bot commented Apr 26, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR extends the aws-sdk auth-mode bypass (already present on the chat path) to the three media-understanding resolution sites so that Bedrock role/SSO/profile deployments can reach image and audio/video tools. The image path in image.ts is correctly fixed, but the audio/video path through resolveProviderExecutionAuth in runner.entries.ts has a gap that will cause it to fail at runtime.

  • runner.entries.ts P1: primaryApiKey = \"\" is passed to collectProviderApiKeysForExecutiondedupeApiKeys strips it (empty string is falsy) → apiKeys = []executeWithApiKeyRotation throws \"No API keys configured for provider...\" before reaching the provider. Bedrock aws-sdk users will still get a failure on audio/video transcription/description despite this fix.

Confidence Score: 3/5

Not safe to merge as-is — the audio/video runner path will still fail for Bedrock aws-sdk users

The image fix works correctly but the fix in runner.entries.ts is functionally broken: the empty-string apiKey is filtered by dedupeApiKeys inside collectProviderApiKeysForExecution, making executeWithApiKeyRotation throw before invoking the provider for all Bedrock aws-sdk deployments without rotation keys.

src/media-understanding/runner.entries.ts — resolveProviderExecutionAuth and its interaction with collectProviderApiKeysForExecution/executeWithApiKeyRotation

Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/media-understanding/runner.entries.ts
Line: 404-412

Comment:
**aws-sdk empty key filtered out before `executeWithApiKeyRotation`**

When `auth.mode === "aws-sdk"` and the key is empty, `primaryApiKey` is set to `""`. That value is then fed into `collectProviderApiKeysForExecution`, which internally calls `dedupeApiKeys`. `dedupeApiKeys` skips any entry where `value.trim()` is falsy (`if (!apiKey || seen.has(apiKey)) { continue; }`), so the `""` is silently dropped.

If the Bedrock deployment has no rotation keys configured (the common case for role/SSO/profile auth), `apiKeys` comes back as `[]`. `executeWithApiKeyRotation` then immediately throws `"No API keys configured for provider 'amazon-bedrock'."` at line 44, before even reaching the `execute` callback — so the audio/video path still fails for Bedrock `aws-sdk` users despite this fix.

The image path in `resolveImageRuntime` works because it passes the empty string directly to `complete()` and never goes through `executeWithApiKeyRotation`.

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "fix(media): allow aws-sdk auth mode for ..." | Re-trigger Greptile

Comment on lines 404 to 412
const primaryApiKey =
!auth.apiKey?.trim() && auth.mode === "aws-sdk" ? "" : requireApiKey(auth, params.providerId);
return {
apiKeys: collectProviderApiKeysForExecution({
provider: params.providerId,
primaryApiKey: requireApiKey(auth, params.providerId),
primaryApiKey,
}),
providerConfig: params.cfg.models?.providers?.[params.providerId],
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 aws-sdk empty key filtered out before executeWithApiKeyRotation

When auth.mode === "aws-sdk" and the key is empty, primaryApiKey is set to "". That value is then fed into collectProviderApiKeysForExecution, which internally calls dedupeApiKeys. dedupeApiKeys skips any entry where value.trim() is falsy (if (!apiKey || seen.has(apiKey)) { continue; }), so the "" is silently dropped.

If the Bedrock deployment has no rotation keys configured (the common case for role/SSO/profile auth), apiKeys comes back as []. executeWithApiKeyRotation then immediately throws "No API keys configured for provider 'amazon-bedrock'." at line 44, before even reaching the execute callback — so the audio/video path still fails for Bedrock aws-sdk users despite this fix.

The image path in resolveImageRuntime works because it passes the empty string directly to complete() and never goes through executeWithApiKeyRotation.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/media-understanding/runner.entries.ts
Line: 404-412

Comment:
**aws-sdk empty key filtered out before `executeWithApiKeyRotation`**

When `auth.mode === "aws-sdk"` and the key is empty, `primaryApiKey` is set to `""`. That value is then fed into `collectProviderApiKeysForExecution`, which internally calls `dedupeApiKeys`. `dedupeApiKeys` skips any entry where `value.trim()` is falsy (`if (!apiKey || seen.has(apiKey)) { continue; }`), so the `""` is silently dropped.

If the Bedrock deployment has no rotation keys configured (the common case for role/SSO/profile auth), `apiKeys` comes back as `[]`. `executeWithApiKeyRotation` then immediately throws `"No API keys configured for provider 'amazon-bedrock'."` at line 44, before even reaching the `execute` callback — so the audio/video path still fails for Bedrock `aws-sdk` users despite this fix.

The image path in `resolveImageRuntime` works because it passes the empty string directly to `complete()` and never goes through `executeWithApiKeyRotation`.

How can I resolve this? If you propose a fix, please make it concise.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 03ba12215c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@@ -401,10 +401,12 @@ async function resolveProviderExecutionAuth(params: {
preferredProfile: params.entry.preferredProfile,
agentDir: params.agentDir,
});
const primaryApiKey =
!auth.apiKey?.trim() && auth.mode === "aws-sdk" ? "" : requireApiKey(auth, params.providerId);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Skip API-key rotation for aws-sdk providers without keys

resolveProviderExecutionAuth now permits auth.mode === "aws-sdk" by setting primaryApiKey to "", but this value is immediately passed into collectProviderApiKeysForExecution and then executeWithApiKeyRotation, which drops empty keys and throws when the resulting list is empty. In the exact role/profile/default-chain setup this commit targets (no static key), audio/video provider entries still fail with No API keys configured for provider ... instead of running with SDK-resolved credentials.

Useful? React with 👍 / 👎.

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: M and removed size: S labels Apr 26, 2026
@truffle-dev
truffle-dev force-pushed the fix/image-tool-allow-aws-sdk-auth-mode branch from 2e5f135 to 1ca8735 Compare April 26, 2026 10:23
@clawsweeper

clawsweeper Bot commented Apr 26, 2026

Copy link
Copy Markdown
Contributor

Codex review: stale review; fresh review needed.

Summary
The latest durable ClawSweeper review was for head d3f9a22729ed5529e8147e16d4fc6e6074d69724, but the PR head is now 5b62189de157787d4330192cbe83d8916663983d. Its old verdict and PR readiness labels are no longer current.

Next step
Run or wait for a fresh ClawSweeper review on the current PR head.

Review history (1 earlier review cycle)
  • reviewed 2026-07-03T09:51:06.561Z sha f4d073e :: needs maintainer review before merge. :: none

@truffle-dev

Copy link
Copy Markdown
Contributor Author

Greptile P1 / Codex P1 — empty-string apiKey getting filtered by dedupeApiKeys before reaching executeWithApiKeyRotation: addressed in commit e79b802559. Added allowEmptyKey to the rotation helper (single empty-key invocation when the deduped list is empty), threaded an allowEmptyExecution flag from resolveProviderExecutionAuth through the audio/video call sites. runner.aws-sdk.test.ts covers the four shapes: no rotation keys, one rotation key, literal-key + aws-sdk fallthrough, non-aws-sdk modes still throw.

Just rebased onto current main; the four pre-existing CI failures (checks-fast-contracts-plugins, checks-node-agentic-commands, checks-node-core, parity gate) trace to baseline state at the prior merge-base and were repaired upstream after the original branch point.

@truffle-dev
truffle-dev force-pushed the fix/image-tool-allow-aws-sdk-auth-mode branch from e79b802 to 5d3defe Compare April 27, 2026 11:07
@truffle-dev

Copy link
Copy Markdown
Contributor Author

Rebased again onto current main (head now 5d3defec55). The three remaining failures from the previous push (model-auth.test.ts:919 plus its two aggregator gates checks-node-core-support-boundary and checks-node-core) traced to a baseline test where the expected source was models.json (local marker) but runtime was returning models.providers.ollama-remote (synthetic local key). Upstream a3144b6bfd fix(agents): preserve explicit Ollama local auth marker (12:00:41Z) ships both the runtime fix in resolveUsableCustomProviderApiKey and the test alignment; this rebase pulls it in.

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 auth-provider 🚨 May break OAuth, tokens, provider routing, model choice, or credentials. labels May 20, 2026
@clawsweeper

clawsweeper Bot commented May 20, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

🔥 Warming up: real-behavior proof passed; findings, security review, or rank-up moves are still in progress.

Hatch command

Comment @clawsweeper hatch when this PR is hatchable.

Hatchability rules:

  • Merged PRs are hatchable.
  • Open PRs are hatchable when they are status: 👀 ready for maintainer look, status: 🚀 automerge armed, or labeled clawsweeper:automerge.
  • Closed unmerged PRs are hatchable only when one of those hatchable labels is still present in the durable record.
What is this egg doing here?
  • Eggs appear after the PR passes real-behavior proof. It is here for vibes, not verdicts: it does not change labels, ratings, merge decisions, or automation.
  • The shell reacts to review momentum: open follow-up work warms it up, re-review makes it wobble, and a clean final review lets it hatch.
  • Hatchability usually comes from sufficient real-behavior proof, no blocking P0/P1/P2 findings, no security attention needed, and clean correctness. A merged PR is already final, so merge makes the egg hatchable independently.
  • The hatch is seeded from this repository and PR number, so the same PR keeps the same creature; the reviewed head SHA can only change safe visual details.
  • Rarity is just collectible sparkle: 🥚 common, 🌱 uncommon, 💎 rare, ✨ glimmer, and 🌈 legendary.

@openclaw-barnacle openclaw-barnacle Bot added the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label May 20, 2026
@truffle-dev
truffle-dev force-pushed the fix/image-tool-allow-aws-sdk-auth-mode branch from 5d3defe to ecbd8a0 Compare May 20, 2026 19:32
@truffle-dev

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (1a7669bc63) — two conflict
sites resolved:

  1. image.ts resolveImageRuntime: upstream added a
    github-copilot OAuth-token-exchange step that needed to
    stay; merged the aws-sdk empty-key escape ahead of it,
    kept the copilot exchange path intact, guarded
    setRuntimeApiKey on a non-empty key so the bedrock path
    doesn't write "" into auth storage.
  2. image.ts resolveMinimaxVlmFallbackRuntime + audio/video
    call sites in runner.entries.ts: upstream added a
    transientRetry config + workspaceDir thread-through;
    merged the allowEmptyKey option alongside both, so the
    rotation helper now respects both transientRetry (read
    ops) and allowEmptyKey (aws-sdk single-empty invocation).

Local re-run: image.test.ts 25/25 ✓ on the rebased commits
(includes the existing bedrock-aws-sdk regression test).

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels May 20, 2026
@truffle-dev
truffle-dev force-pushed the fix/image-tool-allow-aws-sdk-auth-mode branch from ecbd8a0 to 0f1c28f Compare May 21, 2026 00:06
@truffle-dev

Copy link
Copy Markdown
Contributor Author

Rebased again onto 5c4c6a4207 to pick up the latest upstream commits. Previous push was behind by ~11 commits, which surfaced a TS2739 failure in src/commands/status.summary.redaction.test.ts (unrelated to this PR — the SessionStatus mock there had not yet been updated for configuredModel/selectedModel/modelSelectionReason).

No conflicts on rebase; my two commits replay cleanly onto the new tip.

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. proof: sufficient ClawSweeper judged the real behavior proof convincing. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jun 21, 2026
@truffle-dev

Copy link
Copy Markdown
Contributor Author

Status note, since this is now the only open fix for #72031.

The three sibling attempts (#72066, #77479, #97525) all closed without merge; this one was kept open and re-rated after the 06-21 rebase onto the keyless kind: "none" execution-auth path. CI is green across the auth-provider boundary checks.

I know merge-risk: auth-provider warrants a careful human look. If a smaller surface would help that review, I'm glad to split the carve-out or add targeted coverage — just point me at the concern.

@steipete
steipete force-pushed the fix/image-tool-allow-aws-sdk-auth-mode branch from f4d073e to d3f9a22 Compare July 7, 2026 06:04
@steipete
steipete requested a review from a team as a code owner July 7, 2026 06:04
@steipete

steipete commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Land-ready at final prepared head 1a59c97216829f3b9f00d244eef5385764d971a7.

Work completed:

  • Narrowed the contributor patch to the supported Bedrock image/PDF surfaces; removed the speculative audio/video runner expansion.
  • Reused canonical runtime auth availability so config-only auth: "aws-sdk" participates in automatic image/PDF model selection.
  • Allowed the empty API-key sentinel only for aws-sdk with bedrock-converse-stream, without persisting it as a runtime secret.
  • Registered the PDF provider stream before non-native completion.
  • Added focused image/PDF selection, auth, persistence, and stream-registration regressions. Release-note context and contributor credit remain in the PR/public commit history; the prepare guard removed the release-owned root changelog hunk.

Proof:

Before/after: issue #72031 and the reporter's live proof show AWS profile-backed Bedrock image execution failing at OpenClaw's static-key gate before this change, then successfully producing an image through the AWS SDK credential chain after it. Independent live retry reached AWS STS through both locally configured profiles, but both tokens were invalid (InvalidClientTokenId), so no second paid Bedrock call was possible without a valid credential.

@truffle-dev

Copy link
Copy Markdown
Contributor Author

Land-ready at exact head d3f9a22729ed5529e8147e16d4fc6e6074d69724.

Work completed:

  • Narrowed the contributor patch to the supported Bedrock image/PDF surfaces; removed the speculative audio/video runner expansion.
  • Reused canonical runtime auth availability so config-only auth: "aws-sdk" participates in automatic image/PDF model selection.
  • Allowed the empty API-key sentinel only for aws-sdk with bedrock-converse-stream, without persisting it as a runtime secret.
  • Registered the PDF provider stream before non-native completion.
  • Added focused image/PDF selection, auth, persistence, and stream-registration regressions plus the Unreleased changelog credit.

Proof:

Before/after: issue #72031 and the reporter's live proof show AWS profile-backed Bedrock image execution failing at OpenClaw's static-key gate before this change, then successfully producing an image through the AWS SDK credential chain after it. Independent live retry reached AWS STS through both locally configured profiles, but both tokens were invalid (InvalidClientTokenId), so no second paid Bedrock call was possible without a valid credential.

Thank youuu

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5b62189de1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

},
});
const nativeTurnCompleted = await waitForActiveNativeTurnCompletion(activeNativeTurnIds);
const nativeTurnCompleted = await waitForActiveNativeTurnCompletion();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Block on the listed native turn ids

When a resumed thread reports activeTurnIds, this now discards those ids and waits on the route-level completion flag, which is set by any terminal notification for the thread while the route is unbound. In a resumed compact/review turn, a stale or unrelated turn/completed for the same thread can satisfy this wait, so turn/start can run while the active native turn is still in progress—the wedge this guard is meant to avoid. Use the reported active turn ids (or the exact watchNativeTurnCompletion path) for this wait.

Useful? React with 👍 / 👎.

truffle-dev and others added 5 commits July 7, 2026 07:36
Media understanding tools failed for amazon-bedrock deployments using
auth.mode "aws-sdk" (BYOK via role/SSO/profile creds). Each path called
requireApiKey, which throws on the empty-key sentinel before the AWS SDK
credential chain can resolve creds at call time.

The image path (image.ts) resolves auth via getApiKeyForModel; the
audio/video paths route through resolveProviderExecutionAuth. Both now
mirror the chat path's allowMissingApiKeyModes allowance: when the
resolved key is empty and the mode is aws-sdk, execute keyless and let
the SDK resolve credentials. The image path skips setRuntimeApiKey so no
empty-string secret is persisted; audio/video return the kind:"none"
execution auth so the runner bypasses key rotation.

Closes openclaw#72031
@steipete

steipete commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Merged via squash.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling P1 High-priority user-facing bug, regression, or broken workflow. proof: supplied External PR includes structured after-fix real behavior proof. size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: image tool fails for Bedrock with auth mode: aws-sdkrequireApiKey throws even when AWS SDK creds are available

5 participants