fix: prevent plaintext API key leak to models.json for exec/file-source SecretRefs (#34335)#34499
fix: prevent plaintext API key leak to models.json for exec/file-source SecretRefs (#34335)#34499lml2468 wants to merge 3 commits intoopenclaw:mainfrom
Conversation
…cretRefs (openclaw#34335) When a provider's API key is managed via an exec-source SecretRef (e.g. macOS Keychain), the synchronous resolveApiKeyFromProfiles() silently returned undefined, causing two problems: 1. normalizeProviders thought no auth was configured and skipped setting apiKey, but mergeWithExistingProviderSecrets unconditionally preserved any existing plaintext key — defeating the SecretRef security model. 2. Key rotation via Keychain had no effect because models.json kept the stale old key via merge. Fix: - resolveApiKeyFromProfiles returns '__redacted__' sentinel for exec-source SecretRefs instead of undefined. - mergeWithExistingProviderSecrets skips preserving old plaintext keys when the new entry uses the sentinel. - getCustomProviderApiKey filters the sentinel so it never reaches providers. - Runtime async path (resolveApiKeyForProvider) unchanged. AI-assisted (Claude, fully tested) Fixes openclaw#34335
- Add file-source to sentinel check (same sync limitation as exec) - Add getCustomProviderApiKey sentinel filtering tests - Fix lint (no-explicit-any → typed cast)
Greptile SummaryThis PR addresses a real security bug (#34335) where plaintext API keys could be leaked into The core approach is sound and the primary bug is fixed. Key issues found:
Confidence Score: 3/5
Last reviewed commit: 893ec3b |
| import { coerceSecretRef } from "../config/types.secrets.js"; | ||
| /** | ||
| * Sentinel value written to models.json instead of a real API key when the | ||
| * auth source is an exec/keychain SecretRef that cannot be resolved | ||
| * synchronously. The runtime async path (resolveApiKeyForProvider) handles | ||
| * actual credential resolution at request time. See #34335. | ||
| */ | ||
| export const REDACTED_API_KEY_SENTINEL = "__redacted__"; | ||
|
|
||
| import { createSubsystemLogger } from "../logging/subsystem.js"; |
There was a problem hiding this comment.
Constant declared between import statements
REDACTED_API_KEY_SENTINEL is placed between two import blocks (after the coerceSecretRef import on line 3, before the createSubsystemLogger import on line 12). While ES modules hoist imports and this doesn't cause a runtime error, it is non-standard and will likely trigger import-order lint rules. The constant should be placed after all top-level imports.
| import { coerceSecretRef } from "../config/types.secrets.js"; | |
| /** | |
| * Sentinel value written to models.json instead of a real API key when the | |
| * auth source is an exec/keychain SecretRef that cannot be resolved | |
| * synchronously. The runtime async path (resolveApiKeyForProvider) handles | |
| * actual credential resolution at request time. See #34335. | |
| */ | |
| export const REDACTED_API_KEY_SENTINEL = "__redacted__"; | |
| import { createSubsystemLogger } from "../logging/subsystem.js"; | |
| import { coerceSecretRef } from "../config/types.secrets.js"; | |
| import { createSubsystemLogger } from "../logging/subsystem.js"; |
Then declare the sentinel after all imports:
/**
* Sentinel value written to models.json instead of a real API key when the
* auth source is an exec/keychain SecretRef that cannot be resolved
* synchronously. The runtime async path (resolveApiKeyForProvider) handles
* actual credential resolution at request time. See #34335.
*/
export const REDACTED_API_KEY_SENTINEL = "__redacted__";Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/models-config.providers.ts
Line: 3-12
Comment:
**Constant declared between import statements**
`REDACTED_API_KEY_SENTINEL` is placed between two `import` blocks (after the `coerceSecretRef` import on line 3, before the `createSubsystemLogger` import on line 12). While ES modules hoist imports and this doesn't cause a runtime error, it is non-standard and will likely trigger import-order lint rules. The constant should be placed after all top-level imports.
```suggestion
import { coerceSecretRef } from "../config/types.secrets.js";
import { createSubsystemLogger } from "../logging/subsystem.js";
```
Then declare the sentinel after all imports:
```ts
/**
* Sentinel value written to models.json instead of a real API key when the
* auth source is an exec/keychain SecretRef that cannot be resolved
* synchronously. The runtime async path (resolveApiKeyForProvider) handles
* actual credential resolution at request time. See #34335.
*/
export const REDACTED_API_KEY_SENTINEL = "__redacted__";
```
How can I resolve this? If you propose a fix, please make it concise.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| if ( | ||
| typeof existing.apiKey === "string" && | ||
| existing.apiKey && | ||
| newEntry.apiKey !== REDACTED_API_KEY_SENTINEL | ||
| ) { | ||
| preserved.apiKey = existing.apiKey; | ||
| } |
There was a problem hiding this comment.
Sentinel can be propagated forward when user switches away from SecretRef
The guard added here prevents preserving a stale plain-text key when the new entry uses the sentinel. However, it does not prevent an old sentinel value in existing.apiKey (from a previous models.json write) from overwriting a real key in newEntry.apiKey.
Failing scenario:
- User configures an exec SecretRef →
models.jsonis written withapiKeyset to the sentinel. - User switches to a plain API key.
- Next refresh:
newEntry.apiKeyis the real key,existing.apiKeyis still the old sentinel. - The current guard passes (new entry is not the sentinel), so
preserved.apiKeyis set to the old sentinel. { ...newEntry, ...preserved }results in the sentinel silently overwriting the real key.
The fix should also check existing.apiKey !== REDACTED_API_KEY_SENTINEL so that a stale sentinel is never carried forward.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/models-config.ts
Line: 166-172
Comment:
**Sentinel can be propagated forward when user switches away from SecretRef**
The guard added here prevents preserving a stale plain-text key when the *new* entry uses the sentinel. However, it does not prevent an *old* sentinel value in `existing.apiKey` (from a previous `models.json` write) from overwriting a real key in `newEntry.apiKey`.
Failing scenario:
1. User configures an exec SecretRef → `models.json` is written with `apiKey` set to the sentinel.
2. User switches to a plain API key.
3. Next refresh: `newEntry.apiKey` is the real key, `existing.apiKey` is still the old sentinel.
4. The current guard passes (new entry is not the sentinel), so `preserved.apiKey` is set to the old sentinel.
5. `{ ...newEntry, ...preserved }` results in the sentinel silently overwriting the real key.
The fix should also check `existing.apiKey !== REDACTED_API_KEY_SENTINEL` so that a stale sentinel is never carried forward.
How can I resolve this? If you propose a fix, please make it concise.
Additional Comments (1)
When
No real credential leaks, but it produces unnecessary failed requests. The sentinel should be mapped to Prompt To Fix With AIThis is a comment left during a code review.
Path: src/agents/models-config.providers.ts
Line: 1117-1126
Comment:
**Sentinel forwarded into network-discovery calls**
When `resolveApiKeyFromProfiles` returns the sentinel for a HuggingFace or vLLM exec/file SecretRef profile, the value propagates into `buildHuggingfaceProvider` and `buildVllmProvider` as the `apiKey` argument.
- In `buildHuggingfaceProvider`: any non-empty string that is not an uppercase env-var name is used directly as a Bearer token, so `discoverHuggingfaceModels` is called with the sentinel as auth, sending an `Authorization: Bearer <sentinel>` header to the HuggingFace API.
- In `buildVllmProvider`: `discoverVllmModels` similarly forwards the sentinel as a Bearer token.
No real credential leaks, but it produces unnecessary failed requests. The sentinel should be mapped to `undefined` before being passed into discovery builders.
How can I resolve this? If you propose a fix, please make it concise. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 893ec3bf8f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| typeof existing.apiKey === "string" && | ||
| existing.apiKey && | ||
| newEntry.apiKey !== REDACTED_API_KEY_SENTINEL | ||
| ) { |
There was a problem hiding this comment.
Skip preserving old sentinel when key source changes
When a provider previously wrote "__redacted__" (exec/file SecretRef) and later switches to a resolvable key source, this merge path still preserves existing.apiKey because it only checks newEntry.apiKey !== REDACTED_API_KEY_SENTINEL. Since mergedProviders[key] = { ...newEntry, ...preserved }, the stale sentinel overwrites the new key and models.json cannot recover from the placeholder in merge mode, leaving downstream models.json consumers with an unusable credential.
Useful? React with 👍 / 👎.
…inel overwrite - Move REDACTED_API_KEY_SENTINEL after all import statements (Greptile) - Also filter existing.apiKey === sentinel in merge logic, preventing a stale sentinel from overwriting a real key when user switches back from SecretRef to inline key (Greptile + Codex) - Add test for stale sentinel scenario - Update JSDoc: exec/keychain → exec/file
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5960685c42
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| typeof existing.apiKey === "string" && | ||
| existing.apiKey && | ||
| existing.apiKey !== REDACTED_API_KEY_SENTINEL && | ||
| newEntry.apiKey !== REDACTED_API_KEY_SENTINEL |
There was a problem hiding this comment.
Derive sentinel before preserving existing apiKey
In merge mode this guard still preserves leaked plaintext keys for explicit providers that rely on exec/file SecretRefs, because resolveProvidersForMode() runs before normalizeProviders() in ensureOpenClawModelsJson, so newEntry.apiKey is still undefined here (not the sentinel) when the merge happens. In that common path, an old plaintext existing.apiKey from models.json is copied back and never replaced, so migrating a provider from inline key to SecretRef can continue to leave the real key on disk.
Useful? React with 👍 / 👎.
| typeof existing.apiKey === "string" && | ||
| existing.apiKey && | ||
| existing.apiKey !== REDACTED_API_KEY_SENTINEL && | ||
| newEntry.apiKey !== REDACTED_API_KEY_SENTINEL |
There was a problem hiding this comment.
Thanks for the analysis. I looked at this carefully — the concern is a false positive.
The flow is:
resolveProvidersForModelsJson()callsresolveApiKeyFromProfiles()which returnsREDACTED_API_KEY_SENTINELfor exec/file SecretRefs →newEntry.apiKeyIS the sentinel at merge timeresolveProvidersForMode()→ callsmergeWithExistingProviderSecrets()with sentinel already setnormalizeProviders()runs after merge
The Codex analysis seems to confuse normalizeProviders (which normalizes model definitions) with resolveProvidersForModelsJson (which resolves API keys). The sentinel is set in step 1 before merge, not in step 3 after.
For explicit providers with apiKey as a SecretRef object in config: hasConfiguredApiKey is true so it skips the profiles path entirely — the object flows through and is handled by normalizeOptionalSecretInput in step 3.
|
Thanks for the work on this PR and for contributing to this fix path. Closing this as superseded by #38955 (merged), which integrated the relevant fixes from this thread and completed the broader hardening scope for the secrets/models persistence cluster. The merged work now covers source-snapshot persistence, marker/provenance auth handling, audit/apply expansion (including models.json/header residues), and write-mode hardening/tests/docs in one end-to-end fix. Appreciate the contribution here. If there’s a remaining unique change from this PR that should still land, please call it out in a follow-up issue and reference #38955. |
What
Prevent
normalizeProvidersfrom leaking plaintext API keys intomodels.jsonwhen the auth source is an exec or file SecretRef.Why
Fixes #34335 — When a provider's API key is managed via an exec-source SecretRef (e.g. macOS Keychain), two bugs cause the plaintext key to persist on disk:
resolveApiKeyFromProfiles()silently returnsundefinedfor exec/file SecretRefs (sync function cannot resolve async sources), sonormalizeProvidersthinks no auth is configured.mergeWithExistingProviderSecrets()unconditionally preserves any existing plaintextapiKeyfrom a previousmodels.json, defeating the SecretRef security model and breaking key rotation.How
resolveApiKeyFromProfiles()now returns a sentinel value ("__redacted__") for exec/file-source SecretRefs instead ofundefined, signaling that auth IS configured without leaking the actual secret.mergeWithExistingProviderSecrets()skips preserving old plaintext keys when the new entry uses the sentinel.getCustomProviderApiKey()filters the sentinel so it never reaches provider APIs. The runtime async path (resolveApiKeyForProvider) is unchanged and correctly resolves exec SecretRefs at request time.Design alternatives considered
A cleaner long-term approach would be adding an
apiKeySourcefield to the models.json schema to explicitly track auth provenance, avoiding sentinel values entirely. However, this would be a schema change with broader impact, better suited for a separate RFC. The sentinel approach is minimal, backward-compatible, and sufficient for this bug fix.Testing
pnpm buildpassespnpm checkpasses (180 pre-existing tsgo errors from extension deps, unchanged)getCustomProviderApiKeyfiltering