Secrets: harden SecretRef-safe models.json persistence#38955
Secrets: harden SecretRef-safe models.json persistence#38955
Conversation
🔒 Aisle Security AnalysisWe found 1 potential security issue(s) in this PR:
1. 🟡 Symlink/TOCTOU arbitrary file overwrite & chmod via models.json persistence path
DescriptionThe If an attacker can influence
Vulnerable code: const agentDir = agentDirOverride?.trim() ? agentDirOverride.trim() : resolveOpenClawAgentDir();
const targetPath = path.join(agentDir, "models.json");
...
await fs.writeFile(targetPath, next, { mode: 0o600 });
await fs.chmod(targetPath, 0o600).catch(() => {});Impact depends on deployment, but in any scenario where untrusted input can affect RecommendationHarden models.json writes against symlink/hardlink attacks and restrict the writable location.
Example (best-effort illustration): import { constants as FS } from "node:fs";
async function safeWriteModelsJson(targetPath: string, content: string) {
// Refuse symlink targets
try {
const st = await fs.lstat(targetPath);
if (st.isSymbolicLink()) {
throw new Error("Refusing to write models.json via symlink");
}
} catch {
// ok if missing
}
const dir = path.dirname(targetPath);
await fs.mkdir(dir, { recursive: true, mode: 0o700 });
const tmpPath = `${targetPath}.tmp.${process.pid}`;
const fh = await fs.open(tmpPath, FS.O_CREAT | FS.O_TRUNC | FS.O_WRONLY, 0o600);
try {
await fh.writeFile(content, "utf8");
await fh.sync();
} finally {
await fh.close();
}
await fs.rename(tmpPath, targetPath);
}
Analyzed PR: #38955 at commit Last updated on: 2026-03-07T18:02:41Z |
Greptile SummaryThis PR delivers a comprehensive end-to-end fix for the SecretRef plaintext-persistence cluster. The implementation correctly:
The security-critical paths (normalize-before-merge ordering, SecretRef-managed key preservation, marker guards, file mode enforcement, write locking) are correctly implemented with comprehensive test coverage. The code is safe to merge. Confidence Score: 5/5
Last reviewed commit: 146e82f |
|
|
||
| for (const agentId of listAgentIds(config)) { | ||
| if (agentId === "main") { | ||
| paths.add(path.join(resolveUserPath(stateDir), "agents", "main", "agent", "models.json")); | ||
| continue; | ||
| } | ||
| const agentDir = resolveAgentDir(config, agentId); | ||
| paths.add(path.join(resolveUserPath(agentDir), "models.json")); | ||
| } |
There was a problem hiding this comment.
listAgentIds and resolveAgentDir are called in listAgentModelsJsonPaths but not imported. Both functions are exported from ../agents/agent-scope.ts. This causes a ReferenceError at runtime, breaking the entire secrets audit and secrets apply command flow.
Add the missing import at the top of the file:
| for (const agentId of listAgentIds(config)) { | |
| if (agentId === "main") { | |
| paths.add(path.join(resolveUserPath(stateDir), "agents", "main", "agent", "models.json")); | |
| continue; | |
| } | |
| const agentDir = resolveAgentDir(config, agentId); | |
| paths.add(path.join(resolveUserPath(agentDir), "models.json")); | |
| } | |
| import { listAgentIds, resolveAgentDir } from "../agents/agent-scope.js"; |
Without this, the listAgentModelsJsonPaths function will fail on any call.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/secrets/storage-scan.ts
Line: 48-56
Comment:
`listAgentIds` and `resolveAgentDir` are called in `listAgentModelsJsonPaths` but not imported. Both functions are exported from `../agents/agent-scope.ts`. This causes a `ReferenceError` at runtime, breaking the entire `secrets audit` and `secrets apply` command flow.
Add the missing import at the top of the file:
```suggestion
import { listAgentIds, resolveAgentDir } from "../agents/agent-scope.js";
```
Without this, the `listAgentModelsJsonPaths` function will fail on any call.
How can I resolve this? If you propose a fix, please make it concise.| export function isNonSecretApiKeyMarker( | ||
| value: string, | ||
| opts?: { includeEnvVarName?: boolean }, | ||
| ): boolean { | ||
| const trimmed = value.trim(); | ||
| if (!trimmed) { | ||
| return false; | ||
| } | ||
| const isKnownMarker = | ||
| trimmed === MINIMAX_OAUTH_MARKER || | ||
| trimmed === QWEN_OAUTH_MARKER || | ||
| trimmed === OLLAMA_LOCAL_AUTH_MARKER || | ||
| trimmed === NON_ENV_SECRETREF_MARKER || | ||
| isAwsSdkAuthMarker(trimmed); | ||
| if (isKnownMarker) { | ||
| return true; | ||
| } | ||
| return opts?.includeEnvVarName === false ? false : isEnvVarNameMarker(trimmed); | ||
| } |
There was a problem hiding this comment.
isNonSecretApiKeyMarker defaults to treating env-var-like strings as markers when includeEnvVarName is not explicitly false. The regex /^[A-Z][A-Z0-9_]*$/ will match any all-uppercase alphanumeric string — including real API keys from vendors that use this format (e.g., SECRETAPITOKEN).
In src/infra/provider-usage.auth.ts (lines 107, 127, 133), resolveProviderApiKeyFromConfigAndStore calls !isNonSecretApiKeyMarker(key) without options, so legitimate all-uppercase API keys would be incorrectly filtered out as markers, causing silent auth failures.
Consider either:
- Adding a length guard to the env-var-name regex (e.g., minimum 8 chars) to reduce false positives on short tokens
- Requiring explicit
includeEnvVarName: trueat persistence/audit call sites, withfalseas the default for request-path callers
This prevents real credentials from being lost when they happen to match the env-var-name pattern.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/model-auth-markers.ts
Line: 42-60
Comment:
`isNonSecretApiKeyMarker` defaults to treating env-var-like strings as markers when `includeEnvVarName` is not explicitly `false`. The regex `/^[A-Z][A-Z0-9_]*$/` will match any all-uppercase alphanumeric string — including real API keys from vendors that use this format (e.g., `SECRETAPITOKEN`).
In `src/infra/provider-usage.auth.ts` (lines 107, 127, 133), `resolveProviderApiKeyFromConfigAndStore` calls `!isNonSecretApiKeyMarker(key)` without options, so legitimate all-uppercase API keys would be incorrectly filtered out as markers, causing silent auth failures.
Consider either:
1. Adding a length guard to the env-var-name regex (e.g., minimum 8 chars) to reduce false positives on short tokens
2. Requiring explicit `includeEnvVarName: true` at persistence/audit call sites, with `false` as the default for request-path callers
This prevents real credentials from being lost when they happen to match the env-var-name pattern.
How can I resolve this? If you propose a fix, please make it concise.|
@greptile-apps Review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 67ef83311f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (!trimmed || isNonSecretApiKeyMarker(trimmed)) { | ||
| return undefined; |
There was a problem hiding this comment.
Accept uppercase discovery keys as real secrets
toDiscoveryApiKey drops any value recognized by isNonSecretApiKeyMarker, and that marker check treats every ^[A-Z][A-Z0-9_]*$ string as an env-var marker. This misclassifies legitimate uppercase API tokens as non-secrets, so provider discovery for vllm/huggingface is invoked without auth (discoveryApiKey becomes undefined), which can cause authenticated /models discovery to fail even when credentials are actually configured.
Useful? React with 👍 / 👎.
| return { | ||
| apiKey: resolveNonEnvSecretRefApiKeyMarker(keyRef.source), | ||
| source: "non-env-ref", | ||
| }; |
There was a problem hiding this comment.
Keep resolved non-env ref keys for provider discovery
When an auth profile has a non-env keyRef/tokenRef plus a resolved runtime secret value, this branch returns only the persistence marker (secretref-managed) and discards the resolved value for discoveryApiKey. Downstream, resolveImplicitProviders uses discoveryApiKey for buildVllmProvider/buildHuggingfaceProvider, so secured discovery endpoints can return unauthenticated results (often empty model catalogs) despite the profile already containing usable runtime credentials.
Useful? React with 👍 / 👎.
mdlmarkham
left a comment
There was a problem hiding this comment.
Review: Secrets Hardening for models.json ✅
Verdict: Critical security fix. Essential for SecretRef integrity. Ready for merge.
Problem Addressed
SecretRef-managed credentials could still be persisted as plaintext in agents/*/agent/models.json. This defeats the purpose of SecretRef (secrets at rest, resolved at runtime).
Related issues:
Key Fixes
-
Marker-based persistence: SecretRef-managed
apiKeyvalues are persisted as markers (ENV_VAR_NAME,secretref-managed), not resolved secrets. -
Merge precedence fix: Non-empty
apiKeyinmodels.jsononly wins when provider is NOT SecretRef-managed. -
Audit coverage:
secrets auditnow scans generatedmodels.jsonfor:- Plaintext
apiKeyvalues - Sensitive provider header residues
- Plaintext
-
Deterministic serialization: Prevents timing attacks via file write patterns.
Security Impact
Before: Resolved secrets persisted on disk → credential exposure risk
After: Only markers persisted → secrets never touch disk in generated files
Test Coverage
The PR includes:
- Merge precedence tests for SecretRef-managed providers
- Marker persistence validation
- Audit integration tests
Edge Cases Handled
- Provider removed from config but
models.jsonstill has stale credentials → audit catches auth-profiles.jsonshadowing → precedence rules applied- Command paths (
openclaw agent) → same marker rules as gateway startup
Documentation
Good coverage:
- CLI docs updated (
agent.md,models.md,secrets.md) - Concepts updated (
models.md,secrets.md) - Reference updated (
secretref-credential-surface.md)
Minor Observations
-
+2,356 lines — Large change surface. Recommend careful review of:
models-json-snapshot.ts— persistence logicsecrets-audit.ts— audit coverage
-
File mode enforcement — PR mentions "file mode enforcement" but I don't see it in the diff preview. Is this in a separate file?
-
Backward compatibility — Does this break existing
models.jsonfiles with plaintext? They should be re-generated on next startup.
Recommendation: Approve. Critical security hardening, addresses real vulnerability in SecretRef persistence.
|
@mdlmarkham Hi! Maintainer here. Please stop. Do not release a code review bot on this project. It is not needed nor helpful. |
|
@greptile-apps Review |
|
@codex Review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 04a62df0b2
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
src/agents/model-auth-markers.ts
Outdated
| const KNOWN_ENV_API_KEY_MARKERS = new Set([ | ||
| "OPENAI_API_KEY", | ||
| "ANTHROPIC_API_KEY", | ||
| "ANTHROPIC_OAUTH_TOKEN", | ||
| "GEMINI_API_KEY", |
There was a problem hiding this comment.
Include all built-in env markers in allowlist
isNonSecretApiKeyMarker now relies on KNOWN_ENV_API_KEY_MARKERS, but that set omits env var names that resolveEnvApiKey can emit for built-in providers (for example MOONSHOT_API_KEY, VOYAGE_API_KEY, CLOUDFLARE_AI_GATEWAY_API_KEY, HUGGINGFACE_HUB_TOKEN, etc. in src/agents/model-auth.ts). When those providers are env-configured, models.json stores the env var name as the apiKey marker, and this function misclassifies it as plaintext, which causes false secrets audit findings and can make probe/auth paths treat markers as real credentials.
Useful? React with 👍 / 👎.
|
@greptile-apps Review |
|
@codex Review |
4378902 to
146e82f
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 146e82f8fa
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| configuredApiKeyRef.source === "env" | ||
| ? configuredApiKeyRef.id.trim() | ||
| : resolveNonEnvSecretRefApiKeyMarker(configuredApiKeyRef.source); |
There was a problem hiding this comment.
Encode env SecretRef apiKey markers unambiguously
normalizeProviders persists env-backed SecretRefs as the raw env var name (configuredApiKeyRef.id.trim()), but downstream marker detection (isNonSecretApiKeyMarker) only recognizes a fixed allowlist of names. For custom env refs (for example models.providers.custom.apiKey: {source:"env", id:"MY_TEAM_TOKEN"}), the persisted value is later treated as plaintext, which triggers false secrets audit leaks and can make probe/auth flows treat the marker as if it were a real credential string.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c5f1aa3603
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const mergedProviders = await resolveProvidersForMode({ | ||
| mode, | ||
| targetPath, | ||
| providers: normalizedProviders, | ||
| secretRefManagedProviders, |
There was a problem hiding this comment.
Re-normalize providers after merge-mode overlay
ensureOpenClawModelsJson now normalizes providers before calling resolveProvidersForMode, but the merge step can re-introduce raw entries from existing models.json and those entries are written without a second normalization pass. In merge mode, providers that exist only in the previous file (for example after being removed from config) can therefore retain unresolved SecretRef objects or template placeholders in persisted auth fields, which defeats the hardening goal and surfaces as unresolved/invalid credentials on later reads and audits.
Useful? React with 👍 / 👎.
c5f1aa3 to
d960bbd
Compare
d960bbd to
488b3fa
Compare
* origin/main: (37 commits) chore: land #39056 Node version hint sync (thanks @onstash) fix(security): use icacls /sid for locale-independent Windows ACL audit (#38900) fix(ci): refresh detect-secrets baseline offsets fix(security): strip custom auth headers on cross-origin redirects fix(ci): harden diffs viewer request guard and secret scan baseline Secrets: harden SecretRef-safe models.json persistence (#38955) docs(changelog): credit allowlist scoping report refactor(commands): dedupe onboard search perplexity test setup refactor(commands): dedupe message command secret-config tests refactor(cli): dedupe restart health probe setup tests refactor(cron): dedupe interim retry fallback assertions refactor(commands): dedupe model probe target test fixtures refactor(discord): dedupe message preflight test runners refactor(discord): share message handler test scaffolding refactor(discord): dedupe native command ACP routing test setup refactor(slack): dedupe app mention in-flight race setup refactor(slack): reuse shared prepare test scaffolding refactor(plugin-sdk): extract shared channel prelude exports refactor(slack): dedupe app mention race test setup refactor(line): dedupe replay webhook test fixtures ... # Conflicts: # .secrets.baseline # apps/ios/fastlane/Fastfile # src/commands/message.test.ts
(cherry picked from commit 8e20dd2)
(cherry picked from commit 8e20dd2)
Summary
This PR implements a single end-to-end fix path for the models.json plaintext-persistence cluster and related hardening surfaced in the 2026-03-06 external secrets audit.
Primary cluster addressed:
Problem surface addressed
agents/*/agent/models.json.apiKeyvalues even when a provider is SecretRef-managed.models.jsonresidues for providerapiKeyand sensitive provider headers.What changed
models.mode=mergebehavior:apiKeyonly when provider is not SecretRef-managed in current config/auth-profile contextmodelscommand load path uses source+resolved snapshot pairingopenclaw agentpath now sets runtime snapshot with source snapshot before embedded runner pathmodels.providers.*.headers.*secrets audit/applyscans and remediates generatedmodels.jsonresidues forapiKey+ sensitive headersmodels.jsonwrites:0600mode on write and no-content-change pathsawait priornow insidetry/finally)Regression resistance
Added/updated tests across:
openclaw agent)models.jsonapiKeyand headersValidation
Passing focused suites:
pnpm test -- src/commands/agent.test.ts src/commands/models/load-config.test.ts src/agents/models-config.providers.auth-provenance.test.ts src/agents/models-config.runtime-source-snapshot.test.ts src/media-understanding/runner.deepgram.test.tspnpm test -- src/agents/model-auth-markers.test.ts src/infra/provider-usage.auth.normalizes-keys.test.ts src/commands/models/list.probe.targets.test.ts src/secrets/audit.test.ts src/secrets/apply.test.tspnpm test -- src/agents/models-config.write-serialization.test.ts src/agents/models-config.file-mode.test.tspnpm checkNotes