Skip to content

fix(secrets): prevent unresolved SecretRef from crashing embedded agent runs#53577

Merged
obviyus merged 2 commits intoopenclaw:mainfrom
Lukavyi:fix-secretref-message-tool
Mar 24, 2026
Merged

fix(secrets): prevent unresolved SecretRef from crashing embedded agent runs#53577
obviyus merged 2 commits intoopenclaw:mainfrom
Lukavyi:fix-secretref-message-tool

Conversation

@Lukavyi
Copy link
Copy Markdown
Contributor

@Lukavyi Lukavyi commented Mar 24, 2026

Before

CleanShot 2026-03-24 at 12 00 15@2x

After

CleanShot 2026-03-24 at 12 00 24@2x

Summary

When channel monitors (Telegram, Slack, Discord) start, they capture a config snapshot via loadConfig(). If this happens before the secrets runtime snapshot is activated, the captured config still contains raw SecretRef objects (e.g. {source: "exec", provider: "doppler", id: "TG_PERSONAL_BOT_TOKEN"}).

This captured config is then passed as configOverride through the dispatch chain into getReplyFromConfig(), which uses it directly — skipping the snapshot-aware loadConfig() that would return resolved secrets. The stale config propagates into FollowupRun.run.config and eventually crashes runEmbeddedPiAgent() with:

channels.telegram.accounts.personal.botToken: unresolved SecretRef "exec:doppler:TG_PERSONAL_BOT_TOKEN".
Resolve this command against an active gateway runtime snapshot before reading it.

This affects all channels using exec:, env:, or file: SecretRef providers for bot tokens.

Root cause

The dispatch chain from channel monitor to embedded agent:

  1. server-channels.ts: const cfg = loadConfig() at startup (before secrets resolved)
  2. Telegram monitor stores cfg in long-lived closure
  3. bot-message-dispatch.ts: passes stale cfg as configOverride
  4. get-reply.ts:108: const cfg = configOverride ?? loadConfig() — since configOverride exists, loadConfig() is never called
  5. get-reply-run.ts: stores in FollowupRun.run.config
  6. runEmbeddedPiAgent() receives unresolved config — crash

Fix

Add hasUnresolvedChannelSecrets() check in getReplyFromConfig(). When configOverride contains unresolved SecretRef objects in channel token fields, discard it and fall back to loadConfig() which returns the resolved runtime snapshot.

Test plan

  • Verified /new and /reset commands no longer crash with SecretRef error
  • Verified normal message flow works after fix
  • pnpm lint clean
  • pnpm check clean (no new type errors)
  • Tested on real setup with exec:doppler SecretRef for Telegram bot token

Related issues

Codebase search

  • Traced full dispatch chain: server-channels.ts -> channel.ts -> bot-message-dispatch.ts -> dispatch-from-config.ts -> get-reply.ts -> get-reply-run.ts -> runEmbeddedPiAgent
  • Verified runtimeConfigSnapshot is set correctly but bypassed via configOverride
  • Added debug instrumentation (PID, moduleId, clear traces) confirming same-process, no clear, snapshot set but unused

lobster-biscuit

Sign-Off

AI-assisted: Claude Opus 4 + GPT-5.4 (via OpenClaw). Fully tested on real setup. The author understands the code and the fix.

@openclaw-barnacle openclaw-barnacle bot added agents Agent runtime and tooling size: M labels Mar 24, 2026
@Lukavyi Lukavyi force-pushed the fix-secretref-message-tool branch 3 times, most recently from e806462 to 2fb515f Compare March 24, 2026 10:58
@openclaw-barnacle openclaw-barnacle bot added size: S and removed agents Agent runtime and tooling size: M labels Mar 24, 2026
@Lukavyi Lukavyi marked this pull request as ready for review March 24, 2026 10:59
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 24, 2026

Greptile Summary

This PR fixes a crash in getReplyFromConfig() where channel monitors (Telegram, Slack, Discord) capture a config snapshot at startup via loadConfig() before the secrets runtime snapshot is activated. The captured config contains raw SecretRef objects that then propagate through the dispatch chain as configOverride, bypassing the snapshot-aware loadConfig() and ultimately crashing runEmbeddedPiAgent() with an "unresolved SecretRef" error.

The fix introduces a hasUnresolvedChannelSecrets() guard that inspects the configOverride for unresolved SecretRef objects in channel token fields. When detected, the stale override is discarded and loadConfig() is used instead, which returns the correctly resolved runtime snapshot.

Key changes:

  • New hasUnresolvedChannelSecrets() function that checks Telegram botToken (top-level and per-account), and Slack/Discord botToken, appToken, token, and userToken fields.
  • getReplyFromConfig() now conditionally discards a stale configOverride and logs a diagnostic error when fallback is triggered.
  • userToken (previously flagged in review) is now included in the Slack/Discord token-key scan.

Confidence Score: 5/5

  • This PR is safe to merge — it eliminates a real crash with no possible regression, since discarding a config that would crash later is strictly better than keeping it.
  • The fix is narrowly scoped and provably safe: any configOverride that contains an unresolved SecretRef in a channel token field would crash runEmbeddedPiAgent() without this guard, so falling back to loadConfig() is always an improvement. The coerceSecretRef utility is battle-tested and correctly detects all three unresolved-secret forms (plain SecretRef object, legacy form without provider, and ${ENV_VAR} template strings). The userToken concern from the prior review thread has been addressed. pnpm lint and pnpm check are clean. No regressions are possible for callers who supply a valid, fully-resolved configOverride.
  • No files require special attention.

Reviews (4): Last reviewed commit: "fix(secrets): prevent unresolved SecretR..." | Re-trigger Greptile

@Lukavyi Lukavyi force-pushed the fix-secretref-message-tool branch from 2fb515f to abb1863 Compare March 24, 2026 11:05
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2fb515f7ef

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@Lukavyi
Copy link
Copy Markdown
Contributor Author

Lukavyi commented Mar 24, 2026

@codex review
@greptile review

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@Lukavyi Lukavyi force-pushed the fix-secretref-message-tool branch from abb1863 to 51b5d18 Compare March 24, 2026 11:23
@Lukavyi
Copy link
Copy Markdown
Contributor Author

Lukavyi commented Mar 24, 2026

@greptile review

@Lukavyi Lukavyi force-pushed the fix-secretref-message-tool branch from 51b5d18 to 6cd84f1 Compare March 24, 2026 11:34
@Lukavyi
Copy link
Copy Markdown
Contributor Author

Lukavyi commented Mar 24, 2026

@greptile review

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6cd84f104f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@obviyus obviyus force-pushed the fix-secretref-message-tool branch from 6cd84f1 to 5008781 Compare March 24, 2026 12:58
@obviyus obviyus self-assigned this Mar 24, 2026
@obviyus obviyus force-pushed the fix-secretref-message-tool branch from 5008781 to 1749ee7 Compare March 24, 2026 13:05
@openclaw-barnacle openclaw-barnacle bot added channel: msteams Channel integration: msteams size: S and removed size: M labels Mar 24, 2026
@obviyus
Copy link
Copy Markdown
Contributor

obviyus commented Mar 24, 2026

Reworked this to fix the contract, not patch stale configs in getReplyFromConfig().

The real issue was that dispatchReplyFromConfig() always forwarded cfg as if it were a real configOverride, so long-lived channel monitors could leak their startup snapshot into the reply path. This rewrite stops passing cfg implicitly, makes getReplyFromConfig() start from fresh loadConfig(), and only merges an explicit configOverride when a caller actually has a per-message patch to apply.

For this PR, MSTeams now passes only its sender-timezone patch as configOverride, and the normal path no longer needs channel-specific stale-secret detection.

…nt runs

Root cause: Telegram channel monitor captures config at startup before secrets
are resolved and passes it as configOverride into the reply pipeline. Since
getReplyFromConfig() uses configOverride directly (skipping loadConfig() which
reads the resolved runtime snapshot), the unresolved SecretRef objects propagate
into FollowupRun.run.config and crash runEmbeddedPiAgent().

Fix (defense in depth):
- get-reply.ts: detect unresolved SecretRefs in configOverride and fall back to
  loadConfig() which returns the resolved runtime snapshot
- message-tool.ts: try-catch around schema/description building at tool creation
  time so channel discovery errors don't crash the agent
- message-tool.ts: detect unresolved SecretRefs in pre-bound config at tool
  execution time and fall back to gateway secret resolution

Fixes: openclaw#45838
@obviyus obviyus force-pushed the fix-secretref-message-tool branch from 1749ee7 to 753531b Compare March 24, 2026 13:21
@obviyus obviyus merged commit 0a04ef4 into openclaw:main Mar 24, 2026
18 checks passed
@obviyus
Copy link
Copy Markdown
Contributor

obviyus commented Mar 24, 2026

Landed on main.

Thanks @Lukavyi.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 753531b7db

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines 591 to 595
senderTimezone && !cfg.agents?.defaults?.userTimezone
? {
...cfg,
agents: {
...cfg.agents,
defaults: { ...cfg.agents?.defaults, userTimezone: senderTimezone },
},
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid copying stale agent defaults into Teams configOverride

createMSTeamsMessageHandler captures cfg once at startup, but this override now gets merged onto a fresh loadConfig() result in getReplyFromConfig; spreading cfg.agents?.defaults here re-applies the startup snapshot’s entire defaults object on every turn with senderTimezone. That means runtime config updates (for example agents.defaults.model, timeout, or other defaults changed without restart) are silently overwritten for those messages, so Teams can keep using outdated agent defaults even though the new dispatch path is intended to use fresh config.

Useful? React with 👍 / 👎.

@aisle-research-bot
Copy link
Copy Markdown

aisle-research-bot bot commented Mar 24, 2026

🔒 Aisle Security Analysis

We found 1 potential security issue(s) in this PR:

# Severity Title
1 🔵 Low Tenant/instance config bypass: dispatchReplyFromConfig no longer passes resolved cfg into getReplyFromConfig
Vulnerabilities

1. 🔵 Tenant/instance config bypass: dispatchReplyFromConfig no longer passes resolved cfg into getReplyFromConfig

Property Value
Severity Low
CWE CWE-284
Location src/auto-reply/reply/dispatch-from-config.ts:581-647

Description

dispatchReplyFromConfig previously ensured the reply resolver used the already-resolved configuration (cfg) by passing it as a fallback when no override was provided. The change now passes undefined when no override is set.

As a result, getReplyFromConfig will call loadConfig() again, ignoring the cfg object that was provided to dispatchReplyFromConfig.

This is a security risk in deployments where cfg is derived from a restricted/tenant-scoped configuration (e.g., plugin/extension supplies an account-specific config) or where cfg has been sanitized/validated earlier in the request:

  • Input/context: upstream code supplies params.cfg to dispatchReplyFromConfig (potentially tenant-scoped / validated)
  • Behavior change: when params.configOverride is not supplied, the resolver now sees configOverride == null and reloads global config via loadConfig() rather than using the supplied cfg
  • Impact: can cause cross-tenant/config confusion (wrong agent/channel settings, credentials, authorization rules, routing), potentially leading to broken access control or unintended use of privileged configuration

Vulnerable code (new behavior):

// dispatch-from-config.ts
const replyResult = await replyResolver(
  ctx,
  { /* opts */ },
  params.configOverride, // now undefined when no override
);// get-reply.ts
const cfg =
  configOverride == null
    ? loadConfig() // reloads instead of using the already-resolved cfg
    : applyMergePatch(loadConfig(), configOverride);

Recommendation

Ensure the reply resolver uses the same configuration object that dispatchReplyFromConfig was invoked with.

Option A (restore previous safe behavior):

await replyResolver(ctx, opts, params.configOverride ?? cfg);

Option B (change resolver API to accept cfg directly):

  • Update getReplyFromConfig (or the runtime wrapper) to accept a resolved cfg parameter and do not call loadConfig() internally.
  • Only apply overrides as a merge on top of the already-resolved cfg.

This prevents configuration confusion and ensures tenant/account scoping and validation performed upstream cannot be bypassed by a fresh global config reload.


Analyzed PR: #53577 at commit 753531b

Last updated on: 2026-03-24T16:14:54Z

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: msteams Channel integration: msteams size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SecretRef resolution fails at reply time for channel tokens (Slack/Telegram)

2 participants