Skip to content

[Bug]: reply-path secrets.resolve floods gateway log with UNAVAILABLE for model-provider exec SecretRefs (degrades correctly, ~1 line/turn) #96653

Description

@willfivestar

Summary

On every agent reply turn, the gateway logs a noisy failure for secrets.resolve when a model-provider credential is an exec SecretRef, even though the gateway resolved that same secret fine at startup and the turn proceeds normally via local fallback. With an active exec-backed models.providers.<name>.apiKey, this produces ~1 log line per turn (thousands over time):

[ws] ⇄ res ✗ secrets.resolve 6ms errorCode=UNAVAILABLE errorMessage=secrets.resolve failed conn=… id=…

This is the graceful-degrade sibling of the hard-crash exec-SecretRef issues (#50161, #45416, #57272, #82621): here nothing breaks, but the path is needlessly noisy and the real error is invisible (see point 3).

Environment

  • openclaw 2026.6.10, macOS (loopback gateway)
  • models.providers.<name>.apiKey = { source: "exec", provider: "<exec-provider>", id: "value" } (any active exec-backed agent-runtime SecretRef triggers it)

What happens

  1. Every reply calls resolveQueuedReplyExecutionConfig (src/auto-reply/reply/agent-runner-utils.ts) → resolveCommandSecretRefsViaGateway({ commandName: "reply", targetIds: getAgentRuntimeCommandSecretTargetIds() }), which issues a per-turn secrets.resolve RPC over a fresh WS connection.
  2. The gateway handler (src/gateway/server-methods/secrets.ts) tries to resolve the exec-backed models.providers.<name>.apiKey from the active snapshot, throws, and responds UNAVAILABLE "secrets.resolve failed".
  3. The client catches it and falls back to local resolution, which succeeds. Verified by calling the client directly:
    • targetStatesByPath: { "models.providers.<name>.apiKey": "resolved_local" }
    • hadUnresolvedTargets: false
    • diagnostic: "reply: gateway secrets.resolve unavailable (secrets.resolve failed); resolved command secrets locally."

So the secret resolves correctly every turn — only the log is polluted, and the ⚠️ UNAVAILABLE line is misleading during real debugging.

Root cause

resolveCommandSecretRefsViaGateway already skips the gateway RPC and goes straight to local resolution when the configured exec refs are gateway-credential paths (collectActiveGatewayExecSecretRefCredentialPaths). But that skip only covers gateway.auth.*/gateway.remote.*not model-provider / agent-runtime exec refs. So for models.providers.*.apiKey exec refs it always attempts the gateway RPC, which always fails (command-path resolution won't execute exec providers), then always falls back locally.

Two asks

  1. Skip the gateway RPC (or degrade silently) for model-provider/agent-runtime exec refs, the same way it's already skipped for gateway-credential exec refs — since command-path resolution can't serve exec refs anyway, the round-trip is pure overhead + noise.
  2. The real failure reason is unobservable. The handler logs it via params.log?.warn?.(secrets.resolve failed: <reason>), but on a default gateway launch that warn goes to stderr → /dev/null, so operators only ever see the generic UNAVAILABLE. Consider surfacing the underlying reason on the WS error payload (or at info level) so this is diagnosable without source-diving.

Workaround

None clean. Switching the source to env/file (snapshot-resolvable) silences it but adds launch-env wiring and a security trade-off for a cosmetic gain. The behavior is benign, so most operators can ignore it — but it's confusing and floods logs.

Metadata

Metadata

Assignees

Labels

P2Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.clawsweeper:queueable-fixClawSweeper marked this issue as an existing queue_fix_pr work candidate.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.no-staleExclude from stale automation

Type

No type

Fields

Priority

None yet

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions