fix(gateway): prevent per-turn agent eviction from model normalization mismatch and signature instability#21299
fix(gateway): prevent per-turn agent eviction from model normalization mismatch and signature instability#21299fayenix wants to merge 4 commits into
Conversation
Root cause: returns vendor-prefixed model name
(e.g. 'deepseek/deepseek-v4-pro') while AIAgent.__init__ normalizes it by
stripping the prefix ('deepseek-v4-pro'). The comparison at run.py:13131
was always True, causing _evict_cached_agent to fire after EVERY turn.
Fix: normalize _cfg_model the same way AIAgent.__init__ does (importing
normalize_model_for_provider from hermes_cli.model_normalize) before
the equality check. Also add debug logging to _evict_cached_agent so
evictions are traceable in gateway logs.
- gateway/run.py: add model normalization block at line 13131
- gateway/run.py: add logger.debug to _evict_cached_agent
Thanks to Claude for the root-cause analysis and fix.
…g agent cache signature Two independent fixes for session signature instability causing spurious agent evictions mid-conversation: 1. gateway/run.py: Decouple was_auto_reset notice from context_prompt. Previously, the '[System note: session suspended/reset...]' notice was prepended to context_prompt, which feeds into the cache signature hash. Since was_auto_reset is a one-turn transient flag, the sig would flip on the first post-restart/reset turn, evict the cached agent, then flip back on the next turn — causing back-to-back CACHE MISSes (the ec05b8ea→67f721c3→f3d18768→67f721c3 flip-flop). Fix: store as _reset_context_note without modifying context_prompt. Inject into combined_ephemeral AFTER _sig is computed, so the agent still sees the notice but the sig stays stable. 2. gateway/session.py: Remove 'Triggering message: <message_id>' from the Discord IDs block in build_session_context_prompt(). message_id is per-message and would cause _sig to change every single turn if the discord/discord_admin toolset is ever enabled (latent bomb). Root cause analysis: Claude (via fayenix) Branch: fix/discord-voice-fixes
…ping bug) The _reset_context_note variable was defined in _handle_message_with_agent but referenced in run_sync, a nested function inside _run_agent. Python closures can see variables from enclosing scopes, but only up to the nearest enclosing function — run_sync cannot see _reset_context_note because it's in a different function (_handle_message_with_agent), not in _run_agent. Fix: add reset_context_note as a parameter to _run_agent (line 11616), pass it from the call site (line 6070), and reference the parameter name in run_sync instead of the outer function's local. Fixes: NameError "name '_reset_context_note' is not defined" Bug introduced in: d480b91
liuhao1024
left a comment
There was a problem hiding this comment.
The overall fix — deferring the reset-context-note injection past _sig computation and normalizing _cfg_model — looks correct and well-motivated.
One bug on the diagnostic logging path:
_cached_for_log = cached if '_cache_lock' in dir() and cached is not None else Nonedir() (with no arguments) returns local scope names. There is no local variable named _cache_lock in this scope — the lock is _agent_cache_lock — so '_cache_lock' in dir() is always False. This means _cached_for_log is always None and the "CACHE MISS: sig changed (old=%s new=%s)" log will never fire, even when a signature mismatch is the actual cause.
Fix — simplify to just check cached directly:
_cached_for_log = cached
if _cached_for_log:
logger.info("CACHE MISS: sig changed (old=%s new=%s)",
_cached_for_log[1][:8], _sig[:8])
else:
logger.info("CACHE MISS: no cached agent for session %s (sig=%s)", session_key, _sig[:8])
Confirming this bug from a different angle — config model change not taking effect in existing sessionWe hit this in production on our Weixin (WeChat) gateway today. Symptoms: Setup:
Observed:
Resolution:
Root cause analysis: The cached if _agent.model != _cfg_model:
_evict_cached_agent()…failed to trigger because of the normalization mismatch described in this PR. The
This PR's fix to normalize Environment: macOS, gateway with telegram + feishu + weixin platforms, hermes-agent from fork. |
liuhao1024 caught that '_cache_lock' in dir() always returns False because dir() with no arguments checks local scope, and the lock is _agent_cache_lock. Simplify to check cached directly so the 'CACHE MISS: sig changed' log actually fires when it should. Co-authored-by: liuhao1024 <[email protected]>
fayenix
left a comment
There was a problem hiding this comment.
Good catch, thank you! Fixed in 6d2c34e — simplified to just check cached directly. You were absolutely right that dir() with no args only sees local scope, and the lock is _agent_cache_lock, not _cache_lock — so the diagnostic log literally never fired.
Summary
Fixes the root cause of per-turn agent eviction that causes memory provider bridges to spawn on every message (issue #20939).
Two independent bugs in the gateway agent cache were causing the cached
AIAgent(and itsMemoryProvider) to be destroyed and recreated on every turn instead of persisting across messages:_agent.model(normalized byAIAgent.__init__) never matched_cfg_model(raw gateway config with vendor prefix), so the eviction check at ~L13131 fired every turn.was_auto_resetnotice andmessage_idwere feeding into the cache signature hash, causing it to flip between turns.Root Cause Details
1. Model Normalization Mismatch
gateway/run.py~L13131 compares the cached agent's model against the gateway config model to decide whether to evict:_agent.model="deepseek-v4-pro"(normalized byAIAgent.__init__vianormalize_model_for_provider)_cfg_model="deepseek/deepseek-v4-pro"(raw gateway config, vendor-prefixed)"deepseek-v4-pro" != "deepseek/deepseek-v4-pro"→_evict_cached_agent()→AIAgentdestroyed →MemoryProviderdestroyed → next turn creates freshMemoryProvider→initialize()→ new bridge pair. Every. Single. Turn.Fix: Normalize
_cfg_modelusing the samenormalize_model_for_provider()function thatAIAgent.__init__uses, before the equality check.2. Session Signature Instability
Even after the model fix, the cache signature was flip-flopping (
ec05b8ea → 67f721c3 → f3d18768 → 67f721c3). Two unstable inputs were feeding the sig hash:was_auto_resetnotice was prepended tocontext_prompt, which feeds into the sig hash. Being a one-turn transient flag, it flipped the sig on the first post-restart turn, then back.message_idwas in the Discord IDs block inbuild_session_context_prompt(). Per-message value would change the sig every turn.Fix: Decouple the reset notice from
context_prompt(inject intocombined_ephemeralafter sig is computed). Removemessage_idfrom the context prompt.Results (live gateway, ~16 hour burn-in)
Commits
d480b910b81ca7089f83276fb4cd7e2683c3Related
MemoryProviderlayer (belt); this PR fixes the root cause of unnecessary agent evictions at the gateway layer (suspenders). Both should be applied.