Skip to content

fix(gateway): cache eviction caused by signature mismatch and scoping bugs#28846

Closed
fayenix wants to merge 7 commits into
NousResearch:mainfrom
chiefmojo:pr/cache-eviction
Closed

fix(gateway): cache eviction caused by signature mismatch and scoping bugs#28846
fayenix wants to merge 7 commits into
NousResearch:mainfrom
chiefmojo:pr/cache-eviction

Conversation

@fayenix

@fayenix fayenix commented May 19, 2026

Copy link
Copy Markdown
Contributor

Summary

Post-turn agent cache eviction is silently triggered by two independent bugs:

  1. Model normalization mismatch: The cache key is built using the raw model string, but _run_agent() normalizes it (lowercase, provider prefix). When the normalized form differs from the raw string, the signature check fails and the agent is evicted — even though nothing actually changed.

  2. was_auto_reset and message_id destabilization: These per-request fields are included in the cache signature but change between messages, causing false-positive eviction on every turn.

  3. reset_context_note scoping bug: The variable is referenced from an outer scope in _run_agent() instead of being passed as a parameter, causing UnboundLocalError in edge cases.

  4. Diagnostic log bug: The cache eviction debug log prints the wrong variable, making it impossible to diagnose why eviction occurred.

Changes

  • Normalize model string consistently in the cache key builder.
  • Exclude was_auto_reset and message_id from the cache signature.
  • Pass reset_context_note as an explicit parameter to _run_agent().
  • Fix the diagnostic log to print the actual signature values.

Testing

  • Manual: confirmed agents remain cached across turns with same model.
  • Diagnostic logs now show correct before/after signatures.

Related

Resubmission of #21299 (closed without merge — commits preserved locally, re-cherry-picked onto current main).

@alt-glitch alt-glitch added type/bug Something isn't working P1 High — major feature broken, no workaround comp/gateway Gateway runner, session dispatch, delivery labels May 19, 2026
fayenix and others added 4 commits May 22, 2026 21:41
@fayenix
fix(gateway): prevent silent post-turn agent eviction from model norm…
fdd5ae2 
…alization mismatch
@fayenix
fix(gateway): prevent was_auto_reset and message_id from destabilizin…
622643f 
…g agent cache signature

Two independent fixes for session signature instability causing
spurious agent evictions mid-conversation:

1. gateway/run.py: Decouple was_auto_reset notice from context_prompt.
   Previously, the '[System note: session suspended/reset...]' notice
   was prepended to context_prompt, which feeds into the cache signature
   hash. Since was_auto_reset is a one-turn transient flag, the sig
   would flip on the first post-restart/reset turn, evict the cached
   agent, then flip back on the next turn — causing back-to-back
   CACHE MISSes (the ec05b8ea→67f721c3→f3d18768→67f721c3 flip-flop).

   Fix: store as _reset_context_note without modifying context_prompt.
   Inject into combined_ephemeral AFTER _sig is computed, so the agent
   still sees the notice but the sig stays stable.

2. gateway/session.py: Remove 'Triggering message: <message_id>' from
   the Discord IDs block in build_session_context_prompt(). message_id
   is per-message and would cause _sig to change every single turn if
   the discord/discord_admin toolset is ever enabled (latent bomb).

Root cause analysis: Claude (via fayenix)
Branch: fix/discord-voice-fixes
@fayenix
fix(gateway): pass reset_context_note as parameter to _run_agent (sco…
9fcd40a 
…ping bug)

The _reset_context_note variable was defined in _handle_message_with_agent
but referenced in run_sync, a nested function inside _run_agent. Python
closures can see variables from enclosing scopes, but only up to the
nearest enclosing function — run_sync cannot see _reset_context_note
because it's in a different function (_handle_message_with_agent), not
in _run_agent.

Fix: add reset_context_note as a parameter to _run_agent (line 11616),
pass it from the call site (line 6070), and reference the parameter
name in run_sync instead of the outer function's local.

Fixes: NameError "name '_reset_context_note' is not defined"
Bug introduced in: d480b91
@fayenix @liuhao1024
fix(gateway): fix diagnostic log for cache eviction signature mismatch
6ca7b4d 
liuhao1024 caught that '_cache_lock' in dir() always returns False
because dir() with no arguments checks local scope, and the lock is
_agent_cache_lock. Simplify to check cached directly so the
'CACHE MISS: sig changed' log actually fires when it should.

Co-authored-by: liuhao1024 <[email protected]>
@fayenix
fayenix force-pushed the pr/cache-eviction branch from 0a5bbf8 to 6ca7b4d Compare May 23, 2026 04:41
teknium1
teknium1 reviewed Jun 15, 2026
View reviewed changes

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for resubmitting this. The premise is real on current main, but I found two blocking issues in the patch shape.

Problems

  • gateway/run.py:16297 moves the auto-reset note after signature computation. Current main creates a fresh session with the same session_key at gateway/session.py:950-963, and the cache signature does not include session_id (gateway/run.py:12583-12600, call at gateway/run.py:14363-14371). If the old cached agent hits, it can survive across the reset boundary with the old session_id and transcript.
  • gateway/run.py:16286 reads cached outside the only branch that assigns it. Cache-disabled runners exist in tests, e.g. tests/gateway/test_proxy_mode.py:21-23, so this can raise UnboundLocalError when _agent_cache_lock is absent.
  • The PR currently has manual testing only. Existing cache tests cover generic signatures (tests/gateway/test_agent_cache.py:30-46) but not the model-normalization, Discord message_id, or auto-reset boundary cases changed here.

Suggested changes

  • Keep auto-reset as an explicit cache boundary: evict before _run_agent, or include a stable boundary value like session_id in the signature.
  • Initialize cached = None before the cache lookup block.
  • Add targeted regression tests for the three behaviors above.

Automated hermes-sweeper review.

Comment thread gateway/run.py
Comment thread gateway/run.py
@fayenix
fayenix force-pushed the pr/cache-eviction branch from 4f13612 to d2c72c8 Compare June 21, 2026 22:54
@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit labels Jun 29, 2026
@teknium1 teknium1 mentioned this pull request Jun 30, 2026
Merged
teknium1 pushed a commit that referenced this pull request Jun 30, 2026
@fayenix @Rafa-Ross @teknium1
fix(gateway): stop per-turn agent-cache eviction from model + message…
61f5d97 
…_id signature churn

Two independent bugs evicted the cached gateway AIAgent on every turn,
preventing the prompt cache from ever warming:

1. Model normalization mismatch: the post-run fallback-eviction check
   compared _agent.model (stripped in AIAgent.__init__) against the raw
   _resolve_gateway_model() config string. For vendor-prefixed config on
   native providers (e.g. 'deepseek/deepseek-v4-pro' vs 'deepseek-v4-pro')
   this was always unequal, so the agent was evicted after every
   successful run. Normalize _cfg_model the same way (skip aggregators).

2. Discord triggering message_id leaked into the cached system prompt via
   build_session_context_prompt()'s Discord IDs block. message_id changes
   every turn, so the agent-cache signature (computed from the ephemeral
   prompt) changed every Discord turn -> rebuild every message. The id is
   now injected per-turn into the user message (where per-turn content
   belongs and does not touch the cache signature); the cached IDs block
   carries a static pointer to it, preserving reply/react/pin via the
   discord tools.

Adapted from #28846. Bug #1 fix is the contributor's; bug #2 reworked to
be non-destructive (keeps the triggering-id capability instead of deleting
it). Redundant auto-reset eviction (already on main via #9893/#48031) and
the wrong-premise reset_context_note plumbing from the original PR were
dropped.

Co-authored-by: Hermes Agent <[email protected]>
teknium1 pushed a commit that referenced this pull request Jun 30, 2026
@fayenix @Rafa-Ross @teknium1
fix(gateway): stop per-turn agent-cache eviction from model + message…
d6c53dc 
…_id signature churn

Two independent bugs evicted the cached gateway AIAgent on every turn,
preventing the prompt cache from ever warming:

1. Model normalization mismatch: the post-run fallback-eviction check
   compared _agent.model (stripped in AIAgent.__init__) against the raw
   _resolve_gateway_model() config string. For vendor-prefixed config on
   native providers (e.g. 'deepseek/deepseek-v4-pro' vs 'deepseek-v4-pro')
   this was always unequal, so the agent was evicted after every
   successful run. Normalize _cfg_model the same way (skip aggregators).

2. Discord triggering message_id leaked into the cached system prompt via
   build_session_context_prompt()'s Discord IDs block. message_id changes
   every turn, so the agent-cache signature (computed from the ephemeral
   prompt) changed every Discord turn -> rebuild every message. The id is
   now injected per-turn into the user message (where per-turn content
   belongs and does not touch the cache signature); the cached IDs block
   carries a static pointer to it, preserving reply/react/pin via the
   discord tools.

Adapted from #28846. Bug #1 fix is the contributor's; bug #2 reworked to
be non-destructive (keeps the triggering-id capability instead of deleting
it). Redundant auto-reset eviction (already on main via #9893/#48031) and
the wrong-premise reset_context_note plumbing from the original PR were
dropped.

Co-authored-by: Hermes Agent <[email protected]>
@teknium1

teknium1 commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Merged via #55595 — your _cfg_model normalization fix was cherry-picked onto current main with your authorship preserved in git log (commit d6c53dc).

Thanks for catching the per-turn eviction. Notes on what landed vs. what we dropped:

  • Bug 1 (model normalization mismatch): merged as-is — this was the real, non-redundant fix. _cfg_model is now normalized the same way AIAgent.__init__ strips it, so vendor-prefixed native-provider config (deepseek/deepseek-v4-pro) matches agent.model and no longer evicts every turn. Aggregators left untouched.
  • Bug 2 (message_id signature churn): kept the diagnosis, but reworked the fix to be non-destructive. Deleting the "Triggering message" line removed a real capability (reply/react/pin via the discord tools). Instead we moved the volatile message_id out of the cached system prompt and inject it per-turn into the user message, with a static pointer left in the IDs block — preserves both the cache stability and the feature. Added a regression test locking the invariant.
  • Auto-reset eviction (_clear_auto_reset_session_state + the auto-reset _evict_cached_agent): already on current main via Gateway becomes unresponsive when context compression is exhausted in long-running group chat sessions #9893//model switch silently lost when first message after session auto-reset (was_auto_reset flag never consumed) #48031, so dropped as redundant.
  • reset_context_note "scoping bug": didn't reproduce on current main (the variable is assigned and used in the same scope), so that plumbing was dropped.

Appreciate the contribution.

@teknium1 teknium1 closed this Jun 30, 2026
dtera pushed a commit to dtera/hermes-agent that referenced this pull request Jul 1, 2026
@fayenix @Rafa-Ross dterazhao
fix(gateway): stop per-turn agent-cache eviction from model + message…
682d5df 
…_id signature churn

Two independent bugs evicted the cached gateway AIAgent on every turn,
preventing the prompt cache from ever warming:

1. Model normalization mismatch: the post-run fallback-eviction check
   compared _agent.model (stripped in AIAgent.__init__) against the raw
   _resolve_gateway_model() config string. For vendor-prefixed config on
   native providers (e.g. 'deepseek/deepseek-v4-pro' vs 'deepseek-v4-pro')
   this was always unequal, so the agent was evicted after every
   successful run. Normalize _cfg_model the same way (skip aggregators).

2. Discord triggering message_id leaked into the cached system prompt via
   build_session_context_prompt()'s Discord IDs block. message_id changes
   every turn, so the agent-cache signature (computed from the ephemeral
   prompt) changed every Discord turn -> rebuild every message. The id is
   now injected per-turn into the user message (where per-turn content
   belongs and does not touch the cache signature); the cached IDs block
   carries a static pointer to it, preserving reply/react/pin via the
   discord tools.

Adapted from NousResearch#28846. Bug #1 fix is the contributor's; bug NousResearch#2 reworked to
be non-destructive (keeps the triggering-id capability instead of deleting
it). Redundant auto-reset eviction (already on main via NousResearch#9893/NousResearch#48031) and
the wrong-premise reset_context_note plumbing from the original PR were
dropped.

Co-authored-by: Hermes Agent <[email protected]>
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
@fayenix @Rafa-Ross @teknium1
fix(gateway): stop per-turn agent-cache eviction from model + message…
6ac37f0 
…_id signature churn

Two independent bugs evicted the cached gateway AIAgent on every turn,
preventing the prompt cache from ever warming:

1. Model normalization mismatch: the post-run fallback-eviction check
   compared _agent.model (stripped in AIAgent.__init__) against the raw
   _resolve_gateway_model() config string. For vendor-prefixed config on
   native providers (e.g. 'deepseek/deepseek-v4-pro' vs 'deepseek-v4-pro')
   this was always unequal, so the agent was evicted after every
   successful run. Normalize _cfg_model the same way (skip aggregators).

2. Discord triggering message_id leaked into the cached system prompt via
   build_session_context_prompt()'s Discord IDs block. message_id changes
   every turn, so the agent-cache signature (computed from the ephemeral
   prompt) changed every Discord turn -> rebuild every message. The id is
   now injected per-turn into the user message (where per-turn content
   belongs and does not touch the cache signature); the cached IDs block
   carries a static pointer to it, preserving reply/react/pin via the
   discord tools.

Adapted from NousResearch#28846. Bug #1 fix is the contributor's; bug #2 reworked to
be non-destructive (keeps the triggering-id capability instead of deleting
it). Redundant auto-reset eviction (already on main via NousResearch#9893/NousResearch#48031) and
the wrong-premise reset_context_note plumbing from the original PR were
dropped.

Co-authored-by: Hermes Agent <[email protected]>
liuchanchen pushed a commit to liuchanchen/hermes-agent that referenced this pull request Jul 3, 2026
@fayenix @Rafa-Ross @teknium1
fix(gateway): stop per-turn agent-cache eviction from model + message…
0c348b8 
…_id signature churn

Two independent bugs evicted the cached gateway AIAgent on every turn,
preventing the prompt cache from ever warming:

1. Model normalization mismatch: the post-run fallback-eviction check
   compared _agent.model (stripped in AIAgent.__init__) against the raw
   _resolve_gateway_model() config string. For vendor-prefixed config on
   native providers (e.g. 'deepseek/deepseek-v4-pro' vs 'deepseek-v4-pro')
   this was always unequal, so the agent was evicted after every
   successful run. Normalize _cfg_model the same way (skip aggregators).

2. Discord triggering message_id leaked into the cached system prompt via
   build_session_context_prompt()'s Discord IDs block. message_id changes
   every turn, so the agent-cache signature (computed from the ephemeral
   prompt) changed every Discord turn -> rebuild every message. The id is
   now injected per-turn into the user message (where per-turn content
   belongs and does not touch the cache signature); the cached IDs block
   carries a static pointer to it, preserving reply/react/pin via the
   discord tools.

Adapted from NousResearch#28846. Bug #1 fix is the contributor's; bug #2 reworked to
be non-destructive (keeps the triggering-id capability instead of deleting
it). Redundant auto-reset eviction (already on main via NousResearch#9893/NousResearch#48031) and
the wrong-premise reset_context_note plumbing from the original PR were
dropped.

Co-authored-by: Hermes Agent <[email protected]>
Methodician added a commit to Methodician/hermes-agent that referenced this pull request Jul 4, 2026
@Methodician @Rafa-Ross
fix(gateway): stop per-turn agent-cache eviction from model + message…
9b93885 
…_id signature churn

Two independent bugs evicted the cached gateway AIAgent on every turn,
preventing the prompt cache from ever warming:

1. Model normalization mismatch: the post-run fallback-eviction check
   compared _agent.model (stripped in AIAgent.__init__) against the raw
   _resolve_gateway_model() config string. For vendor-prefixed config on
   native providers (e.g. 'deepseek/deepseek-v4-pro' vs 'deepseek-v4-pro')
   this was always unequal, so the agent was evicted after every
   successful run. Normalize _cfg_model the same way (skip aggregators).

2. Discord triggering message_id leaked into the cached system prompt via
   build_session_context_prompt()'s Discord IDs block. message_id changes
   every turn, so the agent-cache signature (computed from the ephemeral
   prompt) changed every Discord turn -> rebuild every message. The id is
   now injected per-turn into the user message (where per-turn content
   belongs and does not touch the cache signature); the cached IDs block
   carries a static pointer to it, preserving reply/react/pin via the
   discord tools.

Adapted from NousResearch#28846. Bug #1 fix is the contributor's; bug NousResearch#2 reworked to
be non-destructive (keeps the triggering-id capability instead of deleting
it). Redundant auto-reset eviction (already on main via NousResearch#9893/NousResearch#48031) and
the wrong-premise reset_context_note plumbing from the original PR were
dropped.

Co-authored-by: Hermes Agent <[email protected]>
caozuohua pushed a commit to caozuohua/hermes-agent that referenced this pull request Jul 5, 2026
@fayenix @Rafa-Ross @caozuohua
fix(gateway): stop per-turn agent-cache eviction from model + message…
285bf39 
…_id signature churn

Two independent bugs evicted the cached gateway AIAgent on every turn,
preventing the prompt cache from ever warming:

1. Model normalization mismatch: the post-run fallback-eviction check
   compared _agent.model (stripped in AIAgent.__init__) against the raw
   _resolve_gateway_model() config string. For vendor-prefixed config on
   native providers (e.g. 'deepseek/deepseek-v4-pro' vs 'deepseek-v4-pro')
   this was always unequal, so the agent was evicted after every
   successful run. Normalize _cfg_model the same way (skip aggregators).

2. Discord triggering message_id leaked into the cached system prompt via
   build_session_context_prompt()'s Discord IDs block. message_id changes
   every turn, so the agent-cache signature (computed from the ephemeral
   prompt) changed every Discord turn -> rebuild every message. The id is
   now injected per-turn into the user message (where per-turn content
   belongs and does not touch the cache signature); the cached IDs block
   carries a static pointer to it, preserving reply/react/pin via the
   discord tools.

Adapted from NousResearch#28846. Bug NousResearch#1 fix is the contributor's; bug NousResearch#2 reworked to
be non-destructive (keeps the triggering-id capability instead of deleting
it). Redundant auto-reset eviction (already on main via NousResearch#9893/NousResearch#48031) and
the wrong-premise reset_context_note plumbing from the original PR were
dropped.

Co-authored-by: Hermes Agent <[email protected]>
(cherry picked from commit d6c53dc)
Jasper6439 pushed a commit to Jasper6439/hermes-agent that referenced this pull request Jul 5, 2026
@fayenix @Rafa-Ross Jasper6439
fix(gateway): stop per-turn agent-cache eviction from model + message…
b3ca6fb 
…_id signature churn

Two independent bugs evicted the cached gateway AIAgent on every turn,
preventing the prompt cache from ever warming:

1. Model normalization mismatch: the post-run fallback-eviction check
   compared _agent.model (stripped in AIAgent.__init__) against the raw
   _resolve_gateway_model() config string. For vendor-prefixed config on
   native providers (e.g. 'deepseek/deepseek-v4-pro' vs 'deepseek-v4-pro')
   this was always unequal, so the agent was evicted after every
   successful run. Normalize _cfg_model the same way (skip aggregators).

2. Discord triggering message_id leaked into the cached system prompt via
   build_session_context_prompt()'s Discord IDs block. message_id changes
   every turn, so the agent-cache signature (computed from the ephemeral
   prompt) changed every Discord turn -> rebuild every message. The id is
   now injected per-turn into the user message (where per-turn content
   belongs and does not touch the cache signature); the cached IDs block
   carries a static pointer to it, preserving reply/react/pin via the
   discord tools.

Adapted from NousResearch#28846. Bug #1 fix is the contributor's; bug NousResearch#2 reworked to
be non-destructive (keeps the triggering-id capability instead of deleting
it). Redundant auto-reset eviction (already on main via NousResearch#9893/NousResearch#48031) and
the wrong-premise reset_context_note plumbing from the original PR were
dropped.

Co-authored-by: Hermes Agent <[email protected]>
habarmc1223-sudo pushed a commit to habarmc1223-sudo/hermes-agent-fluxmem that referenced this pull request Jul 8, 2026
@fayenix @Rafa-Ross @teknium1
fix(gateway): stop per-turn agent-cache eviction from model + message…
429ff38 
…_id signature churn

Two independent bugs evicted the cached gateway AIAgent on every turn,
preventing the prompt cache from ever warming:

1. Model normalization mismatch: the post-run fallback-eviction check
   compared _agent.model (stripped in AIAgent.__init__) against the raw
   _resolve_gateway_model() config string. For vendor-prefixed config on
   native providers (e.g. 'deepseek/deepseek-v4-pro' vs 'deepseek-v4-pro')
   this was always unequal, so the agent was evicted after every
   successful run. Normalize _cfg_model the same way (skip aggregators).

2. Discord triggering message_id leaked into the cached system prompt via
   build_session_context_prompt()'s Discord IDs block. message_id changes
   every turn, so the agent-cache signature (computed from the ephemeral
   prompt) changed every Discord turn -> rebuild every message. The id is
   now injected per-turn into the user message (where per-turn content
   belongs and does not touch the cache signature); the cached IDs block
   carries a static pointer to it, preserving reply/react/pin via the
   discord tools.

Adapted from NousResearch#28846. Bug #1 fix is the contributor's; bug NousResearch#2 reworked to
be non-destructive (keeps the triggering-id capability instead of deleting
it). Redundant auto-reset eviction (already on main via NousResearch#9893/NousResearch#48031) and
the wrong-premise reset_context_note plumbing from the original PR were
dropped.

Co-authored-by: Hermes Agent <[email protected]>
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
@fayenix @Rafa-Ross @teknium1
fix(gateway): stop per-turn agent-cache eviction from model + message…
91d6689 
…_id signature churn

Two independent bugs evicted the cached gateway AIAgent on every turn,
preventing the prompt cache from ever warming:

1. Model normalization mismatch: the post-run fallback-eviction check
   compared _agent.model (stripped in AIAgent.__init__) against the raw
   _resolve_gateway_model() config string. For vendor-prefixed config on
   native providers (e.g. 'deepseek/deepseek-v4-pro' vs 'deepseek-v4-pro')
   this was always unequal, so the agent was evicted after every
   successful run. Normalize _cfg_model the same way (skip aggregators).

2. Discord triggering message_id leaked into the cached system prompt via
   build_session_context_prompt()'s Discord IDs block. message_id changes
   every turn, so the agent-cache signature (computed from the ephemeral
   prompt) changed every Discord turn -> rebuild every message. The id is
   now injected per-turn into the user message (where per-turn content
   belongs and does not touch the cache signature); the cached IDs block
   carries a static pointer to it, preserving reply/react/pin via the
   discord tools.

Adapted from NousResearch#28846. Bug NousResearch#1 fix is the contributor's; bug NousResearch#2 reworked to
be non-destructive (keeps the triggering-id capability instead of deleting
it). Redundant auto-reset eviction (already on main via NousResearch#9893/NousResearch#48031) and
the wrong-premise reset_context_note plumbing from the original PR were
dropped.

Co-authored-by: Hermes Agent <[email protected]>
kulikman pushed a commit to kulikman/hermes-agent that referenced this pull request Jul 16, 2026
@fayenix @Rafa-Ross @teknium1
fix(gateway): stop per-turn agent-cache eviction from model + message…
6200ffe 
…_id signature churn

Two independent bugs evicted the cached gateway AIAgent on every turn,
preventing the prompt cache from ever warming:

1. Model normalization mismatch: the post-run fallback-eviction check
   compared _agent.model (stripped in AIAgent.__init__) against the raw
   _resolve_gateway_model() config string. For vendor-prefixed config on
   native providers (e.g. 'deepseek/deepseek-v4-pro' vs 'deepseek-v4-pro')
   this was always unequal, so the agent was evicted after every
   successful run. Normalize _cfg_model the same way (skip aggregators).

2. Discord triggering message_id leaked into the cached system prompt via
   build_session_context_prompt()'s Discord IDs block. message_id changes
   every turn, so the agent-cache signature (computed from the ephemeral
   prompt) changed every Discord turn -> rebuild every message. The id is
   now injected per-turn into the user message (where per-turn content
   belongs and does not touch the cache signature); the cached IDs block
   carries a static pointer to it, preserving reply/react/pin via the
   discord tools.

Adapted from NousResearch#28846. Bug #1 fix is the contributor's; bug #2 reworked to
be non-destructive (keeps the triggering-id capability instead of deleting
it). Redundant auto-reset eviction (already on main via NousResearch#9893/NousResearch#48031) and
the wrong-premise reset_context_note plumbing from the original PR were
dropped.

Co-authored-by: Hermes Agent <[email protected]>
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
@fayenix @Rafa-Ross @teknium1
fix(gateway): stop per-turn agent-cache eviction from model + message…
0e72644 
…_id signature churn

Two independent bugs evicted the cached gateway AIAgent on every turn,
preventing the prompt cache from ever warming:

1. Model normalization mismatch: the post-run fallback-eviction check
   compared _agent.model (stripped in AIAgent.__init__) against the raw
   _resolve_gateway_model() config string. For vendor-prefixed config on
   native providers (e.g. 'deepseek/deepseek-v4-pro' vs 'deepseek-v4-pro')
   this was always unequal, so the agent was evicted after every
   successful run. Normalize _cfg_model the same way (skip aggregators).

2. Discord triggering message_id leaked into the cached system prompt via
   build_session_context_prompt()'s Discord IDs block. message_id changes
   every turn, so the agent-cache signature (computed from the ephemeral
   prompt) changed every Discord turn -> rebuild every message. The id is
   now injected per-turn into the user message (where per-turn content
   belongs and does not touch the cache signature); the cached IDs block
   carries a static pointer to it, preserving reply/react/pin via the
   discord tools.

Adapted from NousResearch#28846. Bug NousResearch#1 fix is the contributor's; bug NousResearch#2 reworked to
be non-destructive (keeps the triggering-id capability instead of deleting
it). Redundant auto-reset eviction (already on main via NousResearch#9893/NousResearch#48031) and
the wrong-premise reset_context_note plumbing from the original PR were
dropped.

Co-authored-by: Hermes Agent <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@teknium1 teknium1 teknium1 left review comments
+1 more reviewer
@chiefmojo chiefmojo chiefmojo left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

comp/gateway Gateway runner, session dispatch, delivery P1 High — major feature broken, no workaround sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@fayenix @teknium1 @chiefmojo @alt-glitch