fix(gateway): stop per-turn agent-cache eviction from model + message_id signature churn#55595
Merged
Conversation
alt-glitch
added
type/bug
…_id signature churn Two independent bugs evicted the cached gateway AIAgent on every turn, preventing the prompt cache from ever warming: 1. Model normalization mismatch: the post-run fallback-eviction check compared _agent.model (stripped in AIAgent.__init__) against the raw _resolve_gateway_model() config string. For vendor-prefixed config on native providers (e.g. 'deepseek/deepseek-v4-pro' vs 'deepseek-v4-pro') this was always unequal, so the agent was evicted after every successful run. Normalize _cfg_model the same way (skip aggregators). 2. Discord triggering message_id leaked into the cached system prompt via build_session_context_prompt()'s Discord IDs block. message_id changes every turn, so the agent-cache signature (computed from the ephemeral prompt) changed every Discord turn -> rebuild every message. The id is now injected per-turn into the user message (where per-turn content belongs and does not touch the cache signature); the cached IDs block carries a static pointer to it, preserving reply/react/pin via the discord tools. Adapted from #28846. Bug #1 fix is the contributor's; bug #2 reworked to be non-destructive (keeps the triggering-id capability instead of deleting it). Redundant auto-reset eviction (already on main via #9893/#48031) and the wrong-premise reset_context_note plumbing from the original PR were dropped. Co-authored-by: Hermes Agent <[email protected]>
teknium1
force-pushed
the
hermes/hermes-8cc6a6a2
branch
from
June 30, 2026 10:53
b67ecf6 to
61f5d97
Compare
tonydwb
reviewed
Jun 30, 2026
tonydwb
left a comment
tonydwb
left a comment
There was a problem hiding this comment.
Code Review Summary
Verdict: Approved
Two well-targeted fixes in one PR: (1) move Discord message_id from cached system prompt to per-turn user content to prevent agent-cache busting, and (2) normalize _cfg_model in gateway fallback-eviction so vendor-prefixed config matches stripped agent.model. Both fixes preserve prompt caching — the core invariant.
Looks Good
- Correctly identifies the prompt-cache-busting issue with volatile message_id
- The static pointer in system prompt + volatile id in user message is the right pattern
- Model normalization fix prevents unnecessary agent eviction on every turn
- Good test for prompt stability across message_ids
- Release.py author attribution included
Reviewed by Hermes Agent
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stops the gateway from evicting the cached
AIAgenton every turn, so the prompt cache can actually warm. Two independent signature-churn bugs caused a rebuild every message.Changes
gateway/run.py(fallback-eviction): normalize_cfg_modelthe same wayAIAgent.__init__does before comparing to_agent.model, so a vendor-prefixed config value (deepseek/deepseek-v4-pro) matches the stripped agent model (deepseek-v4-pro) on native providers. Aggregators (openrouter, etc.) keep the vendor/model slug and are left untouched. (contributor's fix)gateway/session.py+gateway/run.py: the Discord triggeringmessage_idno longer goes into the cached system prompt (it changes every turn → busts the agent-cache signature). The volatile id is injected per-turn into the user message; the cached IDs block carries a static pointer so reply/react/pin via the discord tools still works.tests/gateway/test_session.py: regression test asserting the cached prompt is byte-stable across changingmessage_id.scripts/release.py: AUTHOR_MAP entry for @fayenix.Root cause
agent.model != _cfg_modelalways true for vendor-prefixed native config → evict every successful turnmessage_idin cached promptValidation
build_session_context_prompt()is now identical across message_ids 1001/2002/3003; raw id absent from the cached prompt; static pointer present.normalize_model_for_provider('deepseek/deepseek-chat','deepseek')→deepseek-chat(matches); openrouter slug untouched.scripts/run_tests.sh tests/gateway/test_session.py tests/gateway/test_agent_cache.py→ 165 passed.Salvage notes
Adapted from #28846 (@fayenix). Bug 1's fix is the contributor's. Bug 2 was reworked to be non-destructive — the original PR deleted the triggering-message line outright (killing reply/react/pin); this keeps the capability and just moves the volatile id off the cached prefix. The original PR's auto-reset eviction block is already on main (#9893/#48031) and the
reset_context_note"scoping bug" did not reproduce on current main, so both were dropped.Infographic