feat(session): session-aware cache boundary manager#56
Merged
Conversation
This was referenced May 2, 2026
Add CacheBoundaryManager to pkg/session. After each Push, the manager promotes stable entries and evaluates the optimal cache_control placement for the next request. Schema changes (session_entries): - inserted_at_push: push count when the entry was first added - stable_since_turn: push count when the entry became stable (0 = not yet) - content_hash: FNV hash for change detection Schema changes (sessions): - push_count: incremented on every Push call - cache_boundary_tokens: last recorded boundary position New types: - CacheBoundaryConfig: Enabled, MinStableTurns (default 2), MinPrefixTokens (default 1024), MaxMarkers (default 4) - CacheBoundaryResult: Markers, TotalStableTokens, Advanced, Retreated - CacheBoundaryMarker: EntryID, TokensUpToHere, StableSinceTurn - BoundaryStats: PushCount, BoundaryTokens, StableEntryCount Behaviour: - RecordPush increments push_count and promotes entries that have survived >= MinStableTurns pushes without modification - Evaluate walks entries in sequence order, finds stable entries whose cumulative token count meets MinPrefixTokens, selects up to MaxMarkers by highest token count, detects advance/retreat vs previous boundary - PushResult now includes CacheBoundary *CacheBoundaryResult - Session now exposes CacheBoundaryTokens and PushCount fields - Config.CacheBoundary wires DefaultCacheBoundaryConfig into the store Co-authored-by: Ona <[email protected]>
Co-authored-by: Ona <[email protected]>
Siddhant-K-code
force-pushed
the
feat/51-session-cache-boundary-manager
branch
from
2a00de4 to
bd409eb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #51
What
Adds
CacheBoundaryManagertopkg/session. After eachPush, the manager promotes stable entries and evaluates the optimalcache_controlplacement for the next request, returning the result inPushResult.CacheBoundary.Changes
pkg/session/cache_boundary.go(new)CacheBoundaryManager:RecordPush,Evaluate,InvalidateEntry,StatsCacheBoundaryConfig:Enabled,MinStableTurns(default 2),MinPrefixTokens(default 1024),MaxMarkers(default 4)CacheBoundaryResult:Markers []CacheBoundaryMarker,TotalStableTokens,Advanced,Retreatedpkg/session/sqlite.gosession_entriesgainsinserted_at_push,stable_since_turn,content_hash;sessionsgainspush_count,cache_boundary_tokensPushcallsRecordPushthenEvaluateafter budget enforcement; result carriesCacheBoundaryGetreturnsPushCountandCacheBoundaryTokenspkg/session/session.goSessionexposesCacheBoundaryTokens,PushCountPushResultcarriesCacheBoundary *CacheBoundaryResultConfigincludesCacheBoundary CacheBoundaryConfigWhy
pkg/sessionalready tracks which entries are present across turns and classifies them by stability (viapreserve_recentand importance scoring). This change uses that existing signal to placecache_controlmarkers automatically. Engineers stop needing to know what a cache boundary is — the session layer handles it.At 50 turns with 10K tokens of stable docs, the boundary manager reduces effective input cost from ~$15K/day to ~$2.5K/day at 10K sessions/day (83% reduction on the stable portion).