feat(model/gemini): add context cache lifecycle management for agent sessions - #1951

Closes #1937

What this adds

Explicit Gemini context caching, ported from adk-python's ContextCacheConfig, ContextCacheRequestProcessor and GeminiContextCacheManager. With runner.Config.ContextCache set, an agent's system instruction, tools and stable conversation prefix are stored once as a Gemini CachedContent. Later requests name the cache instead of resending them.

r, err := runner.New(runner.Config{
	AppName:        "app",
	Agent:          rootAgent,
	SessionService: sessionService,
	ContextCache:   &model.ContextCacheConfig{CacheIntervals: 10, TTL: 30 * time.Minute},
})

What changes for existing users

session/database users must add a column before upgrading, even with caching off.

• The new cache_metadata column on events is part of every insert and select. database.AutoMigrate adds it, as the package doc already requires on every startup.
• Applications that manage the schema themselves must add a nullable text/JSON cache_metadata column first. Otherwise inserts into events fail.
• This is the contract fix(session/database): persist input and output transcriptions #1662 set for the transcription columns. That was a bug fix, and this is a feature, so a maintainer should decide whether it needs the breaking-change label or a release note.

Everything else is inert unless ContextCache is set:

• The request processor returns at once, the Gemini model takes its existing path when CacheConfig is nil, and the new request and response fields are zero. TestGenerateContent_NoContextCache and TestRunner_ContextCacheOff pin this.
• On Vertex AI sessions, an event with CacheMetadata is written with raw_event (eventNeedsRawEvent). Events without it keep the legacy wire format, so nothing changes with caching off.
• An AfterModel callback or plugin that replaces a response keeps the model's CacheMetadata unless the replacement sets its own. A callback cannot clear it.
• REST and ADK Web clients see nothing new. adkrest maps event fields explicitly (server/adkrest/internal/models/event.go). Code that marshals a session.Event directly gets a cacheMetadata key when the field is set.

How it works

• model.ContextCacheConfig: CacheIntervals, TTL, MinTokens and CreateHTTPOptions. Zero fields take adk-python's defaults (10, 30m, 0). runner.New calls Validate and keeps its own copy of the config, with maps and slices nested in CreateHTTPOptions.ExtraBody copied too. Each request gets a further copy.
• model.CacheMetadata on model.LLMResponse, so the session stores it with each event. Active means a cache exists. Fingerprint-only records the prefix the next request has to share before a cache is created. It encodes as adk-python does: snake_case keys, and times as float epoch seconds (cache_metadata.py:60-98), so sessions shared with adk-python read each other's records.
• LLMRequest.CacheConfig, CacheMetadata, PreviousPromptTokenCount: the flow sets these and the model acts on them.
• Request processor (internal/llminternal/context_cache_processor.go), placed after contents as in adk-python's single flow (single_flow.py:67). It finds the agent's latest cache state and prompt size in the session, and counts a new invocation against an active cache (_cache.py:143-146).
• Gemini cache manager (model/gemini/context_cache.go), following gemini_context_cache_manager.py:
  • It fingerprints the model, backend scope, system instruction, canonicalized tools, tool config and leading contents (:310, :229).
  • It reuses a cache until it expires, passes CacheIntervals, or its prefix changes (:261).
  • It creates a cache only once two requests agree on the prefix, and never below MinTokens or the model's minimum: 2048 tokens for Gemini 2.5, 4096 for Gemini 3 (:41-42, :427-437).
  • It sends the request with cachedContent and without the system instruction, tools, tool config and cached contents (:649).
• Cleanup (ContextCacheLedger in internal/llminternal/context_cache_lifecycle.go): model calls record the caches they created or stopped using. When the invocation ends, the runner reads the session as its service holds it and deletes every recorded cache that no agent's latest stored record names. If that read fails or takes over 30s, nothing is deleted and the caches expire with their TTL. It reads only when a cache was created or replaced.
• Persistence: a cache_metadata column in session/database, and the field in raw_event for Vertex AI. adk-python keeps it through its JSON event blobs (schemas/v1.py:222, vertex_ai_session_service.py:494).
• RunLive is untouched, as in adk-python.

Differences from adk-python

Raised on the issue (question 3):

1. Cache state is scoped to the agent's branch and isolation scope, not only its author (_cache.py:127). A parallel worker runs copies of one agent side by side, each on its own branch. Matched on author alone, each copy reads its sibling's prefix as changed and deletes a cache the sibling is still using. The scope is the boundary the contents processor already draws around the history the cache holds.

New in this PR, not discussed on the issue. Each is covered by its own tests and can be dropped:
2. Under a runner, a cache is deleted when the invocation ends, and only if no agent's latest stored record names it. The record is read from the session service, not from the invocation's own session object. adk-python deletes first (gemini_context_cache_manager.py:149). A failed call, or a callback that replaces the reply, then leaves the session naming a deleted cache, and a later request whose prefix matches again is sent with that name and rejected. The stored session is read because the database service adds an event to the invocation's object before its write and leaves it there if the write fails. Without a runner, the model deletes on its own: a replaced cache after a complete response reached the caller, and a created cache when none did.
3. A request that names a cache the provider no longer holds (404, or 403 naming the cached content) is sent once more without it, if nothing has been yielded yet. The missing cache is deleted, and the next request creates a new one. adk-python has no retry.
4. An AfterModel replacement keeps the model's CacheMetadata, for the same reason as 2.
5. A request that already names a cache in its GenerateContentConfig is sent as it is. adk-python overwrites it (:669).

Smaller:
6. The cached fields are left on the caller's request, and a modified copy is sent. adk-python edits it in place (:649-677). The flow and its telemetry read the request after the call.
7. Nothing is logged on a failed create or delete. ADK Go library code has no logger (AGENTS.md). As in adk-python, the request goes ahead uncached, and a cache that is not deleted expires with its TTL.
8. Zero config fields take the defaults, the Go zero-value convention. adk-python validates ge=1/gt=0 (context_cache_config.py:49-65).
9. CacheMetadata.ExpireSoon is left out. Nothing calls it, here or in adk-python.

Notes for reviewers

• Placement (question 1). The types are in model, because LLMRequest and LLMResponse reference them and every model implementation needs them. A separate package would only be imported back into model.
• Apigee. Apigee models delegate to gemini.NewModel and so send cache calls through the proxy. A proxy that does not route cachedContents makes each creation fail, and requests then go uncached.
• Prefix size estimate. It scales the earlier prompt's real token count by the prefix's share of the request's characters, as adk-python does (:532). adk-python counts tools with their null fields included, so the two can disagree about a prefix within a few percent of the model's minimum. The provider has the final word either way.
• New exported API: model.ContextCacheConfig (with Validate), model.CacheMetadata (with Active and the JSON methods), the three LLMRequest fields, LLMResponse.CacheMetadata, and runner.Config.ContextCache. All are additive.

Follow-ups

• Forward the caller's config into agent tool runs, with the TTL capped at 5 minutes as agent_tool.py:294-307 does. It is separable, and written.
• launcher.Config and adkrest.ServerConfig plumbing, as Compaction has, so adk web users can turn caching on.
• A telemetry attribute for the cache action.
• StaticInstruction on llmagent.
• Branch scoping and delete-after-success, proposed upstream to adk-python.

Testing plan

The Gemini API is faked with httptest rather than recorded, so no test makes a network call.

• model/cache_test.go: validation bounds; the JSON wire form both ways, against adk-python's exact output.
• internal/llminternal:
  • the request processor: invocation counting (earlier, current and no invocation; active vs fingerprint-only); author, branch and isolation-scope filtering; copies instead of shared state; a nil config overriding an outer one;
  • the ledger: keeps what a latest record names, reads the store only when something is tracked, deletes nothing when the read fails;
  • the config copy, including nested ExtraBody maps and slices, and a guard that fails if genai adds an HTTPOptions field the copy does not handle.
• model/gemini/context_cache_test.go:
  • every branch of reuse, replacement and refusal, asserting the exact order of API calls;
  • the missing-cache retry (404, 403 naming the cache, 403 for another reason, streaming, and no retry once output was yielded) and the delete of the missing cache with and without a ledger;
  • cleanup when the model call fails, the reply is empty, the consumer stops early, or the context is canceled;
  • the request and cache shapes, defaults and expiry fallback, exact model minimums, streaming, shared HTTP options left untouched, tracking headers, and a caller's own cache left alone.
• runner/context_cache_test.go:
  • three turns through runner.Run, llmagent and the Gemini model, under an llmagent, a sequentialagent and a parallelagent root; a callback rewriting the reply, caching off, validation and config copying;
  • the database service on SQLite where a second writer makes turn 3's write fail, with the live caches checked against the database after each turn;
  • a store that cannot be read or hangs at cleanup (nothing is deleted), and no extra session read on a turn that only reuses a cache;
  • a callback writing into a nested ExtraBody map.
• session/database, session/vertexai: CacheMetadata round trips, and rows with no cache state.
• Race check: all of the above pass with -race -shuffle=on on Go 1.26.10, and golangci-lint v2.3.1 reports nothing in the changed files.
• Revert check: 36 mutants, each applied alone to one guard, branch or constant: expiry and interval comparisons, both model minimums and their boundaries, author, branch and isolation scoping, delete-after-delivery, cleanup surviving cancellation, the JSON wire form, the persistence mappings, and both runner paths. Each ran against only its package's cache tests, and every one fails a test. A no-op control passes. This ran before the last review round, so the guards added in it (reading the stored session, the nested copy, the retry guard) are not covered. An earlier version of this description said 70 mutants, which was wrong.
• Live check: run against the Gemini API (gemini-3.5-flash) with a free-tier key, logging each call's method, path and status.
  • Turn 1 was sent whole and recorded a fingerprint.
  • Turn 2 found turn 1's state, matched the prefix, and made exactly one POST /v1beta/cachedContents, holding the first exchange.
  • The free tier refused it: 429 RESOURCE_EXHAUSTED, TotalCachedContentStorageTokensPerModelFreeTier ... limit=0, requested=8406. The requested count shows the API parsed the cache body.
  • ADK then sent the request uncached (200) and recorded the longer stable prefix, as designed.
  • Not verified live: a successful create and a request sent with cachedContent, because that needs a billing-enabled key. If a reviewer has one, go run ./examples/contextcache/ exercises that path in three turns.