Skip to content

Latest commit

 

History

History
138 lines (93 loc) · 16 KB

keep-warm.md

File metadata and controls

138 lines (93 loc) · 16 KB

Keep-warm

Faber caches the conversation with Anthropic's 5-minute lifetime and the stable system blocks with the 1-hour one (see prompt caching). A session waiting on you, a long build, or a tool call that runs past the conversation entry's lifetime would otherwise resume on a full cache rewrite. With keep-warm enabled, a per-session daemon re-sends the session's exact request a few seconds before that entry expires, which refreshes it. The ping persists no message and runs no tools, but its tokens and cost are counted like any other request.

Provider scope. Pings are sent for any model, but the timing follows the lifetime of Faber's conversation marker, Anthropic's 5 minutes, which a request refreshes. That lifetime is fixed today; Why says why and how to change it. With a provider whose cache works differently, or has no cache, a ping costs a request and buys nothing; leave ping.enabled off there.

How it works

The anchor and the deadline

Every dispatched request (a turn step or a ping) records its dispatch time on the session as cache.lastRequestAt. The daemon derives everything from that anchor:

ping at = lastRequestAt + 5 min - ping.before_expiry

SessionPing runs one loop per armed session. On each pass, evaluate returns one of three decisions:

Decision When
ping The anchor is inside the 5-minute window. The loop sleeps until the deadline, re-checks, and pings if still due.
idle No request has been dispatched yet, the window has already lapsed, too many consecutive pings missed, or the session could not be read. The loop sleeps 10 seconds and re-checks.
stop The session is a subagent. The loop exits.

The anchor decides, not whether a turn is running. A turn that keeps dispatching requests keeps pushing the deadline out, so the daemon stays quiet. A turn parked inside a long tool call dispatches nothing, the deadline arrives, and the ping fires. That is the case it exists for. A ping that overlaps a turn's own request costs one duplicate cache read.

The ping request

ping rebuilds the request the next turn would send: the same history (after compaction filtering and plugin transforms), the same agent resolved from the session's pin, the same tools and the same three system blocks. It appends one user message, ., and calls LLM.stream. The cache markers therefore land where a real turn would put them (see prompt caching).

The ping reads the stream until the first finish-step, then records usage with Session.updateTotals, priced by the same function as a turn (see usage and cost). It writes no message. Only the turn loop persists message state, so a ping that overlaps a turn cannot write a stale copy of history over it.

Bounds

Guard Value Behaviour
PING_TIMEOUT 60 s Aborts a single stalled ping without ending the loop.
MAX_MISSES 2 A ping that never sees start-step (the server streamed nothing) is a miss. After two in a row the daemon stays idle until the next turn re-arms it.
MIN_TICK 1 s Floor under a due ping's sleep, so a deadline that never moves cannot spin the loop.
IDLE_TICK 10 s Re-check interval while there is nothing to ping.

Seeing start-step counts as success: the server accepted the request and read the cached prefix, even if the body later errors.

When pings stop paying

The daemon has no ping limit: it keeps a session warm until you stop, archive or delete it, or two pings in a row fail. Each ping reads the whole cached prefix, while letting the conversation entry expire costs one rewrite of the conversation when you return. Taking the whole context as conversation, a run of pings pays off only while it is shorter than (write price − read price) ÷ read price pings, at one ping every 4 minutes 50 seconds (5 minutes minus ping.before_expiry):

Cache read price 5-minute write Pings to break even Idle time
0.1x 1.25x 11.5 about 56 minutes
0.05x 1.25x 24 about 1 hour 56 minutes
0.025x 1.25x 49 about 3 hours 57 minutes

Context size scales both sides equally, so it does not move the break-even; how much of the context is conversation does. Each ping also reads the system blocks. While their 1-hour entry is alive anyway (within the hour, or longer if another session keeps it warm), the larger their share, the sooner pings stop paying; past the hour, a lone session's system entry expires too, so the two longer break-evens are lower bounds. Before leaving a session for longer than that, Stop it (which disarms pings) and accept one rewrite on return, or leave ping.enabled off.

Intent: who arms and who disarms

Each session persists a keepWarm flag that mirrors whether a daemon is armed. Only explicit actions change it:

Action Effect
Sending a prompt, or a result delivered into the session Arms at turn start (SessionPing.start). A busy turn keeps sliding the deadline, so arming early costs no ping.
Opening a session from the sidebar or overview (POST /session/:sessionID/arm) Arms, but only while the cache is still inside its window.
Stop, archive, delete Disarms and clears keepWarm (Session.stop, SessionPing.stop).
Interrupting a turn (Esc) Leaves keepWarm set.
A plain fetch: reload, reconnect, a second client attaching Reconciles to the stored flag. It re-arms only a session that already has keepWarm and a live cache, and never resurrects a stopped one.

Subagent and headless sessions are never kept warm (Session.attended).

After a server restart, recovery re-arms every session that carries keepWarm and whose anchor is still inside the 5-minute window (Recovery, arm). A machine that was down longer restores nothing.

In the UI

The session status line and the session overview show a countdown ring to the next ping. A session with no scheduled ping (stopped, disabled, or its window lapsed) shows -- and an empty ring. The server publishes the deadline (pingAt) and the clients only render it, so every surface agrees.

Measured

On my own install, with keep-warm on, 2026-07-10 to 2026-10-10: after a pause of 5 to 60 minutes, 96.4% of resumed turns in top-level sessions (1,012 of 1,050) found the conversation still cached, and every month was between 95.1% and 97.8%. The conversation entry expires 5 minutes after its last use; in between, only a ping, or a subagent or fork that copied the conversation, can refresh it. Warm means the turn read from cache and wrote less than 10% of its input.

Month Warm resumes after a 5 to 60 minute pause
July 117 of 121 (96.7%)
August 227 of 236 (96.2%)
September 352 of 370 (95.1%)
October 316 of 323 (97.8%)

What pings cost against the rewrites they prevent has not been measured; When pings stop paying gives the arithmetic.

Configuration

Key Default Effect
ping.enabled false Turns the daemon on. When off, turns still re-anchor the window but no daemon is armed, so no automatic ping fires and the countdown shows --. Cache-safe revert still sends its one-shot probe.
ping.before_expiry 10 Seconds before expiry to send the ping.
{
  "ping": { "enabled": true, "before_expiry": 10 }
}

HTTP: POST /session/:sessionID/arm arms a session that is still warm; POST /session/:sessionID/ping sends a one-shot ping, optionally with a cache probe (see cache-safe revert).

Why

The daemon replaced manual . keep-alive messages that had to be sent and then cleaned out of history; the daemon persists nothing.

Why the conversation gets 5 minutes, not 1 hour. Every turn writes new conversation tokens to the cache: at 1.25x the base input price with a 5-minute lifetime, 2x with a 1-hour one. The 5-minute lifetime is a bet that, in a session that writes often and idles rarely, the cheaper write plus an occasional ping (a cache read, 0.1x) costs less than paying 2x on every write; it has not been measured against a 1-hour conversation lifetime. The system blocks change rarely, so they get the 1-hour lifetime.

Conversation lifetime Every token written A pause of 5 to 60 minutes
5 minutes, with pings 1.25x a cache read of the whole context every 4 minutes 50 seconds
1 hour, no pings 2x nothing

The 1-hour premium (0.75x) is paid once on every conversation token written, roughly the conversation's final size (more if it was compacted, since each stretch before a compaction was written too); a ping re-reads the whole context at the read price. So 5 minutes with pings is cheaper while a session's pings stay under about 0.75 ÷ read price in total: 7.5 at a 0.1x read and 15 at 0.05x, one per 4 minutes 50 seconds of pause (about 36 and 72 minutes), with each pause under an hour. Those counts are for pauses at the end of the session, at full context, with no system prompt. Pauses spread evenly through a growing conversation read about half as much each, which roughly doubles the counts, and earlier pauses stretch them further; pings also read the system prompt, which scales the counts by the conversation's share of the context (a 30,000-token system prompt beside a 200,000-token conversation takes 7.5 down to 6.5). A pause over an hour costs the 1-hour side a full rewrite at 2x, which favours pings. Two things the counts leave out: each ping's few output tokens, at the output price, and pauses that resume cold anyway (3.6% of them in Measured), which cost the 5-minute side a rewrite. A single break is a different comparison, against letting the cache go cold (When pings stop paying).

Claude Code's default depends on billing. Its documentation says it requests a 1-hour lifetime for the main conversation on a Claude subscription within plan usage, and 5 minutes with usage credits, an API key or a cloud provider, unless you choose a lifetime yourself with the promptCacheTtl setting or CLAUDE_CODE_PROMPT_CACHE_TTL (which TTL each request gets). It describes the same trade-off as the table: the longer lifetime "helps when you leave a session idle and come back to it" and "costs more on short bursts of work that never idle past five minutes".

If your sessions idle for long stretches, change the lifetime in three places together: CACHE_TTL in session/ping.ts (ping timing), the conversation markers' ttl in applyCaching (provider/transform.ts), and CACHE_TTL in the app's utils/cache-countdown.ts (the countdown ring). Making them one config key would be a small patch. ping.enabled is off by default, and leaving it off is right for any provider whose cache outlives your pauses.

The remaining design follows from bugs recorded in commit messages:

  • Pinging through a busy turn. The daemon once stood down while a turn was running. A turn stuck in a long tool call moves no anchor, so the window lapsed while the daemon slept, and "a wedged tool call cost the whole cache rather than one ping".
  • The 60-second deadline. One ping stalled for 16 minutes after a network drop. Healthy pings took 1.7 to 39 seconds, so 60 seconds bounds a stall without clipping a live ping, and stays far below the 5-minute TTL.
  • The sleep floor. A ping that returned before dispatching left the deadline already due, so the loop rebuilt the whole session history back to back: "18,420 file reads and 235MB of transient copies per pass", at 100% CPU.
  • Only the turn persists. A ping read the history, a turn finished in the meantime, and the ping wrote its stale copy back, un-finishing the turn. The next request ended in two assistant blocks and the API rejected it. This affected "one damaged record in 23,542 on one machine and one in 31,222 on another".
  • Persisted intent. Opening a session used to re-arm it, so a second client's reload revived a session another client had just stopped. Arming now follows explicit actions, and passive attaches reconcile to the stored flag.
  • Restore gated on cache liveness. After a reboot, "a machine down longer restores nothing, which is right when there is no warm cache left to defend."

Code

  • packages/opencode/src/session/ping.ts: SessionPing.start, stop, probe, warm, evaluate, pause, ping, classify, CACHE_TTL, PING_TIMEOUT, MAX_MISSES
  • packages/opencode/src/session/prompt.ts: the SessionPing.start call at turn start
  • packages/opencode/src/session/index.ts: Session.stop, Session.attended
  • packages/opencode/src/session/recovery.ts: arm (restart re-arm)
  • packages/opencode/src/server/routes/session.ts: /:sessionID/arm, /:sessionID/ping, attach reconcile in the session GET route
  • packages/app/src/utils/cache-countdown.ts: pingCountdown
  • packages/app/src/components/statusline.tsx, packages/app/src/components/dialog-overview.tsx: countdown rings