Skip to content

Chat sessions: persist, list, resume, and manage conversations#536

Merged
tobocop2 merged 30 commits into
mainfrom
feat/sessions
Jul 18, 2026
Merged

Chat sessions: persist, list, resume, and manage conversations#536
tobocop2 merged 30 commits into
mainfrom
feat/sessions

Conversation

@tobocop2

@tobocop2 tobocop2 commented Jul 15, 2026

Copy link
Copy Markdown
Owner

Problem

Chat conversations are ephemeral. The chat screen holds history in memory and clears it on quit, so there is no way to leave a conversation and come back to it. There is also no way to list past conversations or manage them, and none of that history is reachable over the HTTP server, MCP, or the CLI. And once a conversation grows past the model's context window, the oldest turns silently fall out with nothing standing in for them.

Solution

Persist conversations and let people list, resume, rename, and delete them, from the TUI, the HTTP server, MCP, and the CLI. For long conversations, add an opt-in compaction path that folds older turns into a running summary instead of dropping them, with a context indicator by the prompt. Sessions can be turned off entirely (on by default): when off, nothing is written to disk, the ctrl+o binding leaves the footer, and the Sessions view shows a notice instead of an empty screen.

Storage is an append-only JSONL log, one <id>.jsonl file per session under the data dir. Each line is a typed event: meta (first line), title (newest wins, so a rename appends rather than rewrites), message, summary, and origin (ownership transfers; newest wins). The log is only ever appended and fsynced, never rewritten, so the sole failure a crash can leave is a torn final line, which the reader skips. Files sidestep the LanceDB write lock entirely; sessions need no embeddings or vector search.

Conversations save automatically as you chat, titles are auto-derived from the first message and can be renamed, and the whole thing is keyboard-first: a left-docked drawer (ctrl+o or /sessions) as the quick switcher, a full-screen Sessions tab for management, lilbee sessions list / show / rename / delete for the shell, and the full surface over HTTP and MCP so a client (or the Obsidian plugin) can own a conversation end to end.

Compaction is off by default; the default path windows the tail and costs nothing. When enabled, older turns are batched and condensed by the chat model, bounded to a few calls, with a context chip by the prompt showing how full the window is. Accounting is honest: a batch the model fails to condense is reported as stranded, never counted as summarized. Batches that overflow the model's window are split in half and retried, so a token-estimate miss costs an extra small call rather than the turns themselves.

Ownership and concurrent access

Every session is stamped with the surface that created it (tui, mcp, http, or cli; files from before ownership read as tui, which is who wrote them), and the surfaces split into two domains. The TUI, HTTP server, and CLI are one human conversation space, the same way one account's web and mobile apps share chats: a session started in Obsidian lists and resumes in the terminal, and human surfaces append to each other's sessions freely, no claim dance between them. Agent (MCP) sessions are working state, the way agent transcripts and human chats are separate products: they never appear in the TUI or HTTP session lists, and the store refuses appends across the boundary in either direction, so an agent cannot splice its turns into a conversation a human has open -- the failure that motivated this, since session_get used to invite exactly that.

The privacy cut mirrors lilbee's agent-memory model. Over MCP, sessions_list returns only agent-created sessions, and reads or mutations of a human session answer not-found, indistinguishable from a missing id, so a connected agent cannot read your conversations or even probe which ones exist. The human side is the admin side: lilbee sessions list shows every origin (with an Origin column) so stray agent sessions can be found and cleaned up.

Crossing the boundary is explicit, by id, in both directions. An agent takes over a session whose id you hand it by passing claim=true on the append (transfer and append are atomic under the session lock; the session then moves to the agent's space and leaves your lists), and POST /api/sessions/{id}/claim brings one back -- until then appends across the boundary answer 409 with the claim named in the error. Transfers are append-only origin events, and the owning surface rides every meta payload. A read-only token can never claim.

Separately, every append serializes through a per-session file lock (the LanceDB pattern), so two processes writing one id queue instead of interleaving lines; the pinned test races three writers through 75 appends and all 75 land whole.

See it

Auto-save, the ctrl+o drawer, resuming a 240-message conversation, and live compaction folding 237 messages -- boundary rule, toast, and context chip on camera, recorded against a real Qwen3-8B on one RTX 5090 from this branch:

sessions reel

How this was validated

Everything below ran on real hardware (a RunPod A40, EU-SE-1) against real llama-server with real GGUF models, on the exact merged tree. The mocked suite runs in the same gate but is listed separately from the real-model phases: the mocks prove the plumbing, the pod proves the feature.

# Phase What it exercises How Result
1 Project gate lint, format, typecheck, 9,512 tests, 100% coverage make check on the pod pass (one env-dependent test fails on any host with a populated models dir; fails identically on main)
2 Store invariants no turn lost, append-only, torn tail, concurrent writers, prompt never exceeds budget seeded-random load test: 2,000 appends, 40 sessions, 8 writer threads, 4 ctx sizes pass
3 Compaction smoke first real-model summarize call 60 turns through Qwen3-4B found the feature broken (166s, 60/60 stranded, empty summary); root cause: thinking ate the token budget; fixed (think=False + reasoning recovery)
4 Per-model curve fold latency and accounting across sizes 60-turn fold, Qwen3 0.6B / 4B / 8B / 14B, warm GPU 1.3-2.5s, 60/60 condensed, 0 stranded on every model
5 Quality probe does the summary keep what matters diverse conversation, planted salient + trivial facts scored separately both kept, even by the 0.6B
6 CPU-only, laptop-class the hardware compaction was defaulted off for GPU hidden, pinned to 8 cores 0.6B folds 60 turns in 0.72s; 4B in 2.02s
7 Non-native providers reasoning models where think=False is stripped (Ollama / LM Studio / cloud) thinking forced on against the real model 60/60 condensed via reasoning recovery (was: all stranded)
8 Resume at scale replay + render of a large saved conversation 320-turn (640-message) session resumed in the real TUI replay 3ms, rendered, question answered on the resumed history
9 Model switch mid-conversation history and summary inherited across a swap 4B to 0.6B mid-chat on the resumed session survived, kept answering
10 Live in-TUI fold at a small window the full chain: turn, trigger, real summarize, split-on-overflow, summary persisted resumed 640-message session on a 2048-token target found two real bugs (batch sizing ignored estimate error, then the estimate's tail again); fixed with proportional headroom + split-on-overflow; final run folds 637 messages into a 577-char summary, 0 stranded
11 HTTP surface all 7 session routes end to end, token gating real litestar app pass: create 201, tokenless write 401, summary on the wire, sources preserved, client_can_own_a_conversation: true
12 Store scalability many sessions, one huge session 501 sessions + a 1,000-turn session cold list 31ms, warm 4.8ms, 1,000-turn replay 5.7ms
13 Ownership + write lock domain append rules, agent privacy scoping (list/read/rename/delete), claim flows both directions, concurrent writers 24 tests incl. 3 threads racing 75 appends into one file human surfaces share freely, the mcp boundary refuses both ways, foreign reads answer not-found; 75/75 appends land whole, none interleave

GPU latency figures are a lower bound (an A40 outruns most user hardware); the CPU-only 8-core row is the realistic worst case, measured for exactly that reason.

  • SessionStore core (append-only JSONL persistence)
  • Chat lifecycle: auto-save every turn, resume, new chat, sessions on/off toggle
  • Opt-in compaction with context indicator (real-model verified: native, non-native, small windows)
  • TUI surfaces: left drawer, full-screen Sessions tab, /sessions command
  • HTTP + MCP: full session surface (list / get / create / append / summary / rename / delete)
  • Two session domains (TUI/HTTP/CLI share; agent sessions are private and invisible to each other's lists) with explicit claim by id in both directions, and a per-session write lock
  • CLI: lilbee sessions list / show / rename / delete (JSON carries the summary)

tobocop2 added 10 commits July 14, 2026 21:57
Chat conversations were held in memory only and lost on quit. This adds the
persistence core: one <id>.jsonl file per session under the data dir, written
append-only as meta/title/message events. Title is carried as an event so a
rename appends a new line rather than rewriting the file, and the reader skips a
torn final line. Resolves its directory late-bound from cfg so it follows the
configured data dir. The TUI, HTTP, and MCP surfaces build on this in follow-up
commits.
Add list / get / rename / delete for sessions on both surfaces, backed by the
shared SessionStore on the services container. Over HTTP the two reads are
read-only-token accessible and the mutations are not; over MCP a get returns the
full transcript so an agent resumes a conversation by continuing it. The store
joins the services container with a default factory so every surface reaches it
the same way.
Chat conversations now persist as you talk. The first user turn opens a session
(auto-titled from the message) and every user and assistant turn appends to it;
resume replays a saved session back into the chat log and restores its model;
/clear drops the active session so the next turn starts a fresh one. The store
writes under the data dir, which the test harness now isolates like its siblings
so the suite stays hermetic.
A left-docked drawer (ctrl+o or /sessions) is the quick switcher: filter, resume,
rename, and delete without leaving chat, with the active conversation marked. A
full-screen Sessions tab in the nav strip is the management home, sharing the same
list. The drawer focuses the filter for type-to-switch; the tab focuses the list
so the nav keys still cycle. Ctrl+N now skips its completion binding when the chat
input is unfocused, so an overlay can claim it for new-chat.
lilbee sessions list / show / rename / delete manages saved conversations from
the shell, over the same append-only store as the TUI, HTTP, and MCP. Ids resolve
by unique prefix, delete confirms unless --yes, and every command supports --json.
Importing SessionStore at the services module top pulled the config/catalog
import chain and cycled back through get_services during CLI config load. Defer it
to a lazy default factory so the module import stays light.
Adding the Sessions tab and /sessions command shifts the nav-cycle order and the
slash catalog contents; update the bracket/nav-prev sequences and add /sessions to
the CHAT & SESSION group so the catalog lists every registered command.
A session records the chat model it used, and resume restored it. If that model
had since been deleted, restoring the ref hit the model boundary's rejection with
a scary error. Only switch when the model is still installed; otherwise keep the
current model and say the original conversation's model is unavailable.
… meta

Blast-radius follow-ups. Resume loaded the whole transcript into the prompt
history, so a long conversation could overflow a small model's context on the
first turn; window it like a live conversation while still rendering the full log.
Deleting the active session (e.g. from the drawer) then chatting hit the store's
not-found guard and crashed auto-save; reopen a fresh session instead, and let the
worker's assistant write skip a session that vanished mid-stream. Replace the three
copied SessionMeta serializers with dataclasses.asdict.
Trim the sessions package facade to the API its consumers actually import; the
event-tag enum and the dir-name/untitled constants stay internal to the store
module (tests import them from there). Lift the message reconstruction out of the
fold loop so it stops nesting three deep, and annotate the one isinstance guard on
ListView.highlighted_child.
@tobocop2 tobocop2 self-assigned this Jul 16, 2026
tobocop2 added 11 commits July 16, 2026 13:06
Long conversations eventually overflow the model's context window. The
existing behaviour silently drops the oldest turns. This adds an opt-in
alternative that folds older turns into a running summary, plus a context
chip by the prompt that shows how full the window is and whether a
summarize call is in flight.

Compaction is off by default: the lightweight path (windowing the tail)
stays the default and costs nothing. When enabled, older turns are batched
and condensed by the chat model, bounded to a few calls, with honest
accounting -- a batch the model fails to condense is reported as stranded,
never counted as summarized.

Includes the fix a real-model load test surfaced: the summarize call left
thinking enabled, so a reasoning model spent its whole token budget in a
<think> block that llama.cpp force-closed at the limit and strip_reasoning
then deleted in full, returning an empty summary and stranding every turn.
Passing think=False (as the entity extractor already does for its own
internal call) fixes it: verified on Qwen3 0.6B through 14B, GPU and
CPU-only, folding 60 turns in under a second on laptop-class hardware while
keeping the facts that matter. This path is native llama-server only; the
same call over Ollama, LM Studio, or a cloud API still needs the flag.
…ete the HTTP/MCP session surface

Two gaps a real-model load test exposed.

The think=False fix only reaches the native llama-server path; over Ollama, LM
Studio, or a cloud API a reasoning model can still spend its whole budget inside
a <think> block, leaving nothing after the strip. That reasoning is itself a
summary of the turns, so fall back to it rather than strand them. A
non-reasoning model never reaches the fallback: it emits no block, so the strip
returns its answer directly. Verified on a real model with thinking forced on:
60 turns condensed, none stranded, where it used to strand all 60 and return
empty.

And the HTTP and MCP surfaces exposed only list/get/rename/delete, so no client
could create a session, append a turn, or read back the compaction summary
(get dropped it). Add POST /api/sessions, POST /api/sessions/{id}/messages, PUT
/api/sessions/{id}/summary, and put summary on the get response; mirror all of
it in MCP (session_create, session_add_message, session_set_summary, summary on
session_get). A client can now own a conversation end to end, which is what lets
the Obsidian plugin match the TUI. Writes require a full token; the two GETs stay
read-only.
Some people would rather their chats not persist. Add a setting to turn
sessions off. It is on by default, so the behaviour is unchanged unless you
opt out.

When off, conversations are never written to disk, the ctrl+o Sessions
binding leaves the footer (via check_action, so the framework hides it
rather than showing it greyed), and opening the Sessions view -- from the
tab, ctrl+o, or /sessions -- shows a notice that sessions are turned off
instead of an empty screen. The tab stays visible on purpose, so the
feature is still discoverable and the notice has somewhere to explain how
to turn it back on.
Same hole the HTTP and MCP surfaces had: a script that resumes a
conversation from the CLI JSON rebuilds history without what compaction
folded away. The summary field makes the CLI transcript complete.
Found by a live end-to-end run, not by the suite: on a 2048-token window,
compaction sized a batch to the raw remaining room, the chars/4 estimate
under-counted terse text by ~35%, the server's chat template added more, and
the summarize call arrived as ~2666 real tokens. The server rejected it, every
batch failed the same way, and all 637 folded turns were stranded -- dropped
from history with nothing standing in for them, by the very call meant to
preserve them. Larger windows never hit this because the backlog happened to
fit; the failure needs foldable content bigger than the window, which is
exactly the small-model-large-history case compaction exists for.

Batches now claim a documented fraction of the remaining window instead of all
of it, covering the worst-case estimate gap plus the template. The regression
test pins the stronger invariant: estimate times worst-case ratio plus the
reply cap must fit, not merely estimate < window.
The batch headroom fix lowered the odds of a summarize call overflowing the
window; a live run promptly found content that tokenizes 1.8x denser than the
chars/4 estimate and overflowed anyway, this time counting the reply
reservation lilbee's own preflight adds. Chasing the ratio with a smaller
fraction is a losing game: an estimate always has a tail, and the tail's cost
here is stranded turns, already dropped from history by the time the call runs.

Make overflow non-fatal instead: on CONTEXT_OVERFLOW, halve the batch and fold
each half on its own, merging the notes. Halving is feedback where the estimate
is a guess, and it terminates -- depth is log2 of the batch. A single message
too big for the window cannot split and still fails safe with the warning.
The error kind is typed at the provider boundary, so no message matching.
The taskbar's 'almost ready' was renamed to say what is happening; the
thinking header had its own copy of the same phrase for the same phase.
Same rename, same reason: name the phase, don't promise a finish time.
@tobocop2 tobocop2 mentioned this pull request Jul 16, 2026
tobocop2 added 7 commits July 16, 2026 18:23
…h the window

Three user-review findings and a comment pass.

A turn persisted by an API client carries the answer text (which contains its
in-text Sources list) and the structured sources array side by side; rendering
both stacked two source lists on one bubble after resume. The pill row now
yields when the content already carries its list, since the answer's [1]
citations anchor to the in-text one. The "\n\nSources:\n" marker is hoisted to
a single constant shared by the builder and both checkers.

The final rolling summary was capped at 320 tokens regardless of window, which
flattens a long conversation to a few hundred words even on a 32k model. The
ceiling rises to 1024 with ctx/8 still governing below it, so small windows
stay as tight as before, and the prompt's word budget is now derived from the
cap instead of hardcoding 150 words.

The chat_compaction comment carried pre-measurement latency estimates that the
hardware runs falsified; it now states the measured numbers. Verbose
session-diary comments across the diff are trimmed to the constraints they
exist to state. session_add_message takes MessageRole at the boundary, with
tool docstrings trimmed to keep the MCP schema under its size budget.
…reel

The feature had no written surface. The usage guide gains a Sessions section
(auto-save, the drawer / tab / shell routes in, resume with the model-restore
behavior, the plain-JSONL storage story, the off toggle) with a subsection on
what happens when a conversation outgrows the model -- the default windowing
with the context gauge, and opt-in compaction with its measured cost. The CLI
reference gains the sessions command family, the HTTP section now names the
session routes and their token gating, and the MCP skill table gains the seven
session tools with a warning that ignoring the summary on resume silently
loses what compaction condensed.

The site gets a sessions tab in the homepage reel player, a card under "what
it does", and a tutorial section between the chat demos and add-files; the
long-form tutorial doc gains the matching section. All reference the reel
assets at their stable gh-pages names.
A restored transcript could alternate between two citation styles: turns
saved by the TUI carry the numbered clickable Sources list inside their
answer text, while turns written over HTTP/MCP (or seeded) carry only the
structured sources array, which rendered as a separate grey pill row. Same
conversation, two visual languages.

Sources now render exactly one way. A turn arriving with structured sources
but no in-text list gets the identical clickable list synthesized into its
content (source_markdown_link is the shared, now-public helper), and
finish() folds live-passed sources the same way. The pill row, its widget,
its builder, and its CSS are deleted.
Cut the flourishes and rhetorical asides; match the surrounding sections'
density. No content changes, no anchors moved.
…s chat

Every surface had full append access to every session: an MCP agent could
resume and continue a conversation the user had open in the TUI (session_get
even suggested it), interleaving two conversations into one transcript and
feeding the title and summary pipelines garbage. Concurrent surfaces are the
normal case now, so collisions were a matter of time.

Sessions now belong to the surface that created them. The origin (tui, mcp,
http, cli) is stamped in the meta event; files from before ownership read as
tui, which is who wrote them. The store itself refuses a guarded append from
any other surface, so the policy cannot be forgotten by a caller. Handover is
explicit and asymmetric: resuming an agent's session in the TUI is the human
transferring it, and it just works; an agent claims a TUI session only by
passing claim=true on the append (transfer and append are atomic under the
session lock), and an HTTP client only via POST /api/sessions/{id}/claim,
until which appends answer 409 with the claim named in the error. Transfers
are append-only origin events, newest wins, like renames.

Separately, every append now serializes through a per-session file lock, so
two processes writing one id queue instead of interleaving lines; three
threads racing 75 appends land all 75 whole. The origin rides the HTTP meta
payload so clients can see who owns what.
tobocop2 added 2 commits July 17, 2026 00:39
TUI, HTTP, and CLI are one conversation space: human surfaces list only
human sessions and append to each other's freely, so a chat started in
Obsidian resumes in the terminal with no claim dance. Agent (MCP)
sessions are working state on the other side of the boundary: they never
appear in the TUI drawer or HTTP list, and over MCP the session tools
scope to agent-owned sessions, with foreign ids answering not-found so a
connected agent cannot read human conversations or probe which exist.

Crossing the boundary stays explicit and by id: claim=true on an MCP
append takes over a session the user handed the agent (and moves it out
of their lists), POST /claim brings one back. The TUI resume auto-claim
is gone since human-space resumes no longer need a transfer. lilbee
sessions list remains the admin view over every origin and grows an
Origin column.
@tobocop2
tobocop2 marked this pull request as ready for review July 18, 2026 06:05
@tobocop2
tobocop2 merged commit 01974e4 into main Jul 18, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant