Intel-only laptops get their GPU back

lilbee worked out which GPU vendors a machine had by looking at which Vulkan driver files were installed. Mesa ships every vendor's driver together, so an Intel-only laptop looked like it also had an AMD card. AMD won that tie, lilbee disabled the Intel driver, and the engine then reported zero GPUs. Every chat and every ingest failed.

Vendor detection now reads the hardware itself: PCI display controllers in sysfs on Linux, the display-adapter class in the registry on Windows. The driver left enabled always drives a card that is really in the box. Anyone already stuck in this state recovers on upgrade.

A stale GPU pin no longer takes the server down

A GPU placement saved in config.toml that no longer matched the machine was fatal, and the only way out was editing the file by hand. It now falls back to automatic placement, and the CLI, the TUI and the JSON surfaces all say when a saved pin is being ignored, so nothing is silently dropped. Applying a placement by hand still fails loudly, so a pin you just typed that does not fit is still an error.

A crash can no longer leave engines running

Three paths could leave a llama-swap, a llama-server or a GPU device probe alive after lilbee had gone. macOS had no lifetime binding at all, so a killed lilbee orphaned its whole fleet, and an orphaned lilbee serve kept the server lock and made the next start refuse.

Spawned children are now bound to lilbee's lifetime by a death pipe. The kernel closes it however lilbee dies, SIGKILL included, and a watcher reaps the child at EOF. Kernel primitives still win where they exist; the pipe covers macOS and every host where they do not. The device probe and launcher-spawned serve are bound the same way.

A wedged GPU probe can no longer hang lilbee

subprocess.run kills a child when its timeout expires, then waits for it to be reaped with no bound. A process stuck in uninterruptible driver I/O ignores even SIGKILL, so that wait never returns. The memory estimator, the NVLink probe and the live GPU util sampler all had this hole, and the sampler holds a lock while it hangs, so one wedge stalled every other sampler behind it.

All three now share one bounded reaper: kill the whole process group, wait a bounded time, then abandon an unkillable child rather than block the caller.

Content-addressed prebuilt engines consumed by the release builds. Managed by CI; not a user-facing release.

This release is about startup: dev725 was slow to become usable, and that is fixed here, along with an embedding bottleneck that capped ingest throughput.

Startup is fast again

A relaunch with the engine already warm now reaches a usable chat in about 5 seconds instead of 36. The window where the TUI is drawn but still refuses typing drops from about 33 seconds to about 2.

Three separate costs were compounding, all fixed (#600, #602):

• mmap on local disk. The chat server defaulted to --no-mmap, forcing a full read of the weights on every load instead of letting the OS page them in. Local disk now uses mmap; --no-mmap is kept only for network filesystems.
• GGUF metadata cached to disk. Reading a dozen scalar fields meant re-parsing the whole header, 57 to 61 seconds for a 2.6 GB model, on every launch. It is now cached across processes.
• Readiness probe stopped rebuilding SSL. The probe rebuilt a full SSL context on every tick to make plain HTTP calls to localhost, 22.65% of all TUI CPU. All probes now share one client.

Embedding throughput no longer caps ingest

Embedding responses were decoded from JSON float literals, 4096 parsed floats per vector under the GIL, so one CPU core decoded every vector no matter how many GPUs fed it. On an 8-card host this pinned ingest near 160 docs/sec with cards starved. Responses now arrive base64-encoded and decode in a single numpy call (#601).

Keep-warm and model switching behave

• Keep-warm actually keeps warm. A TUI session now holds the engine for its lifetime instead of the idle timer unloading the model after five minutes regardless of the setting. Closing lilbee releases it (#600).
• Model switching waits for the model. Switching chat models keeps the input locked until the new model can serve, names the model it is loading, and warms it eagerly instead of on the next prompt (#600).

Add local files without copying them

add now indexes files where they live instead of copying or symlinking them into a documents directory, so there is no second on-disk copy to keep in sync. Removal is index-only and expands folders and globs, so source bytes are never deleted. Discovery and hashing run in parallel (#590).

Under the hood

Windows tests run on the selector loop with leaked threads surfaced (#593), ollama is pinned in CI (#595), the release smoke test authenticates its health probe (#597), and the sessions load tests drop a redundant fsync barrier (#598).

Full Changelog: v0.6.90b420.dev725...v0.6.90b420.dev726

Highlights

This release is mostly about how lilbee runs its models. Until now every lilbee process on a machine started its own inference engine and loaded its own copy of the weights, so a TUI, an MCP server, and a background sync each paid the full memory and startup cost. They now share one engine (#547). Start a second lilbee and it attaches to the engine that is already resident instead of standing up another one.

The lifecycle around that engine is sturdier. A lilbee that crashes or is killed no longer leaves its engine running with its VRAM pinned (#589), and a single machine serves more agents at once, with crawl-to-markdown conversion moved off the daemon's core loop so a crawl no longer stalls request handling (#583).

GPU support is more dependable across hardware. Detection and memory sizing were fixed for Intel, AMD, NVIDIA, and Apple hosts (#576), and an Intel GPU whose utilization has no readable source now points you at igt-gpu-tools instead of showing nothing (#564).

On the safety side, every HTTP route now requires the session token (#570). The rest of the release is correctness and stability work: placement handlers moved off the event loop with top_k bounded at the MCP and CLI entry points (#585), supported architectures read from the pinned engine rather than the gguf package (#579), and a batch of test-determinism and watchdog fixes that keep the suite honest under load.

Known issue: slow cold start

Cold start on this release is slower to become ready than it should be. The default is currently tuned for network-mounted storage, which is the wrong choice for the common local-disk setup. This is a known mistake. It does not affect correctness or results, and the fix is coming in the next release.

Hint at igt-gpu-tools when an Intel GPU's utilization has no readable source

#564

Auto-dispatch the compat packaging fan-out on release

#577

Let scripted side_effect sequences absorb surplus mock calls

#578

Read supported architectures from the pinned engine, not the gguf package

#579

Require the session token on every HTTP route

#570

Fix GPU detection and memory sizing on Intel, AMD, NVIDIA and Apple hosts

#576

Fix the random ValueError from the catalog tab strip under parallel pytest

#582

Take placement handlers off the event loop and bound top_k at the MCP and CLI entry points

#585

Fix the ollama segfault in the integration job

#584

Share one inference engine across every lilbee process a user runs

#547

Dump wedged-worker stacks with a faulthandler watchdog in CI

#592

Make the vision-dispatch and grid-focus tests deterministic under load

#591

Stop crashed and SIGTERM'd lilbee from orphaning its inference engine

#589

Raise the single-machine agent-fleet ceilings and get crawl conversion off the daemon's core

#583

Full Changelog: v0.6.90b420.dev724...v0.6.90b420.dev725

Fix resuming a session from the drawer by click or enter

#563

Full Changelog: v0.6.90b420.dev723...v0.6.90b420.dev724

What's Changed

• Honor sessions_enabled on every surface, and make the MCP schema budget deterministic by @tobocop2 in #559
• Give agent sessions their own flag, off by default by @tobocop2 in #560
• Wait for async effects in TUI tests instead of pumping the bus once by @tobocop2 in #562
• Give HTTP chat context management: windowing, compaction, and a compaction event by @tobocop2 in #561

Full Changelog: v0.6.90b420.dev722...v0.6.90b420.dev723

Chat sessions: persist, list, resume, and manage conversations

#536

• Conversations save as you chat and survive quitting.
• List, resume, rename, and delete them from the TUI, HTTP server, MCP, and CLI.
• ctrl+o (or /sessions) opens a left-docked quick switcher; a full-screen Sessions tab handles management.
• lilbee sessions list / show / rename / delete from the shell.
• Titles are derived from the first message and can be renamed.
• Storage is an append-only JSONL log, one file per session under the data dir, with a per-session write lock. No LanceDB lock contention.
• Agent sessions are separate from human sessions: MCP-created sessions stay out of the TUI and HTTP lists, and crossing the boundary is explicit by id (claim=true on append, POST /api/sessions/{id}/claim to bring one back).
• Opt-in compaction folds older turns into a running summary instead of dropping them, with a context chip by the prompt. Off by default.
• Sessions can be turned off entirely.

Fix macOS/Vulkan engine index pointing at never-uploaded wheels

#555

Fail loud when the GPU device probe wedges instead of hanging serve forever

#540

Retry a member that died losing the port-bind race

#543

Harden the executable build cells against two runner failure modes

#544

Keep the CI ollama models loaded and warm them at setup

#545

Give the RAG ask tests timeout headroom, and make the opt-in real

#546

Fix a chat-mode toggle crash and the flaky TUI tests

#539

Full Changelog: v0.6.90b420.dev721...v0.6.90b420.dev722

Fix agent launcher chat window and hermes grounding

#534

Wait out a cold embed replica instead of dropping files on 429

#535

Add the towing-limits reel to the README and the tutorial

#538

Fix the chat not scrolling to the end of an answer

#537

Full Changelog: v0.6.90b420.dev720...v0.6.90b420.dev721

A rebuilt search engine

The headline of this release is retrieval. Search, ask, the TUI, and the MCP tools now share one relevance policy, so a query comes back ordered the same way everywhere instead of each path scoring results on its own. The full engineering writeup, including why the ranking algorithm changed, lives in #508.

What changed, by the question you type

Each row is a question shape you can type, before and after, with the measured result from an independent differential evaluation (old pipeline vs this release over one shared index, exact oracles where counting was possible, LLM-judged where grading was needed).

How the ranking works now

Two search arms run over every question: a vector arm that finds passages that mean something similar, and a keyword arm that finds the actual words. Their scores are in different units and can't be blended, so the engine fuses them by reciprocal rank, looking only at where each passage placed in each list, and expresses the result as one bounded [0,1] score. A normalize-and-blend approach was tried first and measured about 21% worse precision, so it was reverted; that regression is now pinned by a synthetic test so it can't come back silently.

Because every passage now carries the same bounded score, the stages that used to each invent their own reconciliation (sort, abstention floor, set-cover selection, concept boost, reranker blend) all compare the same field, which is what turns previously dead knobs back on.

In the honest column: topical ranking quality is not claimed to improve. The raw retrieval layer is at parity with the old pipeline within measurement noise, with ANN recall recovery added for large-index safety. The wins above are the routed fixes and the working relevance score, and they are measured, not asserted.

Also in this release

• Chat answers survive the markdown toggle, and chat_ready reports true only once the engine can actually serve a request. (#533)
• click and pillow bumped to patched versions. (#530)

Full Changelog: v0.6.90b420.dev719...v0.6.90b420.dev720

Multimodal VRAM estimates, fixed at the source

lilbee sizes every model with gguf-parser before placing it. That estimate decides whether a model fits your hardware and how it splits across GPUs.

v0.24.1 badly inflated that number for multimodal projectors. Vision models were being judged too big for cards that could run them fine.

Rather than working around it, the fix went into gguf-parser itself. It landed upstream in gpustack/gguf-parser-go#22 and shipped as v0.25.0, which lilbee now builds against. It corrects:

• unknown projector types
• flash attention handling
• projector type detection
• audio sizing
• images with no declared size

Engine loading fixed

Leave a chat sitting for a while and the model gets unloaded. Coming back to it used to leave you stuck on "the engine is not ready yet", forever. Every retry said the same thing.

The engine now reloads on your next prompt, whether it was unloaded by the idle timer, a crash, or an OOM.

keep-engine-warm does what it says

The setting did the opposite of its name. Turning it on is what made the engine unload on you.

Now: on keeps the model loaded, so prompts answer instantly. Off frees the memory while you are away and reloads on your next prompt.

You can run more than one lilbee at once

Two instances sharing a data dir used to overwrite each other's engine config, because it was written to a single fixed path. A restart would then pick up the sibling's ports and TTL.

The config is now namespaced per process, like the state files already were. Two instances run side by side, each with its own engine.

Quitting one leaves the other running. Quitting both leaves nothing behind, so no orphaned VRAM.

Clear citations

Inline markers were numbered by retrieval position, but the Sources block was a deduplicated file list. An answer could cite [6] above a list with three entries.

Passages are now numbered by source file. The numbers stay stable while streaming and line up with the list, one to one.

Sources used to render as raw home-directory paths that wrapped into a blob. They are now readable labels, and you can click one to open the document.

A model that wrote its own Sources list used to double up with lilbee's. Those get filtered out of the stream now.

TUI fixes

A full pass over the interface, driven end to end as a first-time user.

• Enter was eaten by the completion dropdown, and could rewrite what you typed, so /model ran /models. Enter now sends exactly what you typed.
• Escape used to strand your draft. It now sends.
• The first-run wizard would happily save a chat model that was not installed, whose first prompt could never answer.
• Errors leaked Python. /set top_k abc said invalid literal for int() with base 10. It now says top_k needs a whole number.
• The command palette carried one entry per indexed document, so thousands on a real corpus. It is constant cost now, and every slash command is reachable from any view.
• A missing chat model now fails with a reason that names it.
• Focused buttons follow your theme, engine loading shows real progress, the model bar survives an 80 column terminal, and Ctrl+C no longer needs two presses.