You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Serve record stays "ready" through an in-process engine relaunch, so a supervisor cannot tell the relaunch from a runaway load #700
Splash 1.6.0 (first measured on 1.4.0), M4 Pro 48 GB, macOS 27.2, Qwen3.8-27B, Splash behind a supervisor.
What a supervisor in front of Splash does, and what it reads. On one Mac several engines serve several models to several clients, and a supervisor sits in front of them: it swaps models so the one asked for is loaded and the others are not; it admits requests against memory, holding a request while a load would not fit, waiting for memory before it starts a load, or picking another model; it stops a child whose memory grows while that child should be serving, because on a 48 GB Mac that growth is how a second model sneaks in; and it restarts a child that exited. Every one of those decisions is read from the engines' phases: loading, ready, failed. A relaunch the server starts inside its own ready phase is invisible to all of them: the supervisor reserves no memory for it, holds no requests for it, and reads its growth as a runaway.
What the record says.<runtime>/serve-PORT.json follows the first start: starting, loading, warming, ready, failed. After ready nothing writes it again: on_warming only fires before the first ready (runtime.py, _first_ready), and ready is written once in main. So an engine that fails after it served and relaunches in-process loads a whole model under a record that still says ready, same pid. /status does say it: transport.recovering is true from the failure until the new engine is ready.
What that looks like. Per-second memory history of the supervisor, 2026-10-24:
time
what happened
12:54:15
Splash: Engine failed · native loop did not answer status within 30 s
12:54:30 to 12:54:33
old engine exits: wired 38.5 to 10.7 GB
12:54:35 to 12:54:37
in-process relaunch loads: wired 7.2 to 13.3 GB, record says ready
12:54:37
brake stops it: growth 11.3 GB in the window
12:55:39 to 12:56:10
control: the same load, started by the supervisor itself, growth 11.1 GB, brake disarmed, no trip
13:01:27 to 13:01:48
second failure, same sequence, stopped again
Both relaunches were stopped; the identical load started by the supervisor went through. The only difference is who started the load and what the record said during it.
Proposal. Let the record follow the engine's life, not only its first start: at a relaunch write recovering (or the same loading and warming phases) and ready again when the relaunched engine serves, same pid. /status already carries this as transport.recovering, so this is the file-side twin for a follower that does not poll HTTP, the way the desktop app follows the file. A supervisor that polls can read /status meanwhile, and ours will.
With this, #617 (--engine-failure exit) is not needed: the server keeps the restart decision, the supervisor only needs to see it. I'll say so there.
Activity