Symptom
While running `mm web` and copying files into a configured `memory_dir`, the
header indicator stayed in the "indexing" state for many minutes after the
copy. Server log showed continuous client polling:
```
INFO: 127.0.0.1:53468 - "GET /api/indexing/active HTTP/1.1" 200 OK
INFO: 127.0.0.1:53468 - "GET /api/indexing/active HTTP/1.1" 200 OK
... (repeating every few seconds)
```
The endpoint was answering `{"active": true}` long past the point where any
single-file index should have completed (single `.md` file, watcher path).
Suspicion
`IndexEngine._active_runs` is a manual counter incremented at the entry point
of `index_path`, `index_file`, and `index_path_stream` and decremented in
`finally`. The first two are lock-protected and unlikely to leak, but
`index_path_stream` runs outside `_index_lock` and is an async generator —
if the FastAPI `StreamingResponse` is cancelled (client disconnects, network
flap, browser tab close mid-run, anyio cancellation), the generator's
`finally` block doesn't always execute promptly, and could in pathological
cases not run at all. That would leave `_active_runs > 0` permanently until
the server restarts.
What to verify
- Reproduce by starting an SSE `/api/index/stream` run, killing the
client connection mid-stream, and curling `/api/indexing/active`. If
it answers `true` indefinitely, the leak is confirmed.
- Inspect whether `_active_runs` decrement guarantees survive
`asyncio.CancelledError` paths in async generators consumed by
`StreamingResponse`.
- Check whether watcher-triggered `_reindex` paths (`watcher.py`) are
counted (they go through `index_file`, which IS counted) — and
whether nested concurrent runs ever overlap and one finally is
skipped.
Possible fix shapes
- Replace the manual counter with an `asyncio.Lock` count derived from
active task tracking (`set[asyncio.Task]`), so cancelled tasks self-prune.
- Add a watchdog: if no run has registered/unregistered for N minutes,
reset the counter on the next poll (last-resort).
- Wrap the SSE `async for` body in `try/finally` outside the existing
one to make the decrement re-entrant against double cancellation.
Context
Surfaced during diagnosis of #639 (Sources tab orphan render). The
indexing-active polling itself isn't a critical bug — UI just shows a
spinner — but if confirmed it explains the "indexing forever" reports
that have been hard to reproduce.
Acceptance
- A test that exercises SSE stream cancellation mid-run and asserts
`engine.is_active === False` after the cancellation propagates.
Symptom
While running `mm web` and copying files into a configured `memory_dir`, the
header indicator stayed in the "indexing" state for many minutes after the
copy. Server log showed continuous client polling:
```
INFO: 127.0.0.1:53468 - "GET /api/indexing/active HTTP/1.1" 200 OK
INFO: 127.0.0.1:53468 - "GET /api/indexing/active HTTP/1.1" 200 OK
... (repeating every few seconds)
```
The endpoint was answering `{"active": true}` long past the point where any
single-file index should have completed (single `.md` file, watcher path).
Suspicion
`IndexEngine._active_runs` is a manual counter incremented at the entry point
of `index_path`, `index_file`, and `index_path_stream` and decremented in
`finally`. The first two are lock-protected and unlikely to leak, but
`index_path_stream` runs outside `_index_lock` and is an async generator —
if the FastAPI `StreamingResponse` is cancelled (client disconnects, network
flap, browser tab close mid-run, anyio cancellation), the generator's
`finally` block doesn't always execute promptly, and could in pathological
cases not run at all. That would leave `_active_runs > 0` permanently until
the server restarts.
What to verify
client connection mid-stream, and curling `/api/indexing/active`. If
it answers `true` indefinitely, the leak is confirmed.
`asyncio.CancelledError` paths in async generators consumed by
`StreamingResponse`.
counted (they go through `index_file`, which IS counted) — and
whether nested concurrent runs ever overlap and one finally is
skipped.
Possible fix shapes
active task tracking (`set[asyncio.Task]`), so cancelled tasks self-prune.
reset the counter on the next poll (last-resort).
one to make the decrement re-entrant against double cancellation.
Context
Surfaced during diagnosis of #639 (Sources tab orphan render). The
indexing-active polling itself isn't a critical bug — UI just shows a
spinner — but if confirmed it explains the "indexing forever" reports
that have been hard to reproduce.
Acceptance
`engine.is_active === False` after the cancellation propagates.