Skip to content

fix(symbols): run symbol work in a spawned worker process - #54

Merged
JRedeker merged 2 commits into
mainfrom
work/work-06dbccd495188d66b6dd2ad1
Oct 25, 2026
Merged

JRedeker merged 2 commits into
mainfrom
work/work-06dbccd495188d66b6dd2ad1

Conversation

@JRedeker

Copy link
Copy Markdown
Contributor

Problem

Symbol builds and queries are CPU-bound Python. In the shared daemon they hold the GIL, so every other tool call waits behind a build. After #53, search_text against Vision still took 17.0 s during a 17.1 s symbol build, against 0.47 s alone.

Change

All symbol work runs in one persistent worker process started with spawn (src/lgrep/symbol_worker.py):

  • index checks, refreshes, and bootstraps; explicit index_symbols_folder / index_symbols_repo jobs;
  • search_symbols, get_symbol, get_symbols, search_references, get_file_outline, get_repo_outline;
  • list_repos, invalidate_cache, and prune_symbols, which parse index bodies or evict the worker's index cache.

The daemon routes calls on the existing symbol lane and receives small picklable results. The worker:

  • points stdout at stderr before loading any lgrep module that logs, and forwards its log events to the daemon's sinks with a worker_pid field;
  • caps malloc arenas and trims its heap when idle;
  • receives the shutdown stop event at start. A worker still running 3 s after shutdown begins is killed.

A crashed worker is replaced on the next call. Its in-flight calls return code="symbol_worker_restarted", retryable=true. Unchanged: single-flight refresh per path, a cancelled query does not stop its build, explicit-job status, counters and credential redaction, and the in-process CLI/Python path (InProcessSymbolWorker). No dependency or storage-format change.

The native memory hooks move from lgrep.server.memory to lgrep.memory, so the worker can use them without importing the server package.

Verification

  • tests/test_symbol_worker.py (11 tests) covers search during a real spawned-worker build, an empty stdout, crash replacement, the stop event, the shutdown kill, and retryable tool errors. The responsiveness test fails when the build runs in the daemon: its one overlapping search took 2.45 s, the length of the whole build.
  • Full suite: 1280 passed, 1 xfailed.
  • Live stdio MCP measurement: search_text against Vision during a full index_symbols_folder build (164 files, 88,218 occurrences), fresh symbol store:
code alone median during build median during build max searches overlapped
main (4a51c8d) 0.031 s 0.631 s 1.143 s 3
this branch 0.056 s 0.050 s 0.095 s 15

Cost

The worker is a second process: an idle worker holds about 161 MB RSS. Importing lgrep.storage loads LanceDB through the package __init__, which adds up to about 2 s to a cold worker start and part of that RSS. A follow-up covers it.

Symbol builds and queries are CPU-bound Python. In the shared daemon they
held the GIL, so other tool calls waited behind a build: search_text took
17.0 s during a 17.1 s symbol build against 0.47 s alone.

All symbol work now runs in one persistent worker process started with
spawn: index checks, refreshes, bootstraps, explicit index jobs, symbol
queries, outlines, list_repos, invalidate_cache, and prune_symbols. The
daemon routes calls and receives small results. The worker points stdout
at stderr and forwards its log events to the daemon's sinks, caps malloc
arenas, and receives the shutdown stop event at start. A worker that
outlives the shutdown grace is killed. A crashed worker is replaced and
its in-flight calls return a retryable symbol_worker_restarted error.

The native memory hooks move to lgrep.memory so the worker can use them
without importing the server package.
A replacement worker counted snapshot tokens from 1 again, so a stale
token from a lost worker could name another repository's snapshot. Tokens
are now random. A failure reply is summarized unless it survives a pickle
round trip, so an exception that cannot unpickle no longer makes the
daemon discard a healthy worker.

The symbol lane now carries quick lookups beside index checks, so it
grows to eight threads and the worker to sixteen. Only FastMCP's
outside-a-request error selects the in-process path. The operations doc
names which tools report a lost worker as symbol_worker_restarted.
@JRedeker

Copy link
Copy Markdown
Contributor Author

Live overlap timing on head 762f494, after the review fixes. Method: worktree server over stdio MCP, fresh symbol store, search_text against Vision every 0.25 s during a full index_symbols_folder build of 164 files and 88,218 occurrences.

value
build wall time 3.72 s
search_text alone, median / max 0.030 s / 0.048 s
search_text during build, median / max 0.031 s / 0.034 s
searches overlapping the build 12
server stderr at LGREP_LOG_LEVEL=WARNING 0 bytes

The same harness on main (4a51c8d) measured a 0.631 s median and a 1.143 s max during the build, against 0.031 s alone.

@JRedeker
JRedeker enabled auto-merge (squash) October 25, 2026 05:56
@JRedeker
JRedeker merged commit dfb38ac into main Oct 25, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant