Daemon peak RSS scales ~linearly with query concurrency (29 MB idle -> 3.07 GB at 20 concurrent queries) and releases slowly #1362

Version

codebase-memory-mcp 0.9.1-rc.1

Platform

macOS (Apple Silicon)

Install channel

GitHub release archive / install.sh / install.ps1

Binary variant

standard

What happened, and what did you expect?

Daemon peak working set scales very steeply with query concurrency, and releases only slowly afterwards. On a 63,840-node / 120,805-edge TypeScript graph, 20 concurrent query_graph aggregates took the daemon from a 399 MB resident baseline to 3.07 GB — and it was still at 3.07 GB ~30 s after every query had returned, decaying back over roughly 100 s.

Filing this because the RC notes specifically ask for multi-agent concurrency reports, and "several agents hitting the same projects concurrently" is exactly the shape that produces it. Nothing here is a leak — memory does come back — so this is a peak-working-set / admission-control question rather than the #581 class.

Sequential load shows nothing: 30 sequential runs of the same query left RSS drifting down (256 MB → 214 MB), and the daemon returns to single-digit MB when idle. Only concurrency produces the spike.

Measured decay after the 20-concurrent burst, with no further queries issued:

The concern is the ceiling, not the retention: 3.07 GB is 36% of the memory budget the daemon reports for itself on this box (budget_mb=8601, 24 GB RAM). Two or three agents each firing a handful of concurrent graph queries — the workflow the daemon exists to support — would plausibly exhaust that budget or the machine on a 16 GB laptop. I did not observe an OOM here, so I can't say what happens at the limit.

Expected: either bounded per-query working set, or admission control that queues concurrent heavy queries rather than letting each allocate its own peak simultaneously. The daemon already has "physical-job admission control for parallel indexing" per the release notes — the same idea applied to query execution would cap this.

Reproduction

Needs a reasonably large graph; I used a private 11,788-file Angular monorepo (63,840 nodes / 120,805 edges, --mode full). Any similarly-sized graph should do.

codebase-memory-mcp daemon start
DPID=$(codebase-memory-mcp daemon status | awk '/pid:/{print $2}')
Q="MATCH (a)-[r:CALLS]->(b:Method) RETURN b.qualified_name AS q, count(r) AS c ORDER BY c DESC LIMIT 50"

ps -o rss= -p $DPID                      # baseline

for i in $(seq 1 20); do
  codebase-memory-mcp cli query_graph --project <p> --query "$Q" < /dev/null > /dev/null 2>&1 &
done

# sample while they run and after they finish
for s in $(seq 1 15); do sleep 2; ps -o rss= -p $DPID; done

(Note the < /dev/null — without it the CLI can block on stdin, see #1359.)

Diagnostics trajectory (memory / performance / leak issues)

Scaling by concurrency, same query, same graph, daemon allowed to settle between runs:

Same query, same graph, daemon allowed to settle to idle between rounds, peak taken by
sampling ps -o rss= every 2 s for 16 s. It is close to linear at ~220 MB per
concurrent query up to 10, then sublinear at 20 — each concurrent query appears to
allocate its own working set with no sharing between them. Sixty seconds after the
10-query round RSS was still 1,877 MB.

(The 3.07 GB figure above came from a run whose baseline was 399 MB rather than 29 MB,
because it directly followed an index; the peak was the same order either way.)

Attempts to find a genuine leak, all negative. Sequential load shows no growth at all: 30 sequential runs of this same aggregate followed by 30 search_graph calls took RSS down from 256 MB to 214 MB, and the daemon sat at 6 MB when checked again an hour later. Idle RSS consistently returns to single-digit MB. So the allocator is releasing correctly — consistent with the "harmless on macOS/Linux" note about mem.allocator.not_owned in the RC's Known Issues. The spike is specific to concurrency.

Project scale (if relevant)

11,788 files indexed, 63,840 nodes / 120,805 edges, --mode full, 10,175 .ts files. 24 GB RAM, daemon self-reported budget_mb=8601. Permanent daemon (daemon start), build 0.9.1-rc.1 (e4fe1cf4f764).

Related

• Feature request: --max-memory flag for RAM-constrained hosts #580 (feature request: --max-memory flag for RAM-constrained hosts) — this is the workload that would make such a cap necessary.
• Distinct from bug: memory runaway (12.8GB+ RSS) during index_repository on root paths triggers OOM crash #1241 (12.8 GB RSS during index_repository on root paths) and PHP trait extraction consumes 64 GB memory and continues after MCP request timeout #1265 (PHP trait extraction) — both indexing/extractor-side; this is query-side and only appears under concurrency.
• Not the Memory leak: process grows to 50+ GB virtual memory over hours/days, crashes Windows #581 class: memory is released here.

Confirmations

• I searched existing issues and this is not a duplicate.
• My reproduction uses shareable code (a dummy snippet or a public OSS repository), not proprietary code.

Posted by Claude Code on behalf of @artaommahe