Repository navigation
auto_index uses tens of GB of memory in a non-git folder containing code #713
Description
Version
codebase-memory-mcp 0.8.1
Platform
Linux (x64)
Install channel
GitHub release archive / install.sh / install.ps1
Binary variant
standard
What happened, and what did you expect?
With auto_index=true, opening an agent in a non-git folder that contains code indexes the whole tree and pushes RSS into the tens of GB. auto_index_limit=50000 does not help.
The limit guard counts files with git ls-files | wc -l (src/mcp/mcp.c:4953). In a non-git directory that returns 0, so the limit is never reached and auto-index proceeds with a full filesystem walk.
Expected: the limit applies to non-git roots as well, or auto-index skips a root that has no .git.
Reproduction
The same 60,000 files are indexed when the directory is non-git, but skipped once it is a git repo:
mkdir -p /tmp/ab-nongit /tmp/ab-git
seq 1 60000 | xargs -P8 -I{} touch /tmp/ab-nongit/f{}.py
cp /tmp/ab-nongit/*.py /tmp/ab-git/
( cd /tmp/ab-git && git init -q && git add -A && git -c user.email=t@t -c user.name=t commit -qm x )
codebase-memory-mcp config set auto_index true
INIT='{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"t","version":"1"}}}'
# non-git root: a project .db is created in the cache dir (full index ran)
( cd /tmp/ab-nongit && printf '%s\n' "$INIT" | timeout 8 codebase-memory-mcp )
# git root, same files: no .db (auto-index skipped on the file-count limit)
( cd /tmp/ab-git && printf '%s\n' "$INIT" | timeout 8 codebase-memory-mcp )Logs
Diagnostics trajectory (memory / performance / leak issues)
Project scale (if relevant)
No response
Confirmations
- I searched existing issues and this is not a duplicate.
- My reproduction uses shareable code (a dummy snippet or a public OSS repository), not proprietary code.
Activity
DeusData commentedon Jun 30, 2026
Hey, yes we seen this unfortunate behavior also from other reporters. This is kinda architecturally, so need to think about this a bit. For these cases the "easy" fix is to basically make a hook when you commit sth that this triggers an incremental index rather than using the auto indexer (u can turn it off as you have seen already). This will be patched once we found a good concept
DeusData commentedon Jun 30, 2026
Adding this to the large-repo performance epic. The core acceptance criterion is that
auto_index_limitmust protect non-git roots too, or non-git auto-index should fail closed, so opening an agent in a large plain directory cannot accidentally walk the whole tree.fix(watcher): stop nested non-git dirs and monorepo siblings retrigge…
fix(mcp): free the extraction bitset cache on the in-process auto-ind…
fix(mcp): normalize the autoindex.skip root path in diagnostics (#713)
Merge pull request #2116 from DeusData/fix/713-1826-nongit-guard-open…
DeusData commentedon Sep 11, 2026
Fixed on main by 713764b (via #2116): the non-git auto_index guard is now bounded by the discovery file count rather than admitting an unbounded folder, and the over-limit skip diagnostic names the root cross-platform. Thanks for the detailed report!
fix(mcp): bound the non-git auto_index guard by discovery count, not …
fix(mcp): bound the non-git auto_index guard by discovery count, not …
fix(mcp): free the extraction bitset cache on the in-process auto-ind…
fix(mcp): normalize the autoindex.skip root path in diagnostics (Deus…