Crash in pass=definitions on complex C++ files — still broken in 0.7.0

Crash in `pass=definitions` on complex C++ files — still broken in 0.7.0

Version: 0.7.0 (--version; update confirms it's the latest release)
Platform: Windows 11 x86-64
Status: Marked fixed in a prior issue (which specifically named nlohmann/json.hpp), but still reproduces on
0.7.0.

Repro

Index the unmodified upstream nlohmann/json.hpp v3.12.0
(MIT, https://github.com/nlohmann/json/releases/download/v3.12.0/json.hpp):

mkdir /tmp/repro && cp json.hpp /tmp/repro/
codebase-memory-mcp cli index_repository '{"repo_path":"/tmp/repro"}'
# → Segmentation fault (exit 139 / Windows 0xC0000005)
# Last log line before crash: level=info msg=pass.start pass=definitions files=1

Crashes 100% deterministically. Also reproduces on Eigen:s Eigen/src/LU/Determinant.h (independent confirmation with
a different well-known templated header).

Minimum reproducing fragment: the first ~1015 lines of json.hpp 3.12.0 crash; ~1014 lines do not.

Key observations

It is cumulative, not triggered by a single construct.
Lines [950..1015] (66 lines, same ending) → OK.
Lines [500..1015] (516 lines, same ending) → CRASH.
The crash threshold scales with how much content precedes a given point, not with the local syntax. This rules out a
single bad construct and points to a capacity being exhausted.

The fault address is 0xffffffffffffffff, not 0x0.
Debugger (cdb) shows the crash is an access violation reading from address 0xffffffffffffffff inside
ts_parser_parse_with_options(). The faulting instruction pattern is:

mov rbx, [rbx+8] ; follow "next" pointer in a tree/list node
test rbx, rbx ; null check — catches 0x0 only
je (done)
mov eax, [rbx] ; CRASH: rbx = 0xffffffffffffffff

Tree-sitter uses 0xffffffffffffffff as its internal null-subtree sentinel. The null check (test/je) only catches 0x0,
not the sentinel, so the walk falls through and dereferences it.

The crash is in the vendored tree-sitter runtime, not in the cbm extraction layer.
Call chain (confirmed via cdb + symbol bracketing):

main → handle_index_repository → cbm_pipeline_run
→ cbm_pipeline_pass_definitions → cbm_extract_file
→ ts_parser_parse_with_options ← crash is inside here
[vendored tree-sitter runtime / C++ grammar]

The cbm extraction code (extract_defs.c, c_lsp.c, etc.) correctly uses ts_node_is_null() throughout. The bad
raw-pointer check is in internal/cbm/vendored/ts_runtime/src/.

Why the prior fix didnt hold

The regression test suite (test_stack_overflow.c) covers JS/Python/Go/TS/YAML stack-overflow paths but has no C++
crash regression test. Whatever was fixed before, this path was left uncovered and re-broken (likely by the arena/LSP
refactor in PRs #217/#322/#339 which introduced new AST-walking code).

Additionally: --version reports 0.7.0 but the MCP initialize handshake reports 0.10.0. If the prior fix landed under
a different version label, it may never have shipped in the published binary.

Suggested fix direction

In internal/cbm/vendored/ts_runtime/src/, find linked-list / child-array traversal loops that check if (ptr == NULL)
and replace with the sentinel-aware predicate (equivalent of ts_node_is_null()). The same pattern that
extract_node_stack.h applied to the extraction layer needs to be applied to the runtime traversal code.

A C++ regression test (index a 1000+ line templated C++ file, assert no crash) would prevent recurrence.

Hey, thanks for reporting. I am currently doing a lot of stability / bug hunting work. Will try to reproduce and fix your case.

I hit a closely-related — but I believe distinct — SEGV indexing templated C++, and a defensive
fix has been running on my fork for a few weeks. Sharing in case it's useful data, and because it
suggests templated C++ has more than one crash site.

What I saw: UBSan misaligned-access + SEGV indexing ns-3's src/core/model/ptr.h (templated
operator<, operator==, etc.). Crash was in c_eval_expr_type_inner in the C/C++ LSP layer
(internal/cbm/lsp/c_lsp.c) — c_eval_expr_type was surfacing a stale/mis-typed CBMType* through
chains of template-function registry lookups; the pointer came back unaligned, pointing into string
data.

How it differs from this issue:

	This issue (#424)	What I saw
Crash site	`ts_parser_parse_with_options` (vendored tree-sitter runtime)	`c_eval_expr_type_inner` (cbm LSP layer)
Pass	`pass=definitions` (parse)	type-evaluation / resolve path
Fault pointer	`0xffffffffffffffff` (tree-sitter null-subtree sentinel)	unaligned pointer into string data

So your 0xffff… sentinel evidence points at the runtime walk, whereas mine was registry
corruption in the LSP type evaluator. A fix for one won't cover the other — but both are reached
by the same template-heavy header path, and the "scales with preceding content, not local syntax"
behavior you describe is consistent with a capacity/registry issue upstream of the crash.

My mitigation (defensive guard only — I did not fix the underlying registry corruption):
added cbm_type_pointer_looks_valid() (8-byte alignment + kind range bound + a shallow
FUNC.return_types[0] peek) and made c_eval_expr_type fall back to cbm_type_unknown() when the
result fails validation. unknown propagates correctly through callers, so result shape is
unchanged. It's stopped the crash on ns-3 for us but is admittedly a band-aid over whatever produces
the bad pointer.

Happy to open a PR against this (it's a ~36-line patch in c_lsp.c, pure C, conventional commit
already) with a C++ regression test that indexes a templated header and asserts no crash — which
would also start covering the gap you noted in test_stack_overflow.c. Since it touches the
indexing/LSP path, wanted to check you're interested before sending it. Either way, flagging that
the C++ crash surface is plural, not singular.

@mattall

Hey @mattall, a PR would be appreciated :) Will check. The more info the better to pin it

Note: That PR linked should probably NOT close this issue... The only relation is I saw someone had an issue with C++ files crashing the cbm and remembered I had fixed A crash related to C++ files. But that fix is not this issue :)

@mattall

Thx @mattall

Bind tree-sitter runtime to mimalloc in prod build to fix C++ indexin…

smoke: index a large templated C++ header to reproduce the #424 prod …

Bind sqlite3 and libgit2 to mimalloc explicitly in prod build (#424 f…

Upgrade vendored mimalloc 3.2.8->3.3.2 + platform-correct override; r…

Fix out-of-bounds read in cbm_type_substitute when type_args is short…

Fixed — thank you for the detailed report and the templated-C++ reproductions; they were essential to tracking this down.

The root cause turned out to be an allocator mismatch rather than a tree-sitter sentinel. In the production build mimalloc overrides the global allocator, but on Windows static-MinGW that override was incomplete — ts_malloc resolved to mimalloc while ts_free could resolve to the CRT, corrupting the heap freelist mid-parse on large/templated C++. That is the 0xffff... freed-pointer crash, and it scaled with parse size, which is why bigger/more-templated files hit it.

The fix (eab4148 and the surrounding commits):

upgraded vendored mimalloc 3.2.8 -> 3.3.2, which adds static-CRT malloc override support on Windows;
enabled the real MI_MALLOC_OVERRIDE on MinGW specifically (on Unix the link-order override already worked, and forcing it there actually misbehaves);
bound the tree-sitter, sqlite, and libgit2 runtimes to mimalloc explicitly as defense-in-depth, so allocate and free can never split across allocators on any platform.

Verified green across all platforms, including a Windows smoke job that now indexes a large templated header without crashing. This should be resolved in the next release — please reopen if you still hit it. Thanks again for the careful report!

AWESOME. When (roughly) will there be a release including this?

fix(win): escape spawned command-line args so the index worker gets v…

Navigation Menu

Repository navigation

Crash in `pass=definitions` on complex C++ files — still broken in 0.7.0 #424