Skip to content

SIGSEGV in HNSW parallel inserts — missing hnsw:num_threads on collection creation #974

@vince-mod3

Description

@vince-mod3

What happened?

The MCP server crashes with a SIGSEGV (segmentation fault 11) during HNSW index operations. The crash occurs in repairConnectionsForUpdate / addPoint when ChromaDB's default multi-threaded ParallelFor triggers a race condition in the HNSW graph mutation path. This reproduces on both chromadb <1.0 (hnswlib.cpython-*-darwin.so) and chromadb 1.5.x (chromadb_rust_bindings.abi3.so).
The root cause is mempalace/backends/chroma.py not setting hnsw:num_threads on get_or_create_collection, so ChromaDB defaults to multi-threaded parallel inserts.

Related: #965

What did you expect?

The MCP server should not crash during normal add_drawer / auto-mining operations.
How to reproduce:

  1. Start the mempalace MCP server
  2. Trigger auto-mining on a workspace, or issue multiple add_drawer calls
  3. Server segfaults intermittently during HNSW index insertion

Environment:

  • OS: macOS 26.3.1 (ARM64, M4 Pro)
  • Python version: 3.9.6 and 3.14
  • MemPal version: 3.3.0 (chromadb 1.5.8, also reproduced with 0.6.3)

Suggested fix:

# mempalace/backends/chroma.py
             collection = client.get_or_create_collection(
-                collection_name, metadata={"hnsw:space": "cosine"}
+                collection_name, metadata={"hnsw:space": "cosine", "hnsw:num_threads": 1}
             )

hnsw:num_threads only affects the batch insert path. Mempalace inserts drawers individually so there is zero performance impact. Queries are unaffected.

Update: The fix needs to cover three code paths:

  1. New collections (get_or_create_collection / create_collection) — add "hnsw:num_threads": 1 to metadata
  2. Existing collections (get_collection) — call collection.modify(metadata={"hnsw:space": "cosine", "hnsw:num_threads": 1}) so palaces created before the fix also get single-threaded inserts
  3. v3.1.0 palace.pycreate_collection has no metadata at all (no hnsw:space either)

Without item 2, any palace created before the fix will keep crashing until the user deletes and recreates their palace.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions