Skip to content

auto_ns silently collapses to default when memory_dir basename is non-discriminating (e.g., .../FOO/memory) #296

@memtomem

Description

@memtomem

Summary

With namespace.enable_auto_ns=true and no explicit namespace.rules, all files indexed from a memory_dir whose basename is memory / memories / plans silently land in default namespace — even when the user clearly intended the parent folder name (e.g., the Claude project id) to be the namespace.

Observed on a post-v0.1.12 install with auto-discovered Claude project memory dirs: 766 indexed chunks across 29 ~/.claude/projects/*/memory paths, all with namespace='default'.

Repro

  1. mm init (or rely on the auto_discover migration) so config.indexing.memory_dirs contains entries like ~/.claude/projects/FOO/memory.
  2. In config.json: "namespace": {"enable_auto_ns": true} (no rules, no custom default_namespace).
  3. mm index ~/.claude/projects/FOO/memory (or reindex via web UI).
  4. sqlite3 ~/.memtomem/memtomem.db "SELECT namespace, COUNT(*) FROM chunks GROUP BY namespace"
    → every row is default.

Expected: something like FOO (the Claude project id) for every file in that dir, since that's the only identifier the user actually cares about.

Root cause

packages/memtomem/src/memtomem/indexing/engine.py:_resolve_namespace (L291-326):

if self._ns_config.enable_auto_ns:
    parent = file_path.parent.resolve()
    memory_roots = {Path(d).expanduser().resolve() for d in self._config.memory_dirs}
    if parent not in memory_roots:              # ← guard
        name = parent.name
        if name and name not in (".", ""):
            return name

The guard is intentional: without it, parent.name would be literally "memory" for every Claude-projects file, which is useless as a namespace. But the guard only suppresses the bad option — it doesn't derive the good one. Falls through to default_namespace == "default" → returns NoneChunkMetadata.namespace default ("default") kicks in.

For a memory_dir path like ~/.claude/projects/FOO/memory, the useful namespace lives at memory_dir_path.parent.name (i.e., FOO), not at file_path.parent.name. The current logic has no way to express that.

Scope

Applies to any memory_dir whose basename is non-discriminating (memory, memories, plans, notes, …) — exactly the shape produced by the auto-discovered provider dirs shipped since v0.1.12:

  • ~/.claude/projects/*/memory (claude-memory category)
  • ~/.claude/plans (claude-plans category, single dir so less acute)
  • ~/.codex/memories (codex category, single dir so less acute)

The user's realistic workaround is hand-crafting a namespace.rules entry per project, which defeats the "auto" part of enable_auto_ns.

Proposed directions

Not ordering these — each has tradeoffs:

  1. Extend _resolve_namespace fallback: when the parent-of-file hits memory_roots, fall through to memory_dir_path.parent.name. Gated on a whitelist of "container" basenames (memory / memories / plans) so ordinary memory_dirs aren't surprised. Zero user-config change; implicit behavior shift may surprise existing installs though.
  2. New placeholder for NamespacePolicyRule.namespace: something like {ancestor:N} to pick N levels up. User writes:
    {
      "rules": [
        { "path_glob": "**/.claude/projects/*/memory/**",
          "namespace": "claude:{ancestor:2}" }
      ]
    }
    Explicit + no behavior shift for existing installs, but requires docs + wizard hand-holding.
  3. Ship a preset policy bundle with the mm init provider-dirs step: when the wizard picks Claude-memory dirs, it also appends a matching NamespacePolicyRule (e.g., claude:{parent} with a placeholder that means "project id"). Keeps the wizard as the single source of truth for provider integration.

My instinct is (2) + (3) together — (2) gives the primitive, (3) applies it automatically for the known provider surface, and (1) is left on the table to avoid implicit behavior drift for non-provider memory_dirs that legitimately expect the current guard.

Evidence it's not triggered by the current PR

The DB in question was populated by the pre-existing POST /api/index + mem_index paths, both of which go through _resolve_namespace unchanged since before v0.1.12. Not introduced by #295 (editable widget + per-dir status).

Related

  • fix: memory_dirs editable widget with per-dir index status #295 surfaces this indirectly by making it trivial to trigger reindex from the UI; the namespace bug was silent before because nobody was indexing these provider dirs.
  • Pre-existing NamespacePolicyRule only supports {parent} (immediate parent) — see packages/memtomem/src/memtomem/config.py:_ALLOWED_NS_PLACEHOLDERS.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions