Skip to content

config: env wins over config.json + config.d/ fragment layer (fixes #248)#249

Merged
memtomem merged 6 commits intomainfrom
feat/config-env-precedence
Apr 18, 2026
Merged

config: env wins over config.json + config.d/ fragment layer (fixes #248)#249
memtomem merged 6 commits intomainfrom
feat/config-env-precedence

Conversation

@memtomem
Copy link
Copy Markdown
Owner

Closes #248.

Summary

Resolves the precedence bug where ~/.memtomem/config.json silently
overrode MEMTOMEM_* env vars set in .mcp.json / MCP client env
blocks — i.e. every example our docs ship. Also adds the config.d/
drop-in layer discussed in #248 so integration installers can drop a
fragment without mutating shared state.

Five commits, each independently reviewable:

  1. Phase 1 — fix(config): env vars win over ~/.memtomem/config.jsonload_config_overrides now checks whether the matching MEMTOMEM_<SECTION>__<FIELD> env var is set and skips the override if so. Regression test pins PR docs(mcp): document Claude Code install scopes (local/project/user) #247's exact .mcp.json env block (1 scalar SQLITE_PATH + 1 list MEMORY_DIRS) — both must survive a full mm init + config.json.
  2. Phase 2a — feat(config): add MergeStrategy annotations to list fields — co-locates the merge strategy with the field via Annotated[list[X], APPEND | REPLACE]. 4 APPEND (memory_dirs, exclude_patterns, system_namespace_prefixes, webhook.events), 2 REPLACE (rrf_weights, importance.weights) — REPLACE on positional tuning knobs where appending would misalign element slots. No runtime change on its own.
  3. Phase 2b — feat(config): add ~/.memtomem/config.d/ fragment loader — scans config.d/*.json in lex order, respects per-field strategy, skips fields where env is set. config.json remains the REPLACE-on-set user-override layer (mm init UX unchanged).
  4. test(config): enforce merge strategy declaration on all list fields — walks every Mem2MemConfig section and fails if a list[*] lacks a MergeStrategy in its metadata. Guards against silent "default-to-REPLACE" drift when new fields get added.
  5. docs(config): document precedence and list merge strategies — adds a "Precedence and merge behaviour" section to docs/guides/configuration.md covering the full resolution order and the APPEND/REPLACE table. Other integration docs already advertised the env-wins semantic, so they now work as written.

Resolution order (lowest → highest priority)

defaults → config.d/*.json (lex order, per-field strategy)
         → config.json (REPLACE for every field)
         → MEMTOMEM_* env vars

Testing

  • 14 new tests in test_config_overrides.py covering Phase 1 precedence, Phase 2b merge semantics (APPEND dedup, REPLACE last-wins, scalar last-wins, env-wins-over-fragments, unknown section warn, malformed JSON warn, non-JSON files ignored), and the enforcement test.
  • Full suite: 1633 passed, 46 deselected (ollama marker), 0 failures.
  • ruff check and ruff format --check both clean.
  • mypy clean on the touched files.

Release notes

Breaking: MEMTOMEM_* environment variables now take precedence over ~/.memtomem/config.json. Previously config.json silently clobbered env overrides set in .mcp.json blocks. Any setup that was actively relying on file-wins-over-env behaviour will see env take effect — delete the env var or remove the field from config.json to restore the old outcome.

New: ~/.memtomem/config.d/*.json drop-in fragments are now loaded at startup. Integration installers can drop a per-client fragment (e.g. claude-desktop.json) instead of mutating the shared config.json; removing the file reverses the change.

Test plan

  • Unit tests: precedence + merge semantics
  • Regression test pinning docs(mcp): document Claude Code install scopes (local/project/user) #247's advertised .mcp.json env block
  • Enforcement test on list-field metadata
  • Manual smoke: restart memtomem-server with a config.d/test.json + overlapping config.json + env, confirm resolution matches the documented order end-to-end

Previously, ``load_config_overrides`` unconditionally ``setattr``'d every
field in ``~/.memtomem/config.json`` on top of the env-var-bound config,
which meant ``MEMTOMEM_*`` env vars set in ``.mcp.json`` / MCP client
``env`` blocks were silently ignored for any field the ``mm init``
wizard had persisted. Every integration doc in the repo (cloud-sync,
getting-started, mcp-clients, integrations/*) assumes env vars override
file values; the code did the opposite.

Invert the precedence: before applying a ``config.json`` key, check
whether ``MEMTOMEM_<SECTION>__<FIELD>`` is set in the environment and
skip the override if so. Fields without an env var still load from
``config.json`` as before.

New tests in ``test_config_overrides.py`` pin the expected precedence,
including an explicit regression case for the ``.mcp.json`` env block
shipped in #247 (one scalar ``SQLITE_PATH``, one list ``MEMORY_DIRS``) —
both must take effect post-``mm init``.
Annotate every ``list[*]`` field in config.py with an explicit merge
strategy via ``typing.Annotated`` so future fragment-based loaders
(``~/.memtomem/config.d/``) can pick append vs replace semantics per
field instead of inferring from the type. Keeps the declaration
co-located with the field and makes the strategy part of the type
annotation rather than a separate registry.

- ``APPEND`` (4 fields): memory_dirs, exclude_patterns,
  system_namespace_prefixes, webhook.events — elements are independent,
  duplicates dedup on merge.
- ``REPLACE`` (2 fields): rrf_weights, importance.weights — positional
  tuning knobs where element order/length is semantic; highest-priority
  source wins entirely.

No runtime behaviour change yet; loader landing in a follow-up commit
will read the ``MergeStrategy`` via ``FieldInfo.metadata``.
New ``load_config_d`` scans ``~/.memtomem/config.d/*.json`` in
lexicographic filename order and merges each fragment into the running
config. Intended for integration-installed drop-ins: ``mm init
<client>`` writes one fragment, ``mm uninstall <client>`` removes it.
Idempotent, no mutation of shared state.

Merge rules per field (strategy from ``MergeStrategy`` annotation):

- APPEND list → concatenate with current list, dedup by value
  (``Path`` normalised to string form for comparison).
- REPLACE list → fragment value replaces whatever was there.
- Scalar → last fragment applied wins.
- ``MEMTOMEM_<SECTION>__<FIELD>`` env var set → fragment value for that
  field is skipped (env wins, same rule as config.json).

Layering in ``create_components`` (lowest → highest priority):
defaults → config.d/*.json → config.json → env. ``config.json`` keeps
its current REPLACE-on-set semantics so the ``mm init`` wizard UX
("set my memory dir to X") is unambiguous; fragments are the layer
that respects per-field merge strategies.

Invalid fragments (malformed JSON, non-object top level, unknown
sections) are logged and skipped without crashing startup, matching the
existing ``config.json`` resilience.
Adds ``test_every_list_field_declares_merge_strategy`` that walks every
``Mem2MemConfig`` section and fails if a ``list[*]`` field lacks a
``MergeStrategy`` annotation in its metadata. The failure message lists
the offending fields and tells the contributor to wrap the type in
``Annotated[list[X], APPEND]`` or ``Annotated[list[X], REPLACE]``.

Guards against silent fragment-merge footguns: without this, someone
adding a new positional ``list[float]`` tuning knob without declaring
REPLACE would let a ``config.d/`` fragment accidentally append to it
via the default-to-REPLACE fallback.
Adds a "Precedence and merge behaviour" section to
``docs/guides/configuration.md`` covering the full resolution order
(defaults → ``config.d/*.json`` → ``config.json`` → env vars) and a
table of which ``list[*]`` fields use APPEND vs REPLACE under the
fragment layer.

Also amends the auto-discovered-memory-dirs note to mention the new
``config.d/`` layer alongside ``config.json`` so it stays accurate.

Other integration docs (``mcp-clients.md``, ``cloud-sync.md``,
``integrations/*``) already advertised the env-wins semantic that this
branch just made true in the code, so no wording change is needed there
— the advertised examples now work as documented.

(Commit also folds in ``ruff format`` over ``config.py`` and the new
test file, a no-op reflow of the lines touched in earlier commits.)
…/ embedding-reset)

``mm config show`` and the ``reset`` / ``embedding-reset`` diagnostics
only called ``load_config_overrides`` before the new fragment layer
landed, so ``config.d/*.json`` contributions (e.g. an integration's
``exclude_patterns`` APPEND) were invisible to these commands even
though the running server sees them. Add ``load_config_d`` before
``load_config_overrides`` in all three read-path call sites so the CLI
tells the truth about what the runtime resolves.

``config_set`` is intentionally left alone — ``save_config_overrides``
uses read-merge-write over ``MUTABLE_FIELDS``, so pulling fragment
values into the runtime before save would risk baking fragment state
into ``config.json`` (where it would persist past the fragment file's
removal). Fixing that is a separate refactor of ``save_config_overrides``
to track field provenance; out of scope here.
@memtomem
Copy link
Copy Markdown
Owner Author

Manual smoke — PASS

3 scenarios with HOME=/tmp/mm-smoke-…, config.json + config.d/claude-desktop.json, exercised via mm config show:

S1 — no env, both layers:

  • sqlite_path = /from/cfgjson.db ✓ (config.json only touches)
  • rrf_weights = [0.9, 0.1] ✓ (config.json REPLACE beats fragment [0.5, 0.5])
  • memory_dirs = ['/from/cfgjson'] ✓ (config.json REPLACE wipes fragment's /from/fragment)
  • exclude_patterns = ['*.tmp'] ✓ (fragment APPEND; config.json silent)

S2MEMTOMEM_STORAGE__SQLITE_PATH=/from/env.db set:

  • sqlite_path = /from/env.db ✓ (env wins)
  • Other fields unchanged from S1 ✓

S3 — config.json deleted, fragment only:

  • rrf_weights = [0.5, 0.5] ✓ (fragment REPLACE over default [1.0, 1.0])
  • memory_dirs = [~/.memtomem/memories, /from/fragment] ✓ (fragment APPEND to default)
  • exclude_patterns = ['*.tmp'] ✓ (fragment APPEND to empty default)

Followup commit fix(cli): load config.d fragments in read pathsmm config show / reset / embedding-reset were only calling load_config_overrides, so fragments were invisible to the CLI even though the server saw them. config set left alone intentionally (see commit body — avoids baking fragment state into config.json via save_config_overrides' read-merge-write).

Test-plan checkbox for manual smoke now checked.

@memtomem memtomem merged commit f672d38 into main Apr 18, 2026
7 checks passed
@memtomem memtomem deleted the feat/config-env-precedence branch April 18, 2026 10:21
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 18, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

config: ~/.memtomem/config.json silently overrides MEMTOMEM_* env vars, contradicting all .mcp.json docs

2 participants