Skip to content

cli(init): add --fresh to drop wizard-untouched config leftovers#255

Merged
memtomem merged 1 commit intomainfrom
feat/wizard-fresh-flag
Apr 18, 2026
Merged

cli(init): add --fresh to drop wizard-untouched config leftovers#255
memtomem merged 1 commit intomainfrom
feat/wizard-fresh-flag

Conversation

@memtomem
Copy link
Copy Markdown
Owner

Summary

Phase 2 of the config-leftover-cleanup work. PR #254 added a Preserved
block to mm init that flagged non-default keys inherited from a
previous config but never asked about by the wizard — typically
runtime values pinned by the Web UI's save_config_overrides dump
(memory-dirs add/remove, section save), with mmr.enabled=true as
the canonical example. Surfacing them helped diagnose the silent
persistence pattern, but cleanup still required hand-editing
~/.memtomem/config.json.

mm init --fresh resets every wizard-untouched canonical key whose
value differs from the built-in default, then runs the normal wizard.
Default behaviour is unchanged — --fresh is opt-in.

What this does

A key is dropped iff ALL three gates hold:

  1. Canonical only — present in Mem2MemConfig().model_dump() flat
    key set. Keys outside this shape (manual user additions,
    downgrade-scenario sections, plugin extensions) are preserved
    unconditionally.
  2. Wizard-untouched — not in _flatten_init_data_keys(init_data).
    Wizard-touched keys get overwritten by the merge anyway, so dropping
    them first is redundant.
  3. Not in preserve list_FRESH_PRESERVE_KEYS (12 keys) holds
    credentials (api_key, secret), endpoints (base_url,
    webhook.url), and user-curated lists
    (indexing.exclude_patterns, namespace.rules,
    indexing.supported_extensions, search.system_namespace_prefixes,
    webhook.events). Derivation is documented inline (grep methodology
    against config.py at this commit) so the list can be re-audited
    when new fields land.

A timestamped config.json.bak-<unix-ts> is written before any drop
so the previous state is recoverable. Backup is skipped when the drop
set is empty — avoids cluttering ~/.memtomem/ on repeated --fresh
runs and makes --fresh idempotent on disk. If backup itself fails,
the run aborts before merge with a red error rather than silently
proceeding without recovery.

What this doesn't do

  • Web UI persistence path is unchanged. A web save after --fresh
    can re-pin the dropped values; the Reset block prints a restart
    caveat. Surface fix; root cause stays in follow-ups (1) and (2).
  • No partial drop. All-or-nothing on the drop set. For
    fine-grained control, edit config.json directly or use the web UI.
  • User custom keys preserved. --fresh is "reset wizard-untouched
    canonical settings", not "factory reset".

--help output

  --fresh                         Reset wizard-untouched canonical settings to
                                  their built-in defaults. Preserves user-
                                  added custom keys, credentials
                                  (api_key/secret), endpoints (base_url/url),
                                  and user-curated lists (exclude_patterns,
                                  namespace.rules, etc). Backs up the previous
                                  config.json to config.json.bak-<ts> only if
                                  at least one key is dropped. For fine-
                                  grained control, edit
                                  ~/.memtomem/config.json directly or use the
                                  web UI.

Reset block (example)

Reset to default (--fresh dropped wizard-untouched leftovers):
  [-] mmr.enabled: true → false (default)

Backup saved to: ~/.memtomem/config.json.bak-1776517180

[!] If the web UI is running, restart it to pick up the new config.
    Otherwise a web save may restore the reset values.

Zero-drop case collapses to a single line:

--fresh: no wizard-untouched leftovers to reset.

Tests

TestFreshFlag adds 7 cases (parametrize expands to 18 sub-tests):

  • test_fresh_drops_preserved_mmr — end-to-end happy path, asserts
    Reset block format, backup creation, and section pruning.
  • test_fresh_preserves_user_data_keys (parametrize × 12) —
    structural lockstep between _FRESH_PRESERVE_KEYS and the test
    parametrize: adding a key to one without the other fails this test.
    Each case asserts both path-existence and value equality after a
    round-trip through --fresh.
  • test_fresh_preserves_memory_dirsindexing.memory_dirs is the
    most surprising data-loss path if it ever leaves init_data.
    Verifies via direct _compute_fresh_drops call that the drop set
    never contains it, plus end-to-end that the seed survives.
  • test_fresh_keeps_user_custom_keys — keys outside the canonical
    shape (my_extension) survive --fresh while sibling canonical
    leftovers (mmr) get dropped.
  • test_fresh_no_op_when_nothing_to_reset — zero drops → no backup,
    no Reset block, the no-leftovers message instead.
  • test_fresh_with_non_interactivemm init -y --fresh via
    CliRunner; verifies the Phase-1 Setup-complete summary still
    prints alongside the Reset block.
  • test_fresh_handles_malformed_config — corrupt config.json +
    --fresh falls into the existing unreadable-config recovery path
    (one backup from there, none from --fresh since the post-recovery
    base is empty).
  • test_fresh_is_idempotent — second --fresh run with same state
    produces no second backup and a bit-identical config.
    time.time is monkeypatched so a same-second double-invoke can't
    mask the assertion via filename collision.

Test plan

  • uv run ruff check packages/memtomem/src packages/memtomem/tests
  • uv run ruff format --check packages/memtomem/src packages/memtomem/tests
  • uv run mypy packages/memtomem/src/memtomem/cli/init_cmd.py (advisory) — 0 issues
  • uv run pytest -m "not ollama" packages/memtomem/tests/test_init_cmd.py — 33 passed (14 existing + 19 new incl. parametrize)
  • uv run pytest -m "not ollama" packages/memtomem/tests/ -k "init or config or cli or wizard" — 238 passed
  • Manual: uv run mm init --help--fresh documented as above
  • Manual: seed {"mmr":{"enabled":true},"search":{"rrf_k":120}}, mm init -y --fresh --provider none — both keys reset, Reset block + backup path printed, second run reports no leftovers

Out of scope / follow-ups

This PR closes the wizard-side leak. The underlying silent-persistence
pattern still needs work in three other places, plus one
maintainability cleanup of --fresh itself:

  1. save_config_overrides dumps the full _MUTABLE_FIELDS set.
    Memory-dirs add/remove writes every mutable key to config.json,
    so runtime values (incl. defaults loaded from config.d/) get
    pinned into the override file. Persisting only changed keys would
    keep precedence working without --fresh.
  2. Web UI has no reset-to-default. Symmetric to (1): a user who
    unchecks MMR and saves writes mmr.enabled=false into
    config.json, permanently overriding any config.d/ fragment.
    Per-field reset or drop-equal-to-default-on-write would fix.
  3. Web UI hot-reload of config.json. After mm init --fresh,
    a running web instance still holds the old values in memory; a
    subsequent web save can re-pin the dropped values. mtime watcher
    on config.json would invalidate the cache. (The Reset block's
    "restart the web UI" caveat is the workaround.)
  4. _FRESH_PRESERVE_KEYS is hardcoded and goes stale. New
    credential/endpoint/user-data fields require manual additions to
    the constant; nothing enforces this at PR time. Schema-driven
    alternative: Pydantic Field(json_schema_extra={"fresh_preserve": True})
    or class-level PRESERVE_ON_FRESH: ClassVar[set[str]] collected
    into the constant at import. Defer until preserve list grows or
    drift incident occurs.
  5. --fresh write-after-backup atomicity. If config_path.write_text
    fails after backup succeeds, the orphan .bak-<ts> survives next
    to an unmodified original. Not data-lossy (original intact), but
    the leftover file is confusing. Tempfile + atomic rename, or
    delete-on-write-failure, would tighten this.

Breaking changes

None. --fresh is opt-in; default mm init behaviour is unchanged.

`mm init` (since #254) only flagged non-default keys preserved from a
previous config — e.g. `mmr.enabled=true` left over from the Web UI's
full-config dump. Surfacing them helped diagnose silent persistence,
but cleanup still required hand-editing `~/.memtomem/config.json`.

`--fresh` resets every wizard-untouched canonical key whose value
differs from the built-in default, then runs the normal wizard. The
drop set is filtered by three independent gates:

- Canonical only: keys outside `Mem2MemConfig().model_dump()` are user
  custom extensions (manual edits, downgraded-version sections, plugin
  data) and stay untouched.
- Wizard-touched: keys the current run will overwrite anyway are
  skipped — no point dropping them first.
- Preserve list: 12 keys covering credentials (`api_key`, `secret`),
  endpoints (`base_url`, `webhook.url`), and user-curated lists
  (`indexing.exclude_patterns`, `namespace.rules`,
  `indexing.supported_extensions`, `search.system_namespace_prefixes`,
  `webhook.events`) are never auto-dropped. Derivation is grep-based
  against `config.py` at this commit and documented inline so the list
  can be re-audited when new fields land.

A timestamped `config.json.bak-<ts>` is written before any drop so the
previous state is recoverable. Backup is skipped when the drop set is
empty (avoids cluttering `~/.memtomem/` on repeated `--fresh` runs;
makes `--fresh` idempotent on disk). If the backup write fails the
run aborts before merge — `--fresh` is destructive and proceeding
without recovery would silently lose user data.

The wizard's Setup-complete summary is unchanged. The Preserved block
is replaced by a Reset block listing each `key: before → after
(default)` plus the backup path; with zero drops the block collapses
to a single "no leftovers to reset" line. A web-UI restart caveat is
appended because an in-memory cache from a running web instance can
re-pin dropped values on the next save.

Tests (`TestFreshFlag`, 7 cases incl. parametrize 12):
- preserve-list ↔ test parametrize structural lockstep — adding to one
  without the other fails `test_fresh_preserves_user_data_keys`
- `memory_dirs` data-loss guard via `_compute_fresh_drops` direct call
  (regression catch if memory_dirs ever leaves `init_data`)
- idempotent on second run (no backup churn) — `time.time` monkeypatch
  pins ts so same-second double-invoke can't collide
- malformed-config recovery + `--fresh` interleave (no double backup)
@memtomem memtomem merged commit e8cd610 into main Apr 18, 2026
7 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 18, 2026
@memtomem memtomem deleted the feat/wizard-fresh-flag branch April 18, 2026 13:06
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants