cli(init): add --fresh to drop wizard-untouched config leftovers#255
Merged
cli(init): add --fresh to drop wizard-untouched config leftovers#255
Conversation
`mm init` (since #254) only flagged non-default keys preserved from a previous config — e.g. `mmr.enabled=true` left over from the Web UI's full-config dump. Surfacing them helped diagnose silent persistence, but cleanup still required hand-editing `~/.memtomem/config.json`. `--fresh` resets every wizard-untouched canonical key whose value differs from the built-in default, then runs the normal wizard. The drop set is filtered by three independent gates: - Canonical only: keys outside `Mem2MemConfig().model_dump()` are user custom extensions (manual edits, downgraded-version sections, plugin data) and stay untouched. - Wizard-touched: keys the current run will overwrite anyway are skipped — no point dropping them first. - Preserve list: 12 keys covering credentials (`api_key`, `secret`), endpoints (`base_url`, `webhook.url`), and user-curated lists (`indexing.exclude_patterns`, `namespace.rules`, `indexing.supported_extensions`, `search.system_namespace_prefixes`, `webhook.events`) are never auto-dropped. Derivation is grep-based against `config.py` at this commit and documented inline so the list can be re-audited when new fields land. A timestamped `config.json.bak-<ts>` is written before any drop so the previous state is recoverable. Backup is skipped when the drop set is empty (avoids cluttering `~/.memtomem/` on repeated `--fresh` runs; makes `--fresh` idempotent on disk). If the backup write fails the run aborts before merge — `--fresh` is destructive and proceeding without recovery would silently lose user data. The wizard's Setup-complete summary is unchanged. The Preserved block is replaced by a Reset block listing each `key: before → after (default)` plus the backup path; with zero drops the block collapses to a single "no leftovers to reset" line. A web-UI restart caveat is appended because an in-memory cache from a running web instance can re-pin dropped values on the next save. Tests (`TestFreshFlag`, 7 cases incl. parametrize 12): - preserve-list ↔ test parametrize structural lockstep — adding to one without the other fails `test_fresh_preserves_user_data_keys` - `memory_dirs` data-loss guard via `_compute_fresh_drops` direct call (regression catch if memory_dirs ever leaves `init_data`) - idempotent on second run (no backup churn) — `time.time` monkeypatch pins ts so same-second double-invoke can't collide - malformed-config recovery + `--fresh` interleave (no double backup)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 2 of the config-leftover-cleanup work. PR #254 added a Preserved
block to
mm initthat flagged non-default keys inherited from aprevious config but never asked about by the wizard — typically
runtime values pinned by the Web UI's
save_config_overridesdump(memory-dirs add/remove, section save), with
mmr.enabled=trueasthe canonical example. Surfacing them helped diagnose the silent
persistence pattern, but cleanup still required hand-editing
~/.memtomem/config.json.mm init --freshresets every wizard-untouched canonical key whosevalue differs from the built-in default, then runs the normal wizard.
Default behaviour is unchanged —
--freshis opt-in.What this does
A key is dropped iff ALL three gates hold:
Mem2MemConfig().model_dump()flatkey set. Keys outside this shape (manual user additions,
downgrade-scenario sections, plugin extensions) are preserved
unconditionally.
_flatten_init_data_keys(init_data).Wizard-touched keys get overwritten by the merge anyway, so dropping
them first is redundant.
_FRESH_PRESERVE_KEYS(12 keys) holdscredentials (
api_key,secret), endpoints (base_url,webhook.url), and user-curated lists(
indexing.exclude_patterns,namespace.rules,indexing.supported_extensions,search.system_namespace_prefixes,webhook.events). Derivation is documented inline (grep methodologyagainst
config.pyat this commit) so the list can be re-auditedwhen new fields land.
A timestamped
config.json.bak-<unix-ts>is written before any dropso the previous state is recoverable. Backup is skipped when the drop
set is empty — avoids cluttering
~/.memtomem/on repeated--freshruns and makes
--freshidempotent on disk. If backup itself fails,the run aborts before merge with a red error rather than silently
proceeding without recovery.
What this doesn't do
--freshcan re-pin the dropped values; the Reset block prints a restart
caveat. Surface fix; root cause stays in follow-ups (1) and (2).
fine-grained control, edit
config.jsondirectly or use the web UI.--freshis "reset wizard-untouchedcanonical settings", not "factory reset".
--helpoutputReset block (example)
Zero-drop case collapses to a single line:
Tests
TestFreshFlagadds 7 cases (parametrize expands to 18 sub-tests):test_fresh_drops_preserved_mmr— end-to-end happy path, assertsReset block format, backup creation, and section pruning.
test_fresh_preserves_user_data_keys(parametrize × 12) —structural lockstep between
_FRESH_PRESERVE_KEYSand the testparametrize: adding a key to one without the other fails this test.
Each case asserts both path-existence and value equality after a
round-trip through
--fresh.test_fresh_preserves_memory_dirs—indexing.memory_dirsis themost surprising data-loss path if it ever leaves
init_data.Verifies via direct
_compute_fresh_dropscall that the drop setnever contains it, plus end-to-end that the seed survives.
test_fresh_keeps_user_custom_keys— keys outside the canonicalshape (
my_extension) survive--freshwhile sibling canonicalleftovers (
mmr) get dropped.test_fresh_no_op_when_nothing_to_reset— zero drops → no backup,no Reset block, the no-leftovers message instead.
test_fresh_with_non_interactive—mm init -y --freshviaCliRunner; verifies the Phase-1 Setup-complete summary stillprints alongside the Reset block.
test_fresh_handles_malformed_config— corruptconfig.json+--freshfalls into the existing unreadable-config recovery path(one backup from there, none from
--freshsince the post-recoverybase is empty).
test_fresh_is_idempotent— second--freshrun with same stateproduces no second backup and a bit-identical config.
time.timeis monkeypatched so a same-second double-invoke can'tmask the assertion via filename collision.
Test plan
uv run ruff check packages/memtomem/src packages/memtomem/testsuv run ruff format --check packages/memtomem/src packages/memtomem/testsuv run mypy packages/memtomem/src/memtomem/cli/init_cmd.py(advisory) — 0 issuesuv run pytest -m "not ollama" packages/memtomem/tests/test_init_cmd.py— 33 passed (14 existing + 19 new incl. parametrize)uv run pytest -m "not ollama" packages/memtomem/tests/ -k "init or config or cli or wizard"— 238 passeduv run mm init --help—--freshdocumented as above{"mmr":{"enabled":true},"search":{"rrf_k":120}},mm init -y --fresh --provider none— both keys reset, Reset block + backup path printed, second run reports no leftoversOut of scope / follow-ups
This PR closes the wizard-side leak. The underlying silent-persistence
pattern still needs work in three other places, plus one
maintainability cleanup of
--freshitself:save_config_overridesdumps the full_MUTABLE_FIELDSset.Memory-dirs add/remove writes every mutable key to
config.json,so runtime values (incl. defaults loaded from
config.d/) getpinned into the override file. Persisting only changed keys would
keep precedence working without
--fresh.unchecks MMR and saves writes
mmr.enabled=falseintoconfig.json, permanently overriding anyconfig.d/fragment.Per-field reset or drop-equal-to-default-on-write would fix.
config.json. Aftermm init --fresh,a running web instance still holds the old values in memory; a
subsequent web save can re-pin the dropped values. mtime watcher
on
config.jsonwould invalidate the cache. (The Reset block's"restart the web UI" caveat is the workaround.)
_FRESH_PRESERVE_KEYSis hardcoded and goes stale. Newcredential/endpoint/user-data fields require manual additions to
the constant; nothing enforces this at PR time. Schema-driven
alternative: Pydantic
Field(json_schema_extra={"fresh_preserve": True})or class-level
PRESERVE_ON_FRESH: ClassVar[set[str]]collectedinto the constant at import. Defer until preserve list grows or
drift incident occurs.
--freshwrite-after-backup atomicity. Ifconfig_path.write_textfails after backup succeeds, the orphan
.bak-<ts>survives nextto an unmodified original. Not data-lossy (original intact), but
the leftover file is confusing. Tempfile + atomic rename, or
delete-on-write-failure, would tighten this.
Breaking changes
None.
--freshis opt-in; defaultmm initbehaviour is unchanged.