Skip to content

fix(config): coerce list[BaseSettings] on mutation path#257

Merged
memtomem merged 1 commit intomainfrom
fix/coerce-nested-basemodel
Apr 18, 2026
Merged

fix(config): coerce list[BaseSettings] on mutation path#257
memtomem merged 1 commit intomainfrom
fix/coerce-nested-basemodel

Conversation

@memtomem
Copy link
Copy Markdown
Owner

Summary

PR #253 introduced NamespacePolicyRule as memtomem's first list[BaseSettings] config field. The load path was wired to handle it correctly (via _list_item_type + model_validate in load_config_d and _dedup_key), but the mutation path was left asymmetric — coerce_and_validate, PATCH /api/config, mm config set, and the init wizard all silently passed raw dicts through. Downstream code (indexing/engine.py:121) accesses rule.path_glob on each entry and would AttributeError on a dict.

Adjacent to PR #256 (save-path drop-default); together #253#256 → this PR complete the config-layer round-trip.

Root cause

PR #253 added the new list[BaseSettings] shape but only fixed the read side of the config pipeline. The write side — the three canonical mutation entry points plus the init wizard — kept validating only primitive lists. A user PATCH-ing namespace.rules or running mm config set namespace.rules '[...]' saw no validation error, but the runtime config held dicts where model instances were expected.

What this does

1. coerce_and_validate handles list[BaseSettings]

New branch inside the existing expected_type is list path, gated by:

if isinstance(item_type, type) and issubclass(item_type, BaseSettings):

The isinstance(item_type, type) guard is deliberate — it protects against Union, generic aliases, or other non-class item types that would TypeError inside issubclass. Existing primitive-list callers (search.rrf_weights, importance.weights) take the else path unchanged; all their existing tests still pass.

Inside the branch: JSON strings are parsed (CLI path), lists of dicts are model_validate-ed (Web UI path), already-instance entries pass through, scalars/malformed shapes raise ValueError that propagates as a rejected entry in PATCH or a non-zero CLI exit.

2. MUTABLE_FIELDS["namespace"] + FIELD_CONSTRAINTS["namespace.rules"]

Registers the field at both gates so PATCH /api/config no longer reports namespace.rules: read-only field and mm config set no longer prints not a mutable field. Frontend impact check: grep FIELD_CONSTRAINTS packages/memtomem/src/memtomem/web/static/ → zero hits. UI rendering is unaffected.

3. Unified _json_default JSON serializer

save_config_overrides previously used json.dumps(..., default=str), which would emit BaseSettings.__repr__ for any nested model — a latent round-trip bug. Fixed with _json_default that returns model_dump(mode="json") for BaseSettings and str() for Path. Scope audit via grep "default=str\|default=_json_default" packages/memtomem/src/ confirmed:

  • config.py save_config_overrides → fixed (uses _json_default)
  • cli/init_cmd.py:706 wizard write → also fixed (imports + uses _json_default) for consistency across all config-write paths
  • 7 other default=str call sites (chunking/structured.py, cli/watchdog_cmd.py, cli/config_cmd.py show, server/scheduler.py, server/health_store.py, server/webhooks.py, server/tools/consolidation.py) all serialize non-config payloads (scheduler state, webhook bodies, health snapshots, etc.) — unrelated to BaseSettings config models, intentionally left alone.

Though namespace.rules is currently the only list[BaseSettings] field in play, unifying the helper prevents a future nested model from silently re-introducing the same latent bug.

4. Migration safety for pre-fix installations

Any config.json written by a pre-fix wizard may contain repr-string leftovers for namespace.rules. On load:

  • The entry hits coerce_and_validate via FIELD_CONSTRAINTS["namespace.rules"].
  • Malformed shape (scalar string, list-of-strings) → ValueError.
  • load_config_overrides catches the ValueError, logs a warning, skips the key.
  • The field falls back to its default []. No crash, no data loss beyond the already-corrupt entry.

Test: test_legacy_repr_string_in_config_handled_gracefully covers both scalar-value and list-of-strings cases.

Testing

  • 1694 tests pass (+14 new), ruff + format + mypy clean.
  • New tests in test_cli.py::TestCoerceAndValidate:
    • test_namespace_rules_from_json_string — CLI path
    • test_namespace_rules_from_list_of_dicts_pr253_regression — Web PATCH path; docstring pins the engine.py:121 failure mode that originally motivated this fix
    • test_namespace_rules_passthrough_for_model_instances
    • test_namespace_rules_empty_list
    • test_namespace_rules_rejects_malformed_json
    • test_namespace_rules_rejects_non_list_json
    • test_namespace_rules_rejects_scalar_entry
    • test_namespace_rules_propagates_model_validation_error
  • New tests in test_cli.py::TestSaveConfigOverrides:
    • test_legacy_repr_string_in_config_handled_gracefully — migration safety
    • test_namespace_rules_round_trip — save → load returns validated instances (not dicts)
  • New tests in test_cli.py::TestConfigCLI:
    • test_config_set_namespace_rules_json — end-to-end CLI persistence + reload
    • test_config_set_namespace_rules_rejects_malformed
  • New tests in test_web_routes_extended.py::TestConfigPatch:
    • test_patch_namespace_rules — PATCH apply + runtime type check
    • test_patch_namespace_rules_validation_error — bad rule lands in rejected
  • Manual smoke (4-step): mm config set namespace.rules '[...]' → reload produces NamespacePolicyRule instances → _FRESH_PRESERVE_KEYS contains namespace.rules (PR cli(init): add --fresh to drop wizard-untouched config leftovers #255 preserve list) → subsequent save_config_overrides keeps the rules (non-default, so not dropped by PR fix(config): drop default-valued fields on save to prevent silent leftover pins #256 logic).

Unblocks

  • Web UI namespace rules editor (designed in project_list_basesettings_config_gap.md)
  • mm init wizard rule presets
  • Any future list[BaseSettings] config field

Test plan

  • uv run pytest -m "not ollama" — 1694 passed
  • uv run ruff check + ruff format --check — clean
  • uv run mypy packages/memtomem/src — 0 issues
  • Manual smoke: mm config set namespace.rules '[...]' round-trip, reload produces model instances
  • Manual smoke: _FRESH_PRESERVE_KEYS contains namespace.rules (interop with PR cli(init): add --fresh to drop wizard-untouched config leftovers #255)

PR #253 introduced NamespacePolicyRule as the first list[BaseSettings]
field. Load path handled it correctly via _list_item_type +
model_validate, but mutation path (coerce_and_validate, PATCH /api/config,
mm config set, init wizard) silently passed raw dicts through, causing
AttributeError on rule.path_glob in engine.py:121.

This PR mirrors the load-path pattern on mutation, unifies _json_default
across all config-write sites (fixing a latent repr-string round-trip),
and adds regression guards for both the PR #253 failure mode and legacy
config.json recovery.

Refs: PR #253, #256 (adjacent save-path fix)
@memtomem memtomem merged commit e2ba1e5 into main Apr 18, 2026
7 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 18, 2026
memtomem pushed a commit that referenced this pull request Apr 18, 2026
Backfills 8 PRs (#253, #254, #255, #256, #257, #258, #259, #262)
merged to main between v0.1.9 and 0.1.10 release prep (#263) without
CHANGELOG entries. Entries land in the next release, not 0.1.10.

- Added: #253 (NamespacePolicyRule), #254 (wizard Preserved summary),
  #255 (mm init --fresh), #259 (mm config unset)
- Changed: #256 (save drops default-valued fields)
- Fixed: #257 (list[BaseSettings] mutation coercion), #258 (fragment/
  env drag-in on save), #262 (atomic config.json writes)
memtomem added a commit that referenced this pull request Apr 18, 2026
Backfills 8 PRs (#253, #254, #255, #256, #257, #258, #259, #262)
merged to main between v0.1.9 and 0.1.10 release prep (#263) without
CHANGELOG entries. Entries land in the next release, not 0.1.10.

- Added: #253 (NamespacePolicyRule), #254 (wizard Preserved summary),
  #255 (mm init --fresh), #259 (mm config unset)
- Changed: #256 (save drops default-valued fields)
- Fixed: #257 (list[BaseSettings] mutation coercion), #258 (fragment/
  env drag-in on save), #262 (atomic config.json writes)

Co-authored-by: pandas-studio <[email protected]>
@memtomem memtomem deleted the fix/coerce-nested-basemodel branch April 20, 2026 14:56
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants