Skip to content

Security hardening: input validation, argument whitelisting, concurrency & WAL fixes#647

Merged
bensig merged 1 commit intoMemPalace:developfrom
israads:fix/security-hardening-mcp-kg
Apr 12, 2026
Merged

Security hardening: input validation, argument whitelisting, concurrency & WAL fixes#647
bensig merged 1 commit intoMemPalace:developfrom
israads:fix/security-hardening-mcp-kg

Conversation

@israads
Copy link
Copy Markdown
Contributor

@israads israads commented Apr 11, 2026

Summary

Security hardening addressing findings from audit (ref #401). All changes are security-only — no features, no refactors, no behavior changes for valid inputs.

Findings and fixes by priority

HIGH — Inconsistent input validation

sanitize_name() was applied to write tools (tool_kg_add, tool_diary_write, tool_add_drawer) but not to their companion read/mutate tools. An MCP caller could pass unsanitized strings containing path traversal sequences or special characters to these unprotected tools:

Tool Before After
tool_kg_query Raw entity passed to _entity_id() → SQL sanitize_name() applied
tool_kg_invalidate Raw subject/predicate/object to SQL sanitize_name() applied to all three
tool_kg_timeline Raw entity to SQL sanitize_name() applied
tool_diary_read Raw agent_name into ChromaDB where clause sanitize_name() applied

Also added direction validation in tool_kg_query — previously accepted any string.

HIGH — Unfiltered argument dispatch

handle_request() passed **tool_args directly from the MCP JSON-RPC request to tool handlers with no filtering. This allowed callers to set internal parameters like added_by (default "mcp") and source_file to arbitrary values, spoofing the WAL audit trail.

Fix: Arguments are now whitelisted against the tool's declared input_schema properties before dispatch. Undeclared keys are silently dropped.

MEDIUM — SQLite concurrency (knowledge_graph.py)

KnowledgeGraph uses check_same_thread=False with a shared singleton connection but had no locking. Concurrent MCP requests could corrupt the database or produce inconsistent reads.

Fix: Added threading.Lock() around all read and write operations (add_entity, add_triple, invalidate, query_entity).

MEDIUM — WAL file permission race condition

The WAL file was written first, then chmod(0o600) was applied after — leaving a brief window where the file was world-readable on every single write.

Fix:

  • Pre-create WAL file with touch(mode=0o600) at startup
  • Use os.open() with 0o600 mode for atomic permission-safe writes
  • Eliminated the post-write chmod() call entirely

MEDIUM — Sensitive content in WAL logs

content_preview and entry_preview fields logged the first 200 characters of user content to the plaintext WAL file. This is an information disclosure risk for personal memory content.

Fix: Content preview fields are now redacted in WAL entries ([REDACTED N chars]). The WAL still logs operation type, IDs, timestamps, and metadata — sufficient for audit without exposing content.

LOW — Internal error leakage

tool_check_duplicate and tool_diary_read returned str(e) to MCP callers, potentially exposing internal file paths, database state, or stack traces.

Fix: Generic error messages returned to callers. Full exceptions logged server-side via logger.exception().

LOW — No bounds on numeric parameters

  • max_hops in tool_traverse_graph had no upper bound — could cause runaway graph traversal
  • limit in tool_search had no cap
  • last_n in tool_diary_read had no cap
  • Type coercion (int(value), float(value)) could crash on invalid strings with no error handling

Fix:

  • max_hops capped to [1, 10]
  • limit capped to [1, 50]
  • last_n capped to [1, 100]
  • Type coercion wrapped in try/except, returns JSON-RPC -32602 (Invalid params) on failure

Files changed

  • mempalace/mcp_server.py — All MCP-layer fixes (validation, whitelisting, WAL, error handling, bounds)
  • mempalace/knowledge_graph.py — Threading lock for SQLite concurrency safety

Test plan

  • All 589 existing tests pass (uv run pytest tests/ -x -q)
  • Verify tool_kg_query rejects names with path traversal characters
  • Verify tool_kg_invalidate rejects unsanitized input
  • Verify undeclared arguments (e.g. added_by: "attacker") are dropped in dispatch
  • Verify WAL file is created with 0o600 permissions from the start
  • Verify WAL entries no longer contain content previews
  • Verify tool_traverse_graph with max_hops=999 gets capped to 10
  • Verify invalid type coercion returns -32602 error instead of crashing

🤖 Generated with Claude Code

…g, concurrency safety, and WAL fixes

Addresses findings from security audit (ref MemPalace#401): inconsistent sanitization across MCP tools,
unfiltered argument dispatch allowing audit trail spoofing, SQLite concurrency issues, WAL
permission race condition, sensitive content in logs, and internal error leakage to MCP callers.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@bensig bensig changed the base branch from main to develop April 11, 2026 22:21
@bensig bensig merged commit 58eca50 into MemPalace:develop Apr 12, 2026
6 checks passed
bensig added a commit that referenced this pull request Apr 12, 2026
- WAL write: keep os.open atomic approach from #647
- tool_search: keep _MAX_RESULTS + min_similarity compat from #635
- schema_props filter: remove duplicate (already present above conflict)
jphein added a commit to jphein/mempalace that referenced this pull request Apr 12, 2026
… path

diary_write() and diary_read() accept optional wing parameter.
Stop hook derives project wing from transcript path instead of
hardcoding wing_session-hook. Preserves MemPalace#647's sanitize_name
validation and last_n clamping.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
bensig added a commit that referenced this pull request Apr 12, 2026
1. min_similarity backwards-compat: convert similarity to distance scale
   (1.0 - similarity) instead of passing raw value as max_distance
2. Restore structured error reporting (error + partial fields) in
   tool_status, tool_list_wings, tool_list_rooms, tool_get_taxonomy
   — reverts silent except:pass that dropped #647 security hardening
3. inode cache: remove falsy-zero short-circuit so missing DB file
   triggers reconnect instead of reusing stale client
4. _fetch_all_metadata: check for empty batch before extending/advancing
   offset to prevent infinite loop on concurrent deletion
5. KG initialization: only override path when --palace is explicit;
   default runs use KnowledgeGraph's built-in default path

Co-authored-by: jphein <[email protected]>
bensig added a commit that referenced this pull request Apr 12, 2026
…esolves #635) (#667)

* feat: MCP reliability — inode detection, WAL rotation, metadata cache, search limits

Infrastructure hardening for the MCP server:
- Detect palace DB replacement via inode tracking (repair command support)
- WAL rotation to prevent unbounded WAL growth
- _fetch_all_metadata() + _get_cached_metadata() with 60s TTL for taxonomy/status
- _MAX_RESULTS cap (100) with limit clamping [1, _MAX_RESULTS]
- max_distance parameter for similarity threshold in search
- Handle all notifications/* methods, null arguments, method=None
- Remove duplicate _client_cache = None declarations
- searcher.py max_distance parameter passthrough

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: new MCP tools (get/list/update drawer, hook settings, memories filed), export, normalize

New MCP tools:
- mempalace_get_drawer: fetch single drawer by ID with full content
- mempalace_list_drawers: paginated listing with wing/room filter
- mempalace_update_drawer: update content/wing/room on existing drawers
- mempalace_hook_settings: get/set hook behavior (silent_save, desktop_toast)
- mempalace_memories_filed_away: check latest checkpoint status

Also includes:
- exporter.py: export palace as browsable markdown files
- normalize.py: tool_use/tool_result capture for richer transcript mining
- layers.py: updated for new tool integration
- config.py: hook settings properties (hook_silent_save, hook_desktop_toast)

Depends on PR 3 (reliability) for _MAX_RESULTS, _metadata_cache, WAL logging.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: normalize.py handles string messages and Read offset type mismatch

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: params null guard, L2→cosine docs, empty tool_use_map key guard

- Handle explicit null in MCP params (request.get("params") or {})
- Fix search tool description: L2 → cosine distance (collection uses hnsw:space=cosine)
- Guard against empty string key in tool_use_map from malformed JSONL entries

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: rename ambiguous var 'l' to 'line' (E741 lint)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address code review findings (5 issues)

1. min_similarity backwards-compat: convert similarity to distance scale
   (1.0 - similarity) instead of passing raw value as max_distance
2. Restore structured error reporting (error + partial fields) in
   tool_status, tool_list_wings, tool_list_rooms, tool_get_taxonomy
   — reverts silent except:pass that dropped #647 security hardening
3. inode cache: remove falsy-zero short-circuit so missing DB file
   triggers reconnect instead of reusing stale client
4. _fetch_all_metadata: check for empty batch before extending/advancing
   offset to prevent infinite loop on concurrent deletion
5. KG initialization: only override path when --palace is explicit;
   default runs use KnowledgeGraph's built-in default path

Co-authored-by: jphein <[email protected]>

---------

Co-authored-by: jp <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: jphein <[email protected]>
skuznetsov pushed a commit to skuznetsov/mempalace that referenced this pull request Apr 12, 2026
Merge origin/develop into codex/pg-backend-stage2 after upstream PRs MemPalace#647 and MemPalace#667 landed.

Resolve the MCP server conflict by preserving the new cached metadata helpers, WAL hardening, drawer update/list tools, and exporter changes from develop while keeping the PostgreSQL backend abstraction: MCP collection access now goes through palace.get_collection(...) instead of constructing a Chroma PersistentClient directly.

Keep the large-table optimization for PostgreSQL: metadata pagination no longer performs an exact col.count() pre-pass, MCP status derives total_drawers from fetched metadata rows, and MemoryStack.status() can use backend-provided estimated_count() so PostgreSQL can use catalog/statistics estimates rather than SELECT COUNT(*) on status paths.

Also make tool_update_drawer backend-neutral by using upsert() instead of the Chroma-only update() method, with tests covering the estimated count path.

Verification: uv run pytest -q (671 passed, 106 deselected); uv run ruff check .; git diff --check; git diff --cached --check; precise conflict-marker scan.
jphein added a commit to jphein/mempalace that referenced this pull request Apr 12, 2026
The schema-based argument filter (from MemPalace#647) strips all kwargs not
declared in input_schema. This breaks handlers that accept **kwargs
for pass-through to ChromaDB or other backends.

Add inspect.Parameter.VAR_KEYWORD check before filtering — handlers
with **kwargs receive all arguments unfiltered.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
jphein added a commit to jphein/mempalace that referenced this pull request Apr 12, 2026
The schema-based argument filter (from MemPalace#647) strips all kwargs not
declared in input_schema. This breaks handlers that accept **kwargs
for pass-through to ChromaDB or other backends.

Add inspect.Parameter.VAR_KEYWORD check before filtering — handlers
with **kwargs receive all arguments unfiltered.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
bensig pushed a commit that referenced this pull request Apr 12, 2026
* fix: skip arg whitelist for handlers accepting **kwargs (#572)

The schema-based argument filter (from #647) strips all kwargs not
declared in input_schema. This breaks handlers that accept **kwargs
for pass-through to ChromaDB or other backends.

Add inspect.Parameter.VAR_KEYWORD check before filtering — handlers
with **kwargs receive all arguments unfiltered.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: guard inspect.signature failure, default to filtering

Wrap inspect.signature() in try/except — on failure, default to
filtering (safe fallback). Addresses Copilot feedback on fragility.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
@igorls igorls mentioned this pull request Apr 13, 2026
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants