Skip to content

Harden palace deletion, WAL redaction, and MCP search input handling#739

Merged
igorls merged 9 commits intodevelopfrom
copilot/fix-unauthorized-data-deletion
Apr 13, 2026
Merged

Harden palace deletion, WAL redaction, and MCP search input handling#739
igorls merged 9 commits intodevelopfrom
copilot/fix-unauthorized-data-deletion

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 12, 2026

This addresses the security audit findings around destructive path handling, sensitive data leaking into the WAL, overly permissive query sanitization, and missing validation on MCP read filters. The changes tighten deletion guardrails and bring read-path validation in line with existing write-path protections.

  • Destructive operation guardrails

    • migrate and repair now require the target path to contain chroma.sqlite3 before any destructive action.
    • Both commands now prompt before replacing/removing palace data, with --yes available for explicit non-interactive confirmation.
    • repair also validates any pre-existing backup path before deleting it.
  • WAL redaction coverage

    • Expanded WAL redaction to cover actual sensitive payload fields used by tools:
      • content
      • entry
      • query
      • text
      • document
      • existing preview fields
  • Query sanitization tightening

    • Reduced MAX_QUERY_LENGTH from 500 to 250.
    • Improved trimming so long contaminated inputs preserve the likely search intent while handling wrapping quotes more precisely.
  • MCP read-side input validation

    • Applied sanitize_name-style validation to read/search filters passed through MCP tools, including wing, room, and tunnel filter inputs.
    • Invalid filter values now fail fast instead of being passed to the storage/query layer.

Example of the new destructive-path safety check:

if not contains_palace_database(palace_path):
    print(f"\n  No palace database found at {db_path}")
    return

if not confirm_destructive_action("Repair", palace_path, assume_yes=args.yes):
    return

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • chroma-onnx-models.s3.amazonaws.com
    • Triggering command: /usr/bin/python python -m pytest tests/ -v --ignore=tests/benchmarks (dns block)
    • Triggering command: /usr/bin/python python -m pytest tests/test_migrate.py tests/test_cli.py tests/test_mcp_server.py tests/test_query_sanitizer.py -v (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI linked an issue Apr 12, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Fix unauthorized data deletion in mempalace commands Harden palace deletion, WAL redaction, and MCP search input handling Apr 12, 2026
Copilot AI requested a review from igorls April 12, 2026 22:28
Copilot AI and others added 8 commits April 12, 2026 22:19
@igorls igorls force-pushed the copilot/fix-unauthorized-data-deletion branch from 1f22942 to 2232854 Compare April 13, 2026 01:20
@igorls igorls marked this pull request as ready for review April 13, 2026 01:21
Copilot AI review requested due to automatic review settings April 13, 2026 01:21
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR tightens MemPalace’s safety and validation around destructive palace operations, write-ahead-log (WAL) redaction, MCP read-side filter validation, and query sanitization to address security audit findings.

Changes:

  • Added destructive-operation guardrails for migrate/repair (palace DB presence checks + interactive confirmation with --yes).
  • Expanded WAL redaction to cover additional sensitive payload fields and added tests for redaction + MCP filter validation.
  • Tightened query sanitization (reduced max length, improved tail extraction/quote handling) with additional test coverage.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
mempalace/migrate.py Adds palace DB presence checks and confirmation prompt helpers for destructive migrations.
mempalace/cli.py Wires --yes into migrate/repair, adds safety checks before destructive repair steps.
mempalace/mcp_server.py Expands WAL redaction keys and validates optional read/search filter inputs via sanitize_name.
mempalace/query_sanitizer.py Reduces MAX_QUERY_LENGTH and refines long-query trimming (tail sentence + quote stripping).
tests/test_migrate.py New tests verifying migrate refuses non-palace paths and aborts without confirmation.
tests/test_cli.py Adds/updates repair CLI tests for palace DB gating and confirmation behavior.
tests/test_mcp_server.py Adds tests for read-side filter rejection and WAL sensitive-field redaction.
tests/test_query_sanitizer.py Adds coverage for tail-sentence extraction behavior and updated max query length.
Comments suppressed due to low confidence (1)

mempalace/query_sanitizer.py:172

  • _trim_candidate() is applied in the tail-sentence path even when seg is already within [MIN_QUERY_LENGTH, MAX_QUERY_LENGTH]. Because _strip_wrapping_quotes() can shorten the string, this can return a clean_query shorter than MIN_QUERY_LENGTH (which the docstring says should be treated as extraction failure). After trimming, re-check the length and continue scanning/fall through to tail truncation when it drops below MIN_QUERY_LENGTH.
    for seg in reversed(all_segments):
        seg = seg.strip()
        if len(seg) >= MIN_QUERY_LENGTH:
            candidate = _trim_candidate(seg)
            logger.warning(
                "Query sanitized: %d → %d chars (method=tail_sentence)",
                original_length,
                len(candidate),
            )
            return {
                "clean_query": candidate,
                "was_sanitized": True,
                "original_length": original_length,
                "clean_length": len(candidate),
                "method": "tail_sentence",
            }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread mempalace/query_sanitizer.py Outdated
Comment thread mempalace/migrate.py
Comment thread mempalace/cli.py Outdated
- query_sanitizer: require matching quote pair in _strip_wrapping_quotes
- query_sanitizer: re-check MIN_QUERY_LENGTH after trim in tail_sentence path
- migrate: neutral confirmation message accurate for both migrate and repair
- cli: os.path.normpath instead of rstrip to handle '/' root edge case
@igorls igorls merged commit 41b8601 into develop Apr 13, 2026
6 checks passed
@igorls igorls deleted the copilot/fix-unauthorized-data-deletion branch April 13, 2026 02:14
jphein added a commit to jphein/mempalace that referenced this pull request Apr 13, 2026
Upstream changes: v3.2.0 release, i18n, security hardening (MemPalace#739),
expanded README, CHANGELOG, convo_miner fixes, entity_detector typo fix,
HNSW mtime detection, full-content drawer ID hashing.

Conflict resolutions:
- pyproject.toml: keep our chromadb>=1.5.4 pin (upstream still on 0.5.0)
- CLAUDE.md: keep our fork-specific version (upstream added contributor guide)
- mcp_server.py: take upstream's improved _get_client (DB deletion detection,
  inode!=0 guard, input validation), keep our BLOB seq_id repair call
- miner.py: keep our paginated metadata (upstream added full-scan fallback)
- cli.py: keep our client release before rmtree, take upstream's normpath
- README.md: take upstream's expanded docs, keep our chromadb version
- CONTRIBUTING.md: take upstream's fork-first clone instructions

Co-Authored-By: Claude Opus 4.6 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

security_audit

3 participants