Skip to content

MCP recall/reflect: a mistyped tags_match silently widens the scope to "any" #5371

Closed

Description

@JayOfTheKeyboard

Bug Description

The MCP recall and reflect tools take tags_match: str = "any" in hindsight_api/mcp_tools.py, in both the bank-id and the session-bank copy of each. FastMCP accepts any string, and _parse_tags_match falls back to "any" for anything it doesn't recognise. "any" also admits untagged memories. So a mistyped strict mode quietly widens a recall meant to stay inside one tag, and no error comes back.

The HTTP endpoint rejects the same value with a 422, and the MCP list_memories tool already types the parameter as TagsMatch.

Two things make this easy to hit from an agent:

  • The tool schema shows tags_match as {"type": "string", "default": "any"}, with no enum.
  • Both tools' docstrings say 'any' (match any tag) or 'all' (match all tags). They never name any_strict, all_strict or exact, though the MCP docs page lists all five. An agent that wants "only this tag" has to guess the spelling.

Steps to Reproduce

On main (9269b88), with a scratch server from hindsight-system-tests (embedded Postgres, stub LLM) and the tagged_bank fixture from test_06_tag_filtering.py (one memory tagged typescript, one music, one untagged):

Call MCP recall with query "What do we know about Alice?", tags: ["typescript"], and each tags_match below.

transport tags_match result
MCP recall any_strict TypeScript fact only
MCP recall any-strict TypeScript fact + untagged fact, no error
MCP recall strict TypeScript fact + untagged fact, no error
MCP recall all_strict TypeScript fact only
MCP recall allstrict TypeScript fact + untagged fact, no error
HTTP POST .../memories/recall any-strict 422 Input should be 'any', 'all', 'any_strict', 'all_strict' or 'exact'

Expected Behavior

An unknown tags_match is rejected over MCP, the same way HTTP rejects it, and the tool schema and description list the five valid modes.

Actual Behavior

An unknown value runs as any, so untagged memories come back in a recall meant to be tag-scoped. Nothing in the response shows the mode was ignored.

Version

main @ 9269b88

Possible fix

Type the four signatures as tags_match: TagsMatch = "any", as list_memories already does, and describe all five modes in the docstrings. FastMCP then rejects an unknown mode before the engine is called. I have this on a branch with tests (12 tests; 10 fail on main, all pass with the change), in case it's useful: https://github.com/JayOfTheKeyboard/hindsight/tree/fix/mcp-recall-reflect-tags-match

Two related things I noticed but didn't measure:

Activity

  1. koriyoshi2041 commented on Oct 7, 2026

    @koriyoshi2041
    Contributor

    Confirmed the widening one layer below MCP on exact main (9269b8841): both unknown spellings produce the same SQL predicate as any, while any_strict excludes untagged rows.

    any_strict -> AND tags IS NOT NULL AND tags != '{}' AND tags && $1
    any-strict -> AND (tags IS NULL OR tags = '{}' OR tags && $1)
    strict     -> AND (tags IS NULL OR tags = '{}' OR tags && $1)
    any        -> AND (tags IS NULL OR tags = '{}' OR tags && $1)
    

    The existing focused tag semantics tests are green (7/7), so this is the parser's deliberate fallback rather than a query-builder regression.

    One scope detail for a team-owned fix: the four MCP signatures should be typed as proposed, but GET /v1/default/banks/{bank_id}/documents independently still declares tags_match: str and passes it to the same builder. That endpoint should use TagsMatch too; otherwise MCP becomes strict while a typo on document listing continues to widen. I could not reproduce the same issue on the graph routes: their public query parameters are already typed as TagsMatch on current main.

    Focused checks:

    uv run pytest tests/test_tags_visibility.py -q \
      -k 'test_tags_match_any_includes_untagged or test_tags_match_any_strict_excludes_untagged or test_strict_tags_match'
    # 7 passed
    
  2. JayOfTheKeyboard commented on Oct 10, 2026

    @JayOfTheKeyboard
    ContributorAuthor

    Thanks @koriyoshi2041 for confirming this on main and for finding the documents endpoint.

    I've rebased the fix branch onto current main and added two commits:

    • GET /v1/default/banks/{bank_id}/documents now types tags_match as TagsMatch, so any-strict or strict gets a 422 instead of a 200 that includes untagged documents: JayOfTheKeyboard@b32c4a1
    • GET /v1/default/banks/{bank_id}/graph had the same problem. On main it still declares tags_match: str (default all_strict), and get_graph_data has a comment noting that an unknown mode falls through to any. You're right that /entities/graph is typed, but the memory graph route isn't. It now gets a 422 too: JayOfTheKeyboard@5d3e721

    Each has a test that fails on main with the untagged row in the response and passes with the fix. The OpenAPI spec and the clients are regenerated, and the control plane's documents proxy narrows the value for the new TypeScript type.

    I also checked the other places tags_match comes in as a plain string. The mental model MCP tools validate it before use, list_mental_models and list_directives use a Literal, and the Python client wrapper passes it through to routes that now validate it. So I don't think anything else widens.

    Branch: https://github.com/JayOfTheKeyboard/hindsight/tree/fix/mcp-recall-reflect-tags-match

  3. koriyoshi2041 commented on Oct 10, 2026

    @koriyoshi2041
    Contributor

    Checked the rebased branch at 5d3e7219. The widened scope is now closed at each public entry point I could identify:

    • MCP recall / reflect: invalid modes fail validation before the engine; schemas expose all five modes.
    • GET /documents and GET /graph: invalid modes return 422; valid any_strict still excludes untagged documents.
    • Generated API/client changes match the two HTTP annotations, and git diff --check is clean.

    Focused results:

    tests/test_mcp_tools.py -k RecallReflectTagsMatch
    12 passed
    
    tests/test_graph_filtering.py tests/test_tags_visibility.py
    -k 'test_graph_rejects_unknown_tags_match or test_list_documents_http_any_strict_excludes_untagged or test_list_documents_http_rejects_unknown_tags_match'
    5 passed
    

    The HTTP suite initially hit my local SOCKS environment while loading the embedding model; rerunning without proxy env completed cleanly. I don't see another plain-string public ingress left that reaches the permissive parser.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcore

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions