Skip to content

feat: GET /list — query-free metadata browse by wing/room#16

Open
jphein wants to merge 2 commits intorboarescu:mainfrom
jphein:feat/list-endpoint
Open

feat: GET /list — query-free metadata browse by wing/room#16
jphein wants to merge 2 commits intorboarescu:mainfrom
jphein:feat/list-endpoint

Conversation

@jphein
Copy link
Copy Markdown
Contributor

@jphein jphein commented Apr 30, 2026

Adds a small REST endpoint that wraps mempalace_list_drawers so consumers can enumerate drawers in a wing/room without inventing a search query.

Why

/search with a wing filter quietly falls back to BM25 when the query has no embeddable content, and BM25 doesn't honour the wing filter — so a UI that wants "show me everything in wing=reflect" can't reliably get there through /search. mempalace_list_drawers already does the right thing at the metadata layer; this just exposes it over HTTP so a frontend doesn't have to talk MCP just to browse.

Shape

GET /list?wing=reflect&room=summaries&limit=20&offset=0
  • Either wing or room (or both) can be supplied; with neither, returns the first limit drawers across the whole palace, ordered by mempalace's natural metadata-table order.
  • Pagination via limit / offset — same shape as mempalace_list_drawers.
  • Authenticated via X-Api-Key header on the same code path as every other endpoint.

Why it's safe

  • Read-only — uses the read semaphore, never the write one.
  • No new mempalace dependency: mempalace_list_drawers has been in mempalace 3.x since before the daemon's first release. The endpoint just forwards arguments.
  • 34 lines of main.py and nothing else (the verify-routes.sh probe is held back because that file is being added in chore(scripts): add verify-routes.sh smoke test #9 — happy to fold it in there if you'd prefer, or send a follow-up after chore(scripts): add verify-routes.sh smoke test #9 lands).

Tested in production on the canonical 151K-drawer palace since 2026-04-26 — used by a small reflect-memories UI that needs the wing-scoped listing.

Glad to adjust the response shape, naming, or scope. Thanks for taking a look!

Wraps mempalace's mempalace_list_drawers MCP tool. Useful for any UI
that wants to enumerate drawers in a wing without supplying a search
query — for example a 'browse memories by wing' panel that needs to
list everything in wing=reflect without inventing an embeddable query.
/search with a wing filter falls back to BM25 (which ignores the
filter) when the query has no embeddable content, so a metadata-layer
browse is the right shape for that case.

Pagination via limit/offset; both wing and room are optional. With
neither, returns the first `limit` drawers in mempalace's natural
metadata-table order.

verify-routes.sh probe held back because that file is added by rboarescu#9.
Happy to fold a /list probe into rboarescu#9 or send a follow-up once rboarescu#9 lands.
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new REST convenience endpoint to allow “query-free” drawer browsing via mempalace metadata (wing/room), avoiding /search behavior when there’s no embeddable query content.

Changes:

  • Introduces GET /list with optional wing / room filters.
  • Adds pagination parameters (limit, offset) and forwards them to mempalace_list_drawers.
  • Reuses existing API-key auth path and MCP tool-call plumbing (_check_auth + _call + _unwrap).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread main.py Outdated
Comment on lines +474 to +478
wing=reflect" sorted by recency).

Either ``wing`` or ``room`` (or both) should be supplied; with neither,
returns the first ``limit`` drawers across the whole palace, ordered
by mempalace's natural metadata-table order.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already addressed in the current docstring (main.py:476-479): "Ordering is whatever mempalace_list_drawers returns — currently the natural sqlite metadata-table order, which approximates insertion order but is not guaranteed to be strictly chronological." The earlier "sorted by recency" claim was reworked when I noticed the same contradiction; sorry I never closed the comment loop. Sticking with the metadata-table-order phrasing — it's the actual contract.

jphein added a commit to jphein/palace-daemon that referenced this pull request Apr 30, 2026
…nding queue; trim patch

Filed four upstream PRs on 2026-04-30:
- rboarescu#15  feat: GET /viz status dashboard (stacks on rboarescu#13)
- rboarescu#16  feat: GET /list — query-free metadata browse
- rboarescu#17  feat: DELETE /memory + PATCH /memory
- rboarescu#18  feat(lifespan): auto-migrate Stop-hook checkpoints on startup

Also rebased PR rboarescu#13 onto upstream/main to clear a CHANGELOG conflict
left by upstream's b4aee82 (patch sync) — state went CONFLICTING ->
MERGEABLE / CLEAN.

README:
- Open upstream PRs table: four new rows (rboarescu#15-rboarescu#18) plus a 2026-04-30
  note covering today's rebase + new PRs in one breath.
- Pending PRs queue: now empty. Replaced the four stale rows
  (event-log-frame and graph-endpoint were already in flight via
  rboarescu#11 and rboarescu#13; mempal-fast.py was already upstream via the merged
  PR #4 omnibus; /viz is now PR rboarescu#15) with a brief empty-state note.

CLAUDE.md:
- Patch description trimmed to reflect that the hnsw:num_threads
  enforcement landed upstream via _pin_hnsw_threads(); only the
  log + retry-once slice remains.

patches/mcp_server_get_collection.patch:
- Regenerated against current mempalace develop. The patch is now
  just the "log exception + retry once on cache failure" change.
  Filed upstream as MemPalace/mempalace#1286; once that merges this
  patch retires.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
jphein added a commit to jphein/palace-daemon that referenced this pull request Apr 30, 2026
README:
- Intro paragraph: appended GET /list (PR rboarescu#16), DELETE/PATCH /memory
  (PR rboarescu#17), and lifespan auto-migrate (PR rboarescu#18) to "what this fork
  adds" since they're already on fork main even though the PRs are
  awaiting review upstream.
- Links bar: added docs/typescript-port-plan.md.
- Added a "Cross-repo coordination" subsection under "Recently landed
  in upstream" — calls out the local patches/ directory and the
  in-flight MemPalace/mempalace#1286 that retires it. Also references
  #1142 (RELEASING.md) for completeness.
- Requirements: dropped the stale "kind= searcher filter" justification
  for recommending the fork (kind= was retired in 1.7.1); replaced
  with daemon-strict hook mode + warnings/sqlite-fallback search path
  as the actual current reasons. Added the patches/ re-apply step.
- API table: added /list, DELETE /memory/{id}, PATCH /memory/{id}
  rows so the table reflects what the fork main actually exposes.
- Sources: cross-repo PR note for #1286.

CHANGELOG:
- Added [Unreleased] section: patch trim (Maintenance), TS port plan
  doc, hook-routing-fix.md status header, and a Docs entry covering
  today's README updates + PR rboarescu#13 rebase context.

docs/hook-routing-fix.md:
- Added a Status: SHIPPED header pointing at 62425e3 (2026-04-24
  clients/hook.py introduction) and clarifying mempal-fast.py is the
  simpler successor — same shape as docs/graph-endpoint.md's status
  header.

No daemon behaviour changes.
The previous docstring said both 'sorted by recency' and 'ordered by
mempalace's natural metadata-table order' which contradicted itself.
mempalace_list_drawers returns rows in sqlite metadata-table order —
which approximates insertion order but isn't strictly chronological,
and we shouldn't promise more than the underlying tool delivers.

Replaces the misleading recency claim with an accurate description and
calls out the pagination contract explicitly. No code change.
jphein added a commit to jphein/palace-daemon that referenced this pull request Apr 30, 2026
…boarescu#18 + #1286 to main

Same shape as 152e428 — production at disks runs fork main, so review
amendments to the in-flight upstream PRs need to ship here too.

## main.py — /list docstring (from PR rboarescu#16)

The previous docstring claimed both "sorted by recency" and "ordered
by mempalace's natural metadata-table order", which contradicted
itself. Replaced with an accurate description: ordering is whatever
mempalace_list_drawers returns (sqlite metadata-table order, which
approximates insertion order but isn't strictly chronological).
Pagination contract called out explicitly.

## main.py — PATCH /memory body validation (from PR rboarescu#17)

- await request.json() now wraps in try/except for JSONDecodeError →
  400 with a clear message (was bubbling as 500).
- Validate parsed body is a dict before reading keys.
- Reject empty patches (no content/wing/room) with 400 listing the
  valid keys, instead of forwarding {drawer_id: ...} alone to
  mempalace_update_drawer where it's an ambiguous no-op.

DELETE handler unchanged — no body parsing, nothing to harden.

## main.py — Lifespan auto-migrate ImportError narrowing (from PR rboarescu#18)

Previous `except ImportError:` swallowed transitive dep failures
inside mempalace.migrate (a real bug worth surfacing) as if they
were "feature unavailable" (expected). Distinguish via
`getattr(e, "name", None) == "mempalace.migrate"` — module genuinely
absent → debug log, anything else → warning log so it's visible.

Comment that pointed at `mempalace docs/superpowers/specs/2026-04-25-
checkpoint-collection-split.md` (a path that doesn't exist in this
repo, lives only in the jphein/mempalace fork's superpowers tree)
replaced with a public-facing reference to `mempalace repair --mode
reorganize` — the operator-driven equivalent that's actually
discoverable.

## patches/mcp_server_get_collection.patch (from MemPalace/mempalace #1286)

Regenerated against current upstream develop with the Copilot review
fix folded in: logger.error(..., str(e)) → logger.exception(...).
Now the traceback lands in the daemon log alongside the message
instead of being dropped. Patch behaviour is otherwise identical.

The upstream PR also gained TestGetCollectionRetry — three monkeypatch
tests that pin the retry semantics. Those tests live in mempalace, not
here; they'll arrive on this side automatically once #1286 merges and
the patch retires.

All four fixes are amendments on the corresponding upstream PR
branches (force-pushed earlier today). No behaviour change for the
healthy path; the changes harden failure modes that were caught in
review.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@jphein
Copy link
Copy Markdown
Contributor Author

jphein commented Apr 30, 2026

Pushed 1094a00 to address the Copilot docstring note. The previous wording claimed both "sorted by recency" and "natural metadata-table order", which contradicted itself — replaced with an accurate description (sqlite metadata-table order, approximates insertion order but not strictly chronological) and made the pagination contract explicit. No code change. Thanks for the catch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants