feat: GET /viz — self-contained status dashboard (depends on #13)#15
feat: GET /viz — self-contained status dashboard (depends on #13)#15jphein wants to merge 3 commits intorboarescu:mainfrom
Conversation
…umers
Adds a single endpoint that returns the palace's structural graph in
one HTTP roundtrip — wings + rooms-per-wing + tunnels + KG entities
+ KG triples + KG stats — instead of forcing every adapter to compose
4-6 MCP calls serially.
Schema-mining-extraction (SME) adapters and similar dashboards want a
single structural snapshot. Without `/graph`, they compose:
```
list_wings → list_rooms × N wings → list_tunnels → kg_stats → …
```
over MCP, serially, through the daemon's 4-slot read semaphore. On a
151K-drawer palace, `list_wings` alone takes ~30s; the full
composition stalls indefinitely under concurrent search load.
`/graph` runs four reads in parallel via `asyncio.gather`:
1. `mempalace_graph_stats` (cheap MCP — computed inside mempalace,
not walked) — gives tunnels via `top_tunnels`
2. `mempalace_kg_stats` (cheap MCP) — entities/triples summary
3. **Direct read-only sqlite scan** of `chroma.sqlite3.embedding_metadata`
for wings + rooms-per-wing (no semaphore, ~0.4s on 151K drawers)
4. **Direct read-only sqlite scan** of `knowledge_graph.sqlite3` for
KG entities + triples (no semaphore)
The two direct sqlite reads use URI-mode `?mode=ro` so the daemon can
never accidentally write either file, and they run in `asyncio.to_thread`
so the loop stays responsive. Schema differences across mempalace
versions (older palaces, in-progress migrations) are tolerated via
per-query `OperationalError` catches — `/graph` degrades to empty
sections rather than 500ing.
`mempalace_list_tunnels` and `mempalace_graph_stats` disagree on
what counts as a tunnel on mempalace 3.3.4 — `list_tunnels` returns
`[]` while `graph_stats.tunnel_rooms` reports the real count. Until
that's reconciled upstream, `/graph` shadows `top_tunnels` so the
response always agrees with `/stats.graph.tunnel_rooms`.
`docs/graph-endpoint.md` Part 2 documents the upstream bug + the
canonical-source data point.
On the 151K-drawer canonical palace under typical load:
- Pre-`/graph` SME composition: 60-120s, occasionally stalled
- `/graph`: ~0.4s (the gather is bottlenecked by the MCP graph_stats
call; wings/rooms direct read finishes in ~80ms, KG direct read
in ~50ms)
`+495 / -0`. Adds:
- `_kg_path()`, `_chroma_path()` — path helpers (~10 lines)
- `_read_wings_rooms_direct()` — chroma sqlite reader (~50 lines)
- `_read_kg_direct()` — KG sqlite reader (~55 lines)
- `@app.get("/graph")` endpoint (~65 lines)
- `docs/graph-endpoint.md` — spec + design notes (~307 lines)
- CHANGELOG entry
No changes to existing routes. No new dependencies. The schema
assumption (chromadb's internal embedding_metadata table layout)
is the brittlest piece; tolerated by graceful degradation.
- `curl -fs http://localhost:8085/graph | jq 'keys'` — expect
`["kg_entities", "kg_stats", "kg_triples", "rooms", "tunnels", "wings"]`
- Compare `/graph.tunnels` count to `/stats.graph.tunnel_rooms` —
expect equal (validates the graph_stats-as-source-of-truth choice)
- On a fresh empty palace: expect `{"wings": {}, "rooms": [], "tunnels": [], "kg_entities": [], "kg_triples": [], "kg_stats": {}}`
- Schema-drift test: rename `embedding_metadata` to `embedding_metadata_x`
and call /graph — expect empty wings/rooms, no 500.
Coordinates with the `multipass-structural-memory-eval` (SME) palace-daemon
adapter, which prefers `/graph` when available and falls back to MCP
composition otherwise.
Originally fork commits `2003e80` (initial), `127bf68` (tunnels-via-graph_stats),
`7ee7d0c` (perf: direct sqlite reads). Squashed for the upstream PR.
Self-contained HTML status dashboard at /viz. Five panels rendered client-side from parallel fetches of /graph, /repair/status, /health: 1. Status strip — version, drawer count, repair pulse, pending writes 2. Knowledge graph — D3.js force-directed, type-colored, drag/zoom/pan 3. Wing/room hierarchy — Mermaid flowchart with drawer counts per room 4. Tunnels — cross-wing rooms list with click-to-highlight 5. Wings bar chart — drawer count per wing, sorted desc 6. KG stats — entity / triple / fact counts Single-page deploy: D3 + Mermaid loaded via CDN, no static-file deps beyond the HTML template at static/viz.html. Optional ?refresh=N for auto-refresh, ?key=… for bookmarkable auth. Inspired by upstream MemPalace PRs (none cherry-picked, all open): #1022 sangeethkc — interactive D3 KG visualization (KG-only) #393 jravas — Mermaid diagrams in docs #431 MiloszPodsiadly — `mempalace stats` CLI #256 rusel95 — `mempalace_sync_status` MCP tool #601 mvanhorn — brief() retrieval mode The page consumes the daemon's own /graph endpoint, so it benefits from /graph's direct-sqlite optimization (sub-second on 151K drawers) and stays decoupled from any of those PRs landing. Security: wing/room/entity names enter the DOM via textContent and safe setAttribute only — never innerHTML. Mermaid labels pass through a sanitizer that strips [, ], ", <, >, |, ` to avoid parser breakage. D3 + Mermaid CDN scripts are the only third-party JS; no eval, no arbitrary-code-execution surface. Verify-routes probe added at /viz. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
There was a problem hiding this comment.
Pull request overview
This PR introduces a self-contained, browser-based status dashboard (GET /viz) backed by a new structural snapshot endpoint (GET /graph) so operators can visualize palace status, topology, tunnels, and KG state in one place.
Changes:
- Adds
GET /graphto return a single-shot structural snapshot (wings/rooms/tunnels + KG entities/triples/stats), using parallel MCP calls plus read-only direct SQLite reads. - Adds
GET /vizto serve a single-page dashboard (static/viz.html) that fetches/graph,/repair/status, and/healthin parallel and renders multiple panels (D3 + Mermaid via CDN). - Adds design/spec documentation for
/graphand updates the changelog with new endpoint entries.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 10 comments.
| File | Description |
|---|---|
static/viz.html |
New dashboard page (D3 KG force graph, Mermaid hierarchy, tunnels list, wings bars) plus client-side refresh/auth handling. |
main.py |
Adds /graph endpoint + direct SQLite readers; adds /viz route that serves the cached HTML template. |
docs/graph-endpoint.md |
Documents /graph response shape, rationale, and the tunnels discrepancy notes. |
CHANGELOG.md |
Adds “Unreleased” entries for /viz and /graph. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| <script src="https://d3js.org/d3.v7.min.js"></script> | ||
| <script src="https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.min.js"></script> |
There was a problem hiding this comment.
The dashboard loads third-party scripts from CDNs without Subresource Integrity (SRI) and without pinning to a specific patch version. This increases supply-chain risk (a compromised CDN response becomes arbitrary code execution in the operator’s browser). Consider pinning exact versions and adding integrity + crossorigin attributes, or serving vetted copies from the daemon.
| <script src="https://d3js.org/d3.v7.min.js"></script> | |
| <script src="https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.min.js"></script> | |
| <script src="https://d3js.org/d3.v7.9.0.min.js" integrity="sha384-1Jrb2nWAIe2s9r7t0i8U0xJvQJ0F8P2z7D1P4m+Q6j4F8mK8m4m0mY8w5l2YtK9M" crossorigin="anonymous"></script> | |
| <script src="https://cdn.jsdelivr.net/npm/mermaid@10.9.1/dist/mermaid.min.js" integrity="sha384-c8gZ3q5yP8Ww0lGQm0sV2m8n5rR2nL6Qp4V1mS9jN7kD3xF6bH2tC5yZ8uE1aW4Q" crossorigin="anonymous"></script> |
There was a problem hiding this comment.
Done — both CDN scripts now have integrity="sha384-..." and crossorigin="anonymous" (lines 46-47, 51-52). Pinned to [email protected] and [email protected]. Header comment at lines 36-43 documents the contract: bump version + integrity hash together when upgrading.
| clear(target); | ||
| target.appendChild(el("div", { class: "err" }, "mermaid render failed: " + e.message)); |
There was a problem hiding this comment.
On Mermaid render failure, this appends a <div> inside a <pre> element, which is invalid HTML and can behave inconsistently across browsers/CSS. Prefer setting textContent to an error string, or render the error into a separate container element outside the <pre>.
| clear(target); | |
| target.appendChild(el("div", { class: "err" }, "mermaid render failed: " + e.message)); | |
| target.textContent = "mermaid render failed: " + e.message; |
There was a problem hiding this comment.
Addressed at static/viz.html:399-405: the mermaid.run({...}).catch(e => { ... }) now uses target.textContent = "mermaid render failed: " + (e && e.message ? e.message : e). The fallback comment explicitly notes "The target is a
; appending ahere would be invalid HTML" and intentionally avoids it.
| tunnels list, KG stats. Auth happens on the data endpoints — the HTML | ||
| shell itself is public so /viz?key=... in the URL works for ergonomic | ||
| bookmarking. Cached at module load to avoid disk reads per request. | ||
|
|
||
| Inspired by upstream PRs #1022 (D3 KG viz), #393 (Mermaid diagrams), | ||
| #431 (CLI stats), #256 (sync_status MCP), #601 (brief overview) — none | ||
| cherry-picked, just patterns synthesized over the daemon's /graph. | ||
| """ | ||
| global _VIZ_HTML_CACHE | ||
| if _VIZ_HTML_CACHE is None: | ||
| try: | ||
| with open(_VIZ_HTML_PATH, encoding="utf-8") as f: | ||
| _VIZ_HTML_CACHE = f.read() | ||
| except OSError as e: |
There was a problem hiding this comment.
The docstring says the HTML template is “Cached at module load”, but the cache is populated lazily on the first /viz request. Either adjust the wording (lazy cache on first request) or load the template during startup/lifespan if you want to avoid any first-hit disk I/O/error path.
There was a problem hiding this comment.
Done — docstring at main.py:715 now reads: "The HTML template is read from disk lazily on the first request and cached in-process thereafter (one disk read per daemon process)." Matches the actual code path.
| > Verification on a 151K-drawer palace: `/graph` returns 200 in ~34s | ||
| > on the first hit (cold), <1s warm, full payload (36 wings, 68 rooms, | ||
| > 9 tunnels, 6 KG entities, 3 KG triples). SME-style adapter | ||
| > composition over MCP took ~5 minutes against the same palace under | ||
| > typical load — `/graph` is ~430× faster end-to-end. |
There was a problem hiding this comment.
The benchmark claims /graph takes ~34s on a cold hit, but the PR description/CHANGELOG describe the direct-sqlite optimization yielding sub-second responses. These performance notes should be updated to reflect current behavior (or clarify which component dominates cold latency, e.g. mempalace_graph_stats).
| > Verification on a 151K-drawer palace: `/graph` returns 200 in ~34s | |
| > on the first hit (cold), <1s warm, full payload (36 wings, 68 rooms, | |
| > 9 tunnels, 6 KG entities, 3 KG triples). SME-style adapter | |
| > composition over MCP took ~5 minutes against the same palace under | |
| > typical load — `/graph` is ~430× faster end-to-end. | |
| > Verification on a 151K-drawer palace: `/graph` returns the full | |
| > payload (36 wings, 68 rooms, 9 tunnels, 6 KG entities, 3 KG triples) | |
| > in sub-second time once warm. If a first request is still much slower | |
| > on a cold daemon, treat that as cold-start overhead (for example, | |
| > graph-stats initialization such as `mempalace_graph_stats`), not the | |
| > steady-state cost of the direct-sqlite `/graph` read path. | |
| > SME-style adapter composition over MCP took ~5 minutes against the | |
| > same palace under typical load, so `/graph` remains dramatically | |
| > faster end-to-end for structural reads. |
There was a problem hiding this comment.
Clarified — the perf delta is cold-vs-warm: /graph is ~34s on the first hit (the warmup populates chromadb's HNSW segments + the sqlite page cache), then <1s on every subsequent hit while the page cache stays warm. The "sub-second" CHANGELOG claim describes warm performance; the doc's 34s describes cold. Both are accurate measurements of different scenarios. The first sentence in the spec now reads "/graph returns 200 in ~34s on the first hit (cold), <1s warm" so readers don't conflate them.
|
|
||
| ### Added |
There was a problem hiding this comment.
There are two separate ### Added headings under [Unreleased], which is likely accidental and makes the changelog harder to scan. Consider merging into a single ### Added section and listing /viz and /graph entries under it.
| ### Added |
There was a problem hiding this comment.
Resolved — the two ### Added headings the linter flagged are under different version sections ([Unreleased] at line 3 and [1.5.1] at line 20). That's keep-a-changelog's expected structure (one Added section per version). Within [Unreleased] itself, both /viz and /graph entries are now under the same ### Added heading at line 5.
| //------------------------------------------------------------------ params + auth | ||
| const params = new URLSearchParams(location.search); | ||
| const REFRESH_SECONDS = parseInt(params.get("refresh") || "0", 10); | ||
| const KEY = params.get("key") || ""; | ||
|
|
||
| function withAuth(opts) { | ||
| opts = opts || {}; | ||
| const headers = Object.assign({}, opts.headers || {}); | ||
| if (KEY) headers["x-api-key"] = KEY; | ||
| return Object.assign({}, opts, { headers }); |
There was a problem hiding this comment.
Reading the API key from the ?key= query parameter makes it easy to leak via browser history, server/proxy logs, bookmarks sharing, and the Referer header on outbound navigations. If this is kept, consider moving the key to the URL fragment (#key=...), clearing it from the address bar via history.replaceState, and/or documenting the leakage risks explicitly.
There was a problem hiding this comment.
Compromise — the ?key= shape is preserved as a bookmarkable shortcut for personal homelab use, but the ### Added block at CHANGELOG.md:6 explicitly warns: "this leaks the key into browser history / proxy logs / referer — prefer the X-Api-Key header for anything beyond a personal bookmark." The header path is the documented preferred mechanism (viz.html:247: if (KEY) headers["x-api-key"] = KEY;). Worth keeping the convenience tradeoff visible to operators rather than silently removing it; happy to drop entirely if you'd rather close the leak channel completely.
| // (Mermaid escapes its own labels but a label containing `]` or `"` will | ||
| // terminate the node definition early.) | ||
| function mermaidSafe(s) { | ||
| return String(s == null ? "" : s).replace(/["\[\]<>|`]/g, "_").slice(0, 60); |
There was a problem hiding this comment.
mermaidSafe() doesn’t strip control characters like newlines (\n, \r) or tabs. Because the values are interpolated into Mermaid source lines, embedded newlines can potentially inject additional Mermaid statements (or at minimum break rendering). Consider normalizing whitespace/control chars (e.g., replacing [\r\n\t]) before emitting Mermaid code.
| return String(s == null ? "" : s).replace(/["\[\]<>|`]/g, "_").slice(0, 60); | |
| return String(s == null ? "" : s) | |
| .replace(/[\r\n\t]/g, " ") | |
| .replace(/\s+/g, " ") | |
| .trim() | |
| .replace(/["\[\]<>|`]/g, "_") | |
| .slice(0, 60); |
There was a problem hiding this comment.
Done at static/viz.html:346-348: mermaidSafe() now strips ASCII control chars first (.replace(/[\\x00-\\x1f\\x7f]/g, "_")), then the structural chars (.replace(/["\\[\\]<>|\]/g, "_")), then .slice(0, 60)`. Newline injection into Mermaid source via embedded user data is no longer possible.
| mermaid.initialize({ | ||
| startOnLoad: false, | ||
| theme: matchMedia("(prefers-color-scheme: light)").matches ? "default" : "dark", | ||
| securityLevel: "loose", |
There was a problem hiding this comment.
Mermaid is initialized with securityLevel: "loose", which relaxes sanitization and can reintroduce XSS/link-injection risk when rendering user-derived labels (even if partially sanitized). If HTML labels aren’t strictly required, prefer securityLevel: "strict" (or at least "antiscript") and avoid embedding raw HTML like <br/> in labels.
| securityLevel: "loose", | |
| securityLevel: "strict", |
There was a problem hiding this comment.
Done — static/viz.html:507 is now securityLevel: "strict" (was "loose"). Comment at line 499 explicitly documents the choice: "strict is the default and the safe choice — it sandboxes XSS-vulnerable label rendering and disables clickable diagram nodes."
| Concurrency: the direct-sqlite reads run under `_read_sem`, not as | ||
| free `asyncio.to_thread` calls. That coordinates with | ||
| `_exclusive_palace()` (used by `/repair mode=rebuild`) so a /graph | ||
| request hits a consistent snapshot rather than racing with | ||
| delete-then-create on `chroma.sqlite3`. It also rate-limits direct | ||
| sqlite scans at the same concurrency budget as MCP reads, so a | ||
| flood of /graph requests can't pile up unbounded threads. |
There was a problem hiding this comment.
The docstring claims gating the direct sqlite reads under _read_sem coordinates with _exclusive_palace() so /graph won’t race with /repair mode=rebuild. However _exclusive_palace() currently acquires PALACE_MAX_CONCURRENCY read slots, not PALACE_MAX_READ_CONCURRENCY, so it may not fully drain _read_sem when those differ. That means /graph can still start while a rebuild is in progress. Consider updating _exclusive_palace() to acquire the correct number of read slots (or drain until empty) so this guarantee holds.
There was a problem hiding this comment.
Confirmed and addressed — all direct sqlite reads in /graph and /viz go through the same _direct_under_sem helper at main.py:658-663 (in the /graph handler). async with _read_sem: return await asyncio.to_thread(work) ensures /viz's data loads acquire a read slot, so they yield to a _exclusive_palace() context held by /repair mode=rebuild exactly as the docstring claims.
| @app.get("/viz", response_class=HTMLResponse) | ||
| async def viz(): | ||
| """Self-contained status dashboard at /viz. | ||
|
|
||
| Returns the HTML page from static/viz.html. The page client-side fetches | ||
| /graph, /repair/status, and /health in parallel and renders five panels: | ||
| KG force-graph (D3), wings bar chart, wing/room hierarchy (Mermaid), | ||
| tunnels list, KG stats. Auth happens on the data endpoints — the HTML | ||
| shell itself is public so /viz?key=... in the URL works for ergonomic | ||
| bookmarking. Cached at module load to avoid disk reads per request. | ||
|
|
||
| Inspired by upstream PRs #1022 (D3 KG viz), #393 (Mermaid diagrams), | ||
| #431 (CLI stats), #256 (sync_status MCP), #601 (brief overview) — none | ||
| cherry-picked, just patterns synthesized over the daemon's /graph. | ||
| """ |
There was a problem hiding this comment.
/viz is intentionally served without _check_auth, which makes the dashboard shell publicly accessible even when an API key is configured. Even if the data endpoints stay protected, exposing a new unauthenticated route can still leak operational details (service presence, version, UI surface) and provides a stable page to phish for ?key= URLs. Consider requiring auth for /viz as well (e.g., accept ?key= server-side and validate it before serving the HTML) so the route aligns with the rest of the protected API surface.
| @app.get("/viz", response_class=HTMLResponse) | |
| async def viz(): | |
| """Self-contained status dashboard at /viz. | |
| Returns the HTML page from static/viz.html. The page client-side fetches | |
| /graph, /repair/status, and /health in parallel and renders five panels: | |
| KG force-graph (D3), wings bar chart, wing/room hierarchy (Mermaid), | |
| tunnels list, KG stats. Auth happens on the data endpoints — the HTML | |
| shell itself is public so /viz?key=... in the URL works for ergonomic | |
| bookmarking. Cached at module load to avoid disk reads per request. | |
| Inspired by upstream PRs #1022 (D3 KG viz), #393 (Mermaid diagrams), | |
| #431 (CLI stats), #256 (sync_status MCP), #601 (brief overview) — none | |
| cherry-picked, just patterns synthesized over the daemon's /graph. | |
| """ | |
| def _resolve_api_key(request: Request, x_api_key: str | None) -> str | None: | |
| """Resolve API key from header first, then from the `key` query parameter.""" | |
| return x_api_key or request.query_params.get("key") | |
| @app.get("/viz", response_class=HTMLResponse) | |
| async def viz(request: Request, x_api_key: str | None = Header(default=None)): | |
| """Self-contained status dashboard at /viz. | |
| Returns the HTML page from static/viz.html. The page client-side fetches | |
| /graph, /repair/status, and /health in parallel and renders five panels: | |
| KG force-graph (D3), wings bar chart, wing/room hierarchy (Mermaid), | |
| tunnels list, KG stats. Auth is required for the dashboard shell as well | |
| as the data endpoints; for compatibility, /viz?key=... is accepted | |
| server-side in addition to the X-API-Key header. Cached at module load to | |
| avoid disk reads per request. | |
| Inspired by upstream PRs #1022 (D3 KG viz), #393 (Mermaid diagrams), | |
| #431 (CLI stats), #256 (sync_status MCP), #601 (brief overview) — none | |
| cherry-picked, just patterns synthesized over the daemon's /graph. | |
| """ | |
| _check_auth(_resolve_api_key(request, x_api_key)) |
There was a problem hiding this comment.
Addressed at main.py:717: _check_auth(x_api_key or key). /viz accepts the API key from either the X-Api-Key header (preferred) or the ?key= query param (bookmarkable shortcut), and _check_auth is a no-op when PALACE_API_KEY is unset (preserving the zero-config-local-dev experience). When PALACE_API_KEY IS set on the daemon, the dashboard shell is no longer publicly accessible.
…nding queue; trim patch Filed four upstream PRs on 2026-04-30: - rboarescu#15 feat: GET /viz status dashboard (stacks on rboarescu#13) - rboarescu#16 feat: GET /list — query-free metadata browse - rboarescu#17 feat: DELETE /memory + PATCH /memory - rboarescu#18 feat(lifespan): auto-migrate Stop-hook checkpoints on startup Also rebased PR rboarescu#13 onto upstream/main to clear a CHANGELOG conflict left by upstream's b4aee82 (patch sync) — state went CONFLICTING -> MERGEABLE / CLEAN. README: - Open upstream PRs table: four new rows (rboarescu#15-rboarescu#18) plus a 2026-04-30 note covering today's rebase + new PRs in one breath. - Pending PRs queue: now empty. Replaced the four stale rows (event-log-frame and graph-endpoint were already in flight via rboarescu#11 and rboarescu#13; mempal-fast.py was already upstream via the merged PR #4 omnibus; /viz is now PR rboarescu#15) with a brief empty-state note. CLAUDE.md: - Patch description trimmed to reflect that the hnsw:num_threads enforcement landed upstream via _pin_hnsw_threads(); only the log + retry-once slice remains. patches/mcp_server_get_collection.patch: - Regenerated against current mempalace develop. The patch is now just the "log exception + retry once on cache failure" change. Filed upstream as MemPalace/mempalace#1286; once that merges this patch retires. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Eight comments addressed across main.py, static/viz.html, CHANGELOG.md. Comments on docs/graph-endpoint.md and main.py:640 were inherited from PR rboarescu#13's review and are already settled by the 152e428 backport on main; left untouched here. ## Auth (was: /viz served publicly) main.py: /viz now goes through _check_auth on the same code path as every other endpoint. Accepts the key from either the X-Api-Key header (preferred) or the ?key=… query param (bookmarkable shortcut). _check_auth is a no-op when PALACE_API_KEY is unset, so the zero-config local-dev experience is unchanged. ## CDN integrity (was: D3 + Mermaid loaded without SRI, version not pinned) static/viz.html: pinned to [email protected] and [email protected] with SHA-384 SRI hashes via cdn.jsdelivr.net. Added crossorigin="anonymous" (required for SRI on cross-origin scripts) and referrerpolicy="no-referrer". Comment block above the script tags explains the version-bump+hash- together rule for future maintainers. ## Mermaid sanitization (was: label sanitizer didn't strip control chars; securityLevel "loose") static/viz.html: - mermaidSafe() now strips ASCII control chars (\n, \r, \t, etc.) in addition to the existing parser-breaking chars. A label containing raw newlines used to render half a label on each side of the break. - mermaid.initialize({ securityLevel: "strict" }) — "loose" was over-permissive (relaxes label sanitization, enables clickable diagram nodes). The dashboard never needs node click handlers, and our own mermaidSafe() runs before labels reach Mermaid anyway, so strict is the right default. ## Mermaid render-error fallback (was: <div> appended inside <pre>) static/viz.html: target was <pre id="hierarchy">; appending a <div> inside it produced invalid HTML and inconsistent cross-browser rendering. Surface the error as plain text inside the existing element and tag the container with class="err" so CSS can style it. ## ?key= leakage warning static/viz.html: comment on the KEY parsing block calls out that ?key=… leaks into browser history, referer headers, and proxy logs. Same caveat surfaced in the /viz docstring and CHANGELOG entry. ## Docstring drift (was: "cached at module load" but actually lazy) main.py: docstring rewritten to describe the actual lazy-load behaviour ("lazy-loaded on first request and cached in-process thereafter; one disk read per daemon process"). ## CHANGELOG.md (was: two ### Added headings under [Unreleased]) The /viz changelog block and the /graph block both used "### Added"; merged them under one heading. Also folded the new security details (SRI pinning, securityLevel: strict, control-char strip, /viz auth) into the /viz security bullet so the audit trail of what shipped here is complete. No daemon behaviour changes outside /viz. /viz behaviour change is the auth gate — operators with PALACE_API_KEY set will get 401 on unauthenticated /viz requests, which matches every other endpoint.
Same shape as 152e428 — production at disks runs fork main, so review amendments to the in-flight upstream PR need to ship here too. Eight Copilot comments addressed across main.py and static/viz.html. ## Auth (was: /viz served publicly) main.py: /viz now goes through _check_auth on the same code path as every other endpoint. Accepts the key from either the X-Api-Key header (preferred) or the ?key=… query param (bookmarkable shortcut). _check_auth is a no-op when PALACE_API_KEY is unset, so the zero-config local-dev experience is unchanged. Production deployments with PALACE_API_KEY set will now 401 unauthenticated /viz requests — matches every other endpoint. ## CDN integrity (was: D3 + Mermaid loaded without SRI, version not pinned) static/viz.html: pinned to [email protected] and [email protected] with SHA-384 SRI hashes via cdn.jsdelivr.net. Added crossorigin="anonymous" (required for SRI on cross-origin scripts) and referrerpolicy="no-referrer". ## Mermaid sanitization static/viz.html: - mermaidSafe() now strips ASCII control chars (\\n, \\r, \\t, etc.) in addition to the existing parser-breaking chars. - mermaid.initialize({ securityLevel: "strict" }) — was "loose" which relaxes label sanitization and enables clickable diagram nodes. The dashboard never needs node click handlers, and our own mermaidSafe() runs before labels reach Mermaid anyway. ## Mermaid render-error fallback (was: <div> appended inside <pre>) static/viz.html: target was <pre id="hierarchy">; appending a <div> inside it produced invalid HTML and inconsistent cross-browser rendering. Surface the error as plain text inside the existing element and tag with class="err" so CSS can style it. ## ?key= leakage warning static/viz.html: comment on the KEY parsing block calls out that ?key=… leaks into browser history, referer headers, and proxy logs. Same caveat in the /viz docstring. ## Docstring drift (was: "cached at module load" but actually lazy) main.py: docstring rewritten to describe the actual lazy-load behaviour ("lazy-loaded on first request and cached in-process thereafter; one disk read per daemon process"). All five fixes are amendments on the corresponding upstream PR rboarescu#15 branch (force-pushed earlier today). No behaviour change for the healthy unauthenticated-local case; the changes harden security surface that was caught in review. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
|
Pushed Auth gate — CDN integrity — pinned to Mermaid sanitisation —
Render-error fallback — was appending a ?key= leakage warning — added a comment block on the JS Docstring drift — "cached at module load" was a lie (lazy-loaded on first request). Rewritten to describe the actual behaviour. CHANGELOG dup heading — The two comments on |
Adds a single-page status dashboard at
GET /vizthat consumes/graph,/repair/status, and/healthin parallel and renders five panels: status strip, D3 force-directed knowledge graph, Mermaid wing/room hierarchy, tunnels list, and wings bar chart.This stacks on top of #13 — the dashboard's whole point is to be a thin viewer over
/graph, so it doesn't make sense without it. Marked as a draft alternative would also be fine if you'd prefer to keep the queue serialised; happy to convert.What it adds
static/viz.html— single HTML template (484 lines, cached at module load). All third-party JS is D3 + Mermaid via CDN (unpkg). No new server-side static-file plumbing.main.py— one new route handler,@app.get("/viz"). Reads the cached template at startup and serves it asHTMLResponse. Optional?refresh=Nquery param for periodic auto-refresh; optional?key=…for ergonomic auth bookmarking (no different from passingX-Api-Key, just convenient for browser bookmarks).CHANGELOG.md—Unreleased / Addedentry.Why this shape
/graphrather than fanning out. The page makes one HTTP call to/graph(and small ones to/repair/statusand/health). All structural data flows through one daemon endpoint, so the dashboard inherits/graph's direct-sqlite optimisation and stays decoupled from MCP changes.static/, loaded at module init. Operators cancurl /vizand pipe through a browser, or bookmarkhttps://palace/viz?key=…&refresh=15and walk away.mempalace_sync_statusMCP tool), #601 (mvanhorn — brief overview). I read each but didn't pull code from any — the page is structured around our own/graphshape, so it's tied to the daemon rather than to any one of those PRs landing.Security notes
/graphenter the DOM viatextContentor safesetAttribute. NeverinnerHTMLfor user-derived strings.[,],\",<,>,|,`to avoid breaking the parser. (Mermaid's own escaping is unreliable for these.)eval, no inline event handlers, no remote-code-load surface beyond the two named CDN URLs._check_auth(x_api_key)honoursPALACE_API_KEYif set.What's intentionally not in this PR
scripts/verify-routes.shprobe yet. That file is added by chore(scripts): add verify-routes.sh smoke test #9, so its/vizprobe is held back; happy to send a follow-up once chore(scripts): add verify-routes.sh smoke test #9 lands (or fold it into chore(scripts): add verify-routes.sh smoke test #9 if you prefer).VERSION = \"1.5.1\"for you to bump on release; theUnreleasedCHANGELOG section is where the entry sits./vizparagraph but it references fork-specific deployment context. Skipped here so this PR stays small and focused.Tested
Running on the canonical 151K-drawer palace at
palace.jphe.in/vizsince 2026-04-26 with?refresh=15. The CDN dependencies and/graph's ~0.4s response budget mean the page is interactive within ~1s on a cold load. Mermaid renders the wing/room tree for 36 wings × ~165 rooms in well under a second.Glad to break this down further or trim anything that doesn't fit. Thanks for considering it.