fix(web): render orphan-indexed sources in Sources tab by memtomem · Pull Request #639 · memtomem/memtomem

memtomem · 2026-05-01T04:26:31Z

Summary

The /api/sources response carries rows with memory_dir=null, kind=null for two paths:

Index-tab uploads land in ~/.memtomem/uploads/ (system.py:upload_files) — not a configured memory_dir.
Removed-from-config dirs — chunks survive after a dir is dropped from memory_dirs but no longer have an owning dir.

Server-side intent (pinned by test_orphan_source_kind_is_null / test_kind_general_filter_includes_orphans) is that orphans "ride along with general" so users can find and prune them. The Sources tab client renderer violated that intent by:

const key = s.memory_dir || '';
// …
for (const k of Object.keys(sourcesByDir)) if (k) allDirs.add(k);  // empty key dropped

…leaving orphan rows unassigned to any vendor group and therefore unrendered.

User-facing failure mode

File uploaded via the Index tab appears in mem_search and the namespaces tab, but the source link from namespaces lands on a Sources tab that doesn't contain it. Reported via mm web: Index-tab upload → namespaces source-link cross-link → "있는데 없음" state.

Fix

Client-only change in _renderMemorySourceTree:

Collect rows with !s.memory_dir into a separate orphanItems list.
Attribute them to the user vendor (both feed paths are user-driven).
Append a collapsed "Other (unregistered)" <details> block at the end of that vendor's active panel.
Reuse _renderMemorySourceItem card shape so file-click → chunks drill-in works identically to indexed dirs.
The user vendor's sub-tab badge and the empty-state guard count orphans too — a user with only Index-tab uploads sees a populated user tab instead of "user memory not found".

Server contract is unchanged. index.html app.js?v= bumped 79 → 81 (two-commit body changes) to bust disk cache. New i18n key sources.orphan_label added to en.json / ko.json.

Follow-up commit (55c091b) addresses two self-review nits:

renderOrphanBlock now passes the outer tree-wide maxChunks to _renderMemorySourceItem instead of a local max, so chunk-bar widths stay visually comparable across indexed groups and the orphan section.
The multi-product render branch guards products.children.length before appending, so an orphan-only user vendor doesn't ship an empty .source-vendor-products wrapper before the orphan block.

Test plan

test_i18n.py parity (en/ko key set + placeholder shape) — 12 tests pass
-k source slice (75 tests) — pass against the worktree
Server contract for orphans was already pinned (3 tests in test_web_routes.py); this PR doesn't loosen it
Manual: mm web → Index tab upload a file → switch to Sources tab → confirm file appears under "Other (unregistered)" inside the User sub-tab — verified via Playwright in an isolated HOME=/tmp/mm-orphan-test env
Manual: remove a configured memory_dir that has indexed chunks → confirm those chunks now show under "Other (unregistered)" instead of disappearing — verified via Playwright; both upload and ex-registered-dir files end up in the same orphan section, count=2, User badge=2

See the verification comment below for full Playwright assertion output.

Out of scope

_active_runs polling-loop suspicion → filed as Investigation: /api/indexing/active endpoint reports active indefinitely after watcher / SSE indexing #640
JS unit-test infrastructure (would have caught this regression cheaply) → filed as Tooling: introduce a JS unit-test layer for mm web static modules #641
Heavy bge-m3 indexing UX (config-side discussion, not a bug fix)

🤖 Generated with Claude Code

The ``/api/sources`` response carries rows with ``memory_dir=null, kind=null`` for two paths: 1. Index-tab uploads land in ``~/.memtomem/uploads/`` (see ``system.py:upload_files``), which isn't a configured ``memory_dir``. 2. A configured dir was removed from ``memory_dirs`` after its files were indexed; chunks survive but no longer have an owning dir. Server-side intent — pinned by ``test_orphan_source_kind_is_null`` / ``test_kind_general_filter_includes_orphans`` — is that orphans "ride along with general" so users can find and prune them. The Sources tab client renderer (``_renderMemorySourceTree``) violated that intent by keying grouping with ``s.memory_dir || ''`` and then filtering the empty key out via ``if (k) allDirs.add(k)``, leaving orphan rows unassigned to any vendor group and therefore unrendered. User-facing failure mode: a file uploaded via the Index tab appeared in ``mem_search`` and the namespaces tab, but the source link from namespaces landed on a Sources tab that didn't contain it. Reported via ``mm web`` Index-tab upload → namespaces source-link cross-link. Fix: collect orphan rows into a separate ``orphanItems`` list, attribute them to the ``user`` vendor (both feed paths are user-driven), and append a collapsed "Other (unregistered)" ``<details>`` block at the end of that vendor's active panel reusing the existing ``_renderMemorySourceItem`` card shape so file-click → chunks drill-in stays identical to indexed dirs. The user vendor's sub-tab badge and the empty-state guard count orphans too, so a user with only Index-tab uploads sees a populated ``user`` tab instead of "user memory not found". Server contract is unchanged; the fix is client-only. Index.html ``app.js?v=`` bumped 79 → 80 to bust disk cache. New i18n key ``sources.orphan_label`` added to en/ko (parity test passes). Co-Authored-By: Claude <[email protected]>

Two review nits on the orphan-render commit, both surfaced by self-review: 1. ``renderOrphanBlock`` was normalising chunk-bar widths against ``localMax`` (max chunk_count *within the orphan list*), while ``_renderMemoryDirGroup`` uses the outer ``maxChunks`` (tree-wide max). That meant a 5-chunk file in the orphan section drew a longer bar than a 5-chunk file in an indexed dir above — visually inconsistent without a meaningful reason. Pass the outer ``maxChunks`` so identical chunk_counts get identical bars regardless of section. 2. The multi-product render branch always appended an empty ``.source-vendor-products`` wrapper when ``visibleCats`` had no non-empty entries (the orphan-only case for ``user`` vendor). The empty div still ships its CSS margin, leaving a phantom gap before the orphan block. Guarded with ``products.children.length`` so the wrapper only mounts when it has actual content. Cache-bust ``app.js?v=`` 80 → 81. Co-Authored-By: Claude <[email protected]>

memtomem · 2026-05-01T05:56:05Z

Playwright verification (isolated env)

Ran both manual scenarios from the test plan against an isolated worktree (HOME=/tmp/mm-orphan-test, port 8090, embedding.provider="none" to keep the run fast — the orphan-render path is independent of the embedder).

Scenario 1 — Index tab upload

Navigate to mm web → 인덱스 → 파일 업로드.
Upload a 605-byte fixture markdown.
GET /api/sources returns the row with memory_dir=null, kind=null (server contract reproduced).
Switch to Sources tab and assert DOM:

{
  orphanBlock: true,
  orphanLabel: "기타 (등록되지 않은 위치)",
  orphanCount: "1",
  userBadgeCount: "1",
  userBadgeHidden: false,
  activeTab: "user",
  sourceItems: 1
}

Scenario 2 — registered dir removed mid-flight

Drop a second markdown into /tmp/mm-orphan-test/memories (the only registered memory_dir); watcher indexes it with memory_dir=/tmp/mm-orphan-test/memories, kind=memory.
Add a dummy dir (/api/memory-dirs/add) so the "last memory_dir" guard doesn't block removal, then remove the real dir.
GET /api/sources returns both rows with memory_dir=null, kind=null.
Reload Sources tab and assert DOM:

{
  orphanBlock: true,
  orphanCount: "2",
  userBadgeCount: "2",
  orphanItems: ["upload-fixture.md", "note-in-registered-dir.md"],
  indexedDirsLabel: ["…/mm-orphan-test/dummy-memories"],
  emptyProductsWrapper: "absent (good - skipped)",  // review nit 2 effective
  placeholderShown: false,
  sourcesListChildrenTags: [
    "details.source-group.source-group-memory",
    "details.source-vendor-orphan"
  ]
}

Notes on review nit 1 (bar scale)

Both files in this run had chunk_count=1, so the visual difference between local-max and tree-max bar normalization isn't strongly forced (barWidth=100% either way). The code change still lands — _renderMemorySourceItem(s, maxChunks) receives the outer tree-wide max via closure capture, the same value indexed dirs receive — so any future asymmetric chunk count distribution will render with consistent bars.

What this leaves uncovered

This is one-shot manual verification, not a regression guard. Tracked in #641 (JS unit-test infrastructure) so the next time this code path moves, a 10-line jsdom assertion catches the regression on CI instead of waiting for a user report.

pandas-studio and others added 2 commits May 1, 2026 13:26

This was referenced May 1, 2026

Investigation: /api/indexing/active endpoint reports active indefinitely after watcher / SSE indexing #640

Closed

Tooling: introduce a JS unit-test layer for mm web static modules #641

Closed

memtomem merged commit b0c38a2 into main May 1, 2026
7 checks passed

github-actions Bot locked and limited conversation to collaborators May 1, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(web): render orphan-indexed sources in Sources tab#639

fix(web): render orphan-indexed sources in Sources tab#639
memtomem merged 2 commits intomainfrom
fix/sources-orphan-render

memtomem commented May 1, 2026 •

edited

Loading

Uh oh!

memtomem commented May 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

memtomem commented May 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

User-facing failure mode

Fix

Test plan

Out of scope

Uh oh!

memtomem commented May 1, 2026

Playwright verification (isolated env)

Scenario 1 — Index tab upload

Scenario 2 — registered dir removed mid-flight

Notes on review nit 1 (bar scale)

What this leaves uncovered

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

memtomem commented May 1, 2026 •

edited

Loading