Skip to content

fix: skip unreachable reparse points in detect_rooms_from_folders (WinError 448)#558

Merged
bensig merged 1 commit intoMemPalace:developfrom
OthmanAdi:fix/windows-reparse-point-oserror
Apr 11, 2026
Merged

fix: skip unreachable reparse points in detect_rooms_from_folders (WinError 448)#558
bensig merged 1 commit intoMemPalace:developfrom
OthmanAdi:fix/windows-reparse-point-oserror

Conversation

@OthmanAdi
Copy link
Copy Markdown
Contributor

What does this PR do?

mempalace init crashes on Windows when a project directory contains a reparse point (git-submodule junction, dev-drive mount, or any path Windows marks as an untrusted mount point).

iterdir() lists the entry without error, but the subsequent item.is_dir() call internally calls stat(), which raises OSError — specifically:

OSError: [WinError 448] The path cannot be traversed because it contains
an untrusted mount point: '...\skills\pr-perfect'

This aborts detect_rooms_from_folders entirely, so mempalace init never completes.

Fix: wrap every is_dir() call in detect_rooms_from_folders with try/except OSError: continue. Both passes (top-level and one-level-deep nested) are covered. The function now skips inaccessible entries and continues scanning the rest of the directory tree.

Reproducer

Any Windows project with a git submodule checked out as a junction point, or any directory on a Windows Dev Drive that contains a third-party reparse point, will hit this. Confirmed on Windows 11 with Python 3.12 and chromadb 0.6.3.

How to test

Two new tests in test_room_detector_local.py mock Path.is_dir to raise OSError for a specific path and assert the function:

  1. Does not raise
  2. Still discovers rooms from the remaining accessible directories
python -m pytest tests/test_room_detector_local.py -v -k "oserror"
# 2 passed

Full suite:

python -m pytest tests/ -v

Checklist

  • Tests pass (python -m pytest tests/ -v)
  • No hardcoded paths
  • Linter passes (ruff check .)

@web3guru888
Copy link
Copy Markdown

Good targeted fix for a real Windows edge case — the WinError 448 reparse point crash is a genuine pain point that would silently block mempalace init for any Windows user with git submodules or Dev Drive mounts.

The fix itself is correct. iterdir() happily lists reparse points; it is the subsequent stat() call inside is_dir() that raises. Wrapping all three is_dir() calls in try/except OSError: continue at both scan levels is the right approach — no overreach, no EAFP philosophy violation.

One note on the tests: The patch.object(bad.__class__, "is_dir", patched_is_dir) approach patches at the class level (PosixPath or WindowsPath), which means the mock affects all Path instances during that with block. The if self == bad guard prevents it from breaking unrelated paths, but it is a class-level patch rather than instance-level. This is fine for these tests since the guard is correct, but worth being aware of — if future tests in the same file run in parallel they could see an unexpected patch. A monkeypatch.setattr approach targeting just the method on the specific path would be safer in pytest, but this works correctly as written.

Edge case worth noting for follow-up: The second iterdir() pass iterates top-level items again to walk one level deeper. If a top-level path raises OSError on is_dir(), it is correctly caught in the first pass — but the second pass calls iterdir() on the parent again and will encounter the same bad entry. The try/except in the second pass catches it there too, so it is handled. Just confirming the fix is complete end-to-end, which it is.

LGTM — this closes a real Windows blocker with minimal footprint.

@OthmanAdi OthmanAdi force-pushed the fix/windows-reparse-point-oserror branch from d042677 to 7ab657e Compare April 10, 2026 17:52
Copy link
Copy Markdown

@web3guru888 web3guru888 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: fix: skip unreachable reparse points (WinError 448)

This is a well-scoped, correctly-targeted Windows crash fix. The root cause is well-understood — iterdir() happily yields entries for reparse points and junctions, but is_dir() internally calls stat() which raises OSError when Windows deems the mount point untrusted. The fix is minimal and correct.

What's done well

Coverage is complete. Both scan passes (detect_rooms_from_folders has a top-level pass and a one-level-deep nested pass) now have the guard. It would be easy to fix only the first pass and miss the second — the author didn't.

Exception type is precise. Using except OSError (not a bare except Exception) is the right call. OSError is exactly what Windows raises for reparse-point stat errors, and catching it narrowly avoids accidentally silencing unrelated failures like permission errors on normal directories.

Variable naming is consistent. The second pass uses item_is_dir and subitem_is_dir to avoid clobbering the outer variable. Clean.

Tests cover both scan depths. Two tests: one for the top-level pass, one for the nested pass. Both use patch.object on Path.is_dir to inject the exact WinError 448 string and assert the function (a) doesn't raise and (b) still discovers rooms from accessible siblings. That's the right testing strategy for OS-specific error injection on non-Windows CI.

Issues found

No warning/logging on skip. The PR silently skips inaccessible entries. For a user running mempalace init who has important directories under junction points, there's currently no feedback that anything was skipped. A logger.debug("Skipping %s: %s", item, exc) or even a logger.warning would help diagnostics without being noisy in the normal case. The module doesn't currently import logging — a one-liner addition.

item.iterdir() in the nested pass is unguarded. After the outer is_dir guard succeeds, the nested pass calls item.iterdir() directly (line ~148). On some Windows configurations, a reparse point that passes is_dir() can still fail on iterdir() itself. Worth wrapping that call too:

try:
    subitems = list(item.iterdir())
except OSError:
    continue
for subitem in subitems:
    ...

continue in the nested subitem guard is structurally different. In the outer loop a continue skips the whole item. In the inner loop the continue only skips the current subitem — which is correct, but the asymmetry is worth a comment for future readers.

Test monkeypatching is class-level. patch.object(bad.__class__, "is_dir", patched_is_dir) patches Path.is_dir for all Path instances in the test (the closure filters by identity, but another thread calling is_dir during the with block could theoretically see patched behavior). In a single-threaded pytest run this is fine; just flagging for awareness.

Suggestions

  1. Add import logging + logger = logging.getLogger("mempalace") to room_detector_local.py and emit a debug log on skip.
  2. Wrap item.iterdir() in the nested pass with try/except OSError: continue.
  3. Add a brief # WinError 448 comment above each guard so the next person searching for that error code finds it immediately (the PR description mentions it, but code comments are more discoverable).

Overall verdict

Approve with minor suggestions. The core fix is correct and complete for the stated bug. The unguarded iterdir() in the nested pass is a real edge case worth addressing. The logging gap is a UX issue, not a correctness issue. Neither blocks merge — this is a crash fix for Windows users and the code is right.


Reviewed by MemPalace-AGI — autonomous research system with perfect memory

@OthmanAdi OthmanAdi force-pushed the fix/windows-reparse-point-oserror branch from 7ab657e to ece3f3d Compare April 11, 2026 08:33
@OthmanAdi
Copy link
Copy Markdown
Contributor Author

Good catches — addressed all three suggestions in the latest push:

  1. Logging on skip — added import logging + logger = logging.getLogger(__name__) and emit logger.debug("Skipping %s: %s", item, exc) at each guard site (top-level is_dir, nested is_dir, and nested iterdir).

  2. Unguarded item.iterdir() in the nested pass — wrapped with try/except OSError: continue and extracted to subitems = list(item.iterdir()) before the inner loop. Added a third test (test_detect_rooms_from_folders_skips_iterdir_oserror) that mocks iterdir() itself to raise and confirms the function still discovers accessible siblings.

  3. # WinError 448 comments — added above each guard so the error code is grep-able in the source.

All 3 OSError tests pass. ruff check + ruff format --check clean.

On Windows, projects containing git-submodule junctions or dev-drive
reparse points cause iterdir() to list the entry successfully but
Path.is_dir() to raise OSError when it calls stat() internally.

Reproducer: any Windows project with a submodule checked out as a
junction (e.g. skills/pr-perfect) crashes mempalace init with:
  OSError: [WinError 448] The path cannot be traversed because it
  contains an untrusted mount point

Fix: wrap every is_dir() call in detect_rooms_from_folders with
try/except OSError so the scanner skips inaccessible entries and
continues rather than aborting.

Covers both the top-level pass and the one-level-deep nested pass.
Two new tests mock the OSError on specific paths and verify the
function returns correct rooms from the remaining accessible entries.
Copy link
Copy Markdown

@web3guru888 web3guru888 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three concerns addressed:

  • Logging on skip ✓ (debug-level, right verbosity)
  • Unguarded wrapped ✓
  • Third test for iterdir mock ✓

Clean fix. LGTM.

@bensig bensig changed the base branch from main to develop April 11, 2026 22:21
@bensig bensig requested a review from igorls as a code owner April 11, 2026 22:21
@bensig bensig merged commit 9c4b730 into MemPalace:develop Apr 11, 2026
6 checks passed
NickShtefan pushed a commit to NickShtefan/mempalace that referenced this pull request Apr 12, 2026
…mPalace#558)

On Windows, projects containing git-submodule junctions or dev-drive
reparse points cause iterdir() to list the entry successfully but
Path.is_dir() to raise OSError when it calls stat() internally.

Reproducer: any Windows project with a submodule checked out as a
junction (e.g. skills/pr-perfect) crashes mempalace init with:
  OSError: [WinError 448] The path cannot be traversed because it
  contains an untrusted mount point

Fix: wrap every is_dir() call in detect_rooms_from_folders with
try/except OSError so the scanner skips inaccessible entries and
continues rather than aborting.

Covers both the top-level pass and the one-level-deep nested pass.
Two new tests mock the OSError on specific paths and verify the
function returns correct rooms from the remaining accessible entries.
ichoosetoaccept pushed a commit to detailobsessed/mempalace that referenced this pull request Apr 13, 2026
…iterdir OSError)

- Expand ~ in split command dir/output-dir args (upstream MemPalace#361)
- Use yield+close in KG test fixture to prevent SQLite leaks (MemPalace#450)
- Use char/3.8 token estimate for consistency (MemPalace#609)
- Handle OSError from iterdir on broken symlinks/reparse points (MemPalace#558)

Closes DOT-488.
ichoosetoaccept pushed a commit to detailobsessed/mempalace that referenced this pull request Apr 13, 2026
…iterdir OSError)

- Expand ~ in split command dir/output-dir args (upstream MemPalace#361)
- Use yield+close in KG test fixture to prevent SQLite leaks (MemPalace#450)
- Use char/3.8 token estimate for consistency (MemPalace#609)
- Handle OSError from iterdir on broken symlinks/reparse points (MemPalace#558)

Closes DOT-488.
ichoosetoaccept pushed a commit to detailobsessed/mempalace that referenced this pull request Apr 13, 2026
…iterdir OSError)

- Expand ~ in split command dir/output-dir args (upstream MemPalace#361)
- Use yield+close in KG test fixture to prevent SQLite leaks (MemPalace#450)
- Use char/3.8 token estimate for consistency (MemPalace#609)
- Handle OSError from iterdir on broken symlinks/reparse points (MemPalace#558)

Closes DOT-488.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants