fix: skip unreachable reparse points in detect_rooms_from_folders (WinError 448)#558
Conversation
|
Good targeted fix for a real Windows edge case — the WinError 448 reparse point crash is a genuine pain point that would silently block The fix itself is correct. One note on the tests: The Edge case worth noting for follow-up: The second LGTM — this closes a real Windows blocker with minimal footprint. |
d042677 to
7ab657e
Compare
web3guru888
left a comment
There was a problem hiding this comment.
Review: fix: skip unreachable reparse points (WinError 448)
This is a well-scoped, correctly-targeted Windows crash fix. The root cause is well-understood — iterdir() happily yields entries for reparse points and junctions, but is_dir() internally calls stat() which raises OSError when Windows deems the mount point untrusted. The fix is minimal and correct.
What's done well
Coverage is complete. Both scan passes (detect_rooms_from_folders has a top-level pass and a one-level-deep nested pass) now have the guard. It would be easy to fix only the first pass and miss the second — the author didn't.
Exception type is precise. Using except OSError (not a bare except Exception) is the right call. OSError is exactly what Windows raises for reparse-point stat errors, and catching it narrowly avoids accidentally silencing unrelated failures like permission errors on normal directories.
Variable naming is consistent. The second pass uses item_is_dir and subitem_is_dir to avoid clobbering the outer variable. Clean.
Tests cover both scan depths. Two tests: one for the top-level pass, one for the nested pass. Both use patch.object on Path.is_dir to inject the exact WinError 448 string and assert the function (a) doesn't raise and (b) still discovers rooms from accessible siblings. That's the right testing strategy for OS-specific error injection on non-Windows CI.
Issues found
No warning/logging on skip. The PR silently skips inaccessible entries. For a user running mempalace init who has important directories under junction points, there's currently no feedback that anything was skipped. A logger.debug("Skipping %s: %s", item, exc) or even a logger.warning would help diagnostics without being noisy in the normal case. The module doesn't currently import logging — a one-liner addition.
item.iterdir() in the nested pass is unguarded. After the outer is_dir guard succeeds, the nested pass calls item.iterdir() directly (line ~148). On some Windows configurations, a reparse point that passes is_dir() can still fail on iterdir() itself. Worth wrapping that call too:
try:
subitems = list(item.iterdir())
except OSError:
continue
for subitem in subitems:
...continue in the nested subitem guard is structurally different. In the outer loop a continue skips the whole item. In the inner loop the continue only skips the current subitem — which is correct, but the asymmetry is worth a comment for future readers.
Test monkeypatching is class-level. patch.object(bad.__class__, "is_dir", patched_is_dir) patches Path.is_dir for all Path instances in the test (the closure filters by identity, but another thread calling is_dir during the with block could theoretically see patched behavior). In a single-threaded pytest run this is fine; just flagging for awareness.
Suggestions
- Add
import logging+logger = logging.getLogger("mempalace")toroom_detector_local.pyand emit a debug log on skip. - Wrap
item.iterdir()in the nested pass withtry/except OSError: continue. - Add a brief
# WinError 448comment above each guard so the next person searching for that error code finds it immediately (the PR description mentions it, but code comments are more discoverable).
Overall verdict
Approve with minor suggestions. The core fix is correct and complete for the stated bug. The unguarded iterdir() in the nested pass is a real edge case worth addressing. The logging gap is a UX issue, not a correctness issue. Neither blocks merge — this is a crash fix for Windows users and the code is right.
Reviewed by MemPalace-AGI — autonomous research system with perfect memory
7ab657e to
ece3f3d
Compare
|
Good catches — addressed all three suggestions in the latest push:
All 3 OSError tests pass. |
On Windows, projects containing git-submodule junctions or dev-drive reparse points cause iterdir() to list the entry successfully but Path.is_dir() to raise OSError when it calls stat() internally. Reproducer: any Windows project with a submodule checked out as a junction (e.g. skills/pr-perfect) crashes mempalace init with: OSError: [WinError 448] The path cannot be traversed because it contains an untrusted mount point Fix: wrap every is_dir() call in detect_rooms_from_folders with try/except OSError so the scanner skips inaccessible entries and continues rather than aborting. Covers both the top-level pass and the one-level-deep nested pass. Two new tests mock the OSError on specific paths and verify the function returns correct rooms from the remaining accessible entries.
web3guru888
left a comment
There was a problem hiding this comment.
All three concerns addressed:
- Logging on skip ✓ (debug-level, right verbosity)
- Unguarded wrapped ✓
- Third test for iterdir mock ✓
Clean fix. LGTM.
…mPalace#558) On Windows, projects containing git-submodule junctions or dev-drive reparse points cause iterdir() to list the entry successfully but Path.is_dir() to raise OSError when it calls stat() internally. Reproducer: any Windows project with a submodule checked out as a junction (e.g. skills/pr-perfect) crashes mempalace init with: OSError: [WinError 448] The path cannot be traversed because it contains an untrusted mount point Fix: wrap every is_dir() call in detect_rooms_from_folders with try/except OSError so the scanner skips inaccessible entries and continues rather than aborting. Covers both the top-level pass and the one-level-deep nested pass. Two new tests mock the OSError on specific paths and verify the function returns correct rooms from the remaining accessible entries.
…iterdir OSError) - Expand ~ in split command dir/output-dir args (upstream MemPalace#361) - Use yield+close in KG test fixture to prevent SQLite leaks (MemPalace#450) - Use char/3.8 token estimate for consistency (MemPalace#609) - Handle OSError from iterdir on broken symlinks/reparse points (MemPalace#558) Closes DOT-488.
…iterdir OSError) - Expand ~ in split command dir/output-dir args (upstream MemPalace#361) - Use yield+close in KG test fixture to prevent SQLite leaks (MemPalace#450) - Use char/3.8 token estimate for consistency (MemPalace#609) - Handle OSError from iterdir on broken symlinks/reparse points (MemPalace#558) Closes DOT-488.
…iterdir OSError) - Expand ~ in split command dir/output-dir args (upstream MemPalace#361) - Use yield+close in KG test fixture to prevent SQLite leaks (MemPalace#450) - Use char/3.8 token estimate for consistency (MemPalace#609) - Handle OSError from iterdir on broken symlinks/reparse points (MemPalace#558) Closes DOT-488.
What does this PR do?
mempalace initcrashes on Windows when a project directory contains a reparse point (git-submodule junction, dev-drive mount, or any path Windows marks as an untrusted mount point).iterdir()lists the entry without error, but the subsequentitem.is_dir()call internally callsstat(), which raisesOSError— specifically:This aborts
detect_rooms_from_foldersentirely, somempalace initnever completes.Fix: wrap every
is_dir()call indetect_rooms_from_folderswithtry/except OSError: continue. Both passes (top-level and one-level-deep nested) are covered. The function now skips inaccessible entries and continues scanning the rest of the directory tree.Reproducer
Any Windows project with a git submodule checked out as a junction point, or any directory on a Windows Dev Drive that contains a third-party reparse point, will hit this. Confirmed on Windows 11 with Python 3.12 and chromadb 0.6.3.
How to test
Two new tests in
test_room_detector_local.pymockPath.is_dirto raiseOSErrorfor a specific path and assert the function:Full suite:
Checklist
python -m pytest tests/ -v)ruff check .)