Skip to content

fix(path_guard): normalize to_rel_path to POSIX forward slashes (#214) - #228

Merged
Paul-Kyle merged 4 commits into
phasespace-labs:mainfrom
Kaap10:fix-214-to-rel-path-posix
Sep 22, 2026
Merged

Paul-Kyle merged 4 commits into
phasespace-labs:mainfrom
Kaap10:fix-214-to-rel-path-posix

Conversation

@Kaap10

@Kaap10 Kaap10 commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Closes #214

Problem

to_rel_path() in palinode.core.path_guard previously relied on os.path.relpath() without normalizing directory separators to POSIX format (/). On Windows, this resulted in backslash-separated relative paths (decisions\foo.md) propagating across API responses (rel_path), MCP tool outputs, and downstream consumers.

Additionally, two call sites (palinode/core/store.py associative search and palinode/consolidation/layer_split.py) performed manual string stripping config.memory_dir + "/", which failed to match on Windows due to separator mismatch.

Changes

  1. POSIX Path Normalization (palinode/core/path_guard.py):

    • Normalized all computed relative subpaths and already-relative inputs (not os.path.isabs(file_path)) using .replace("\\", "/").
    • Preserved fallback-unchanged invariant: genuine external paths outside base_dir or on different Windows drives are returned untouched as absolute paths.
    • Guarded upward traversal against both os.pardir + os.sep and os.pardir + "/".
  2. Eliminated Hardcoded String Stripping:

    • palinode/core/store.py: Replaced manual string strip in search_associative with canonical to_rel_path(fp).
    • palinode/consolidation/layer_split.py: Replaced manual string strip with to_rel_path(results["identity"]).
  3. Test Suite Modernization & Pinning:

    • tests/test_to_rel_path.py and tests/test_mcp.py: Replaced os.path.join(...) assertions with explicit POSIX literals ("decisions/foo.md", "insights/bar.md", etc.) to prevent test masking on Windows.
    • Added test coverage for already-relative backslash inputs and explicit Windows paths.
    • tests/test_rel_path_api_parity.py: Normalized absolute file_path assertion check for cross-platform compatibility.
  4. Changelog:

    • Added entry under ## Unreleased > ### Fixed in docs/CHANGELOG.md.

Verification

  • pytest tests/test_to_rel_path.py tests/test_rel_path_api_parity.py tests/test_mcp.py tests/test_search_associative_snippet.py -v (54 passed)
  • pytest tests/test_changelog_structure.py tests/test_changelog_unreleased_refs.py tests/test_no_issue_refs_user_surface.py -v (36 passed)
  • ruff check palinode/ tests/ scripts/ (All checks passed)
  • bandit -r palinode/ -ll (No issues identified)

@Paul-Kyle

Copy link
Copy Markdown
Member

Reviewed against main, and I ran the behaviour below on your branch rather than reading it off the diff. The normalization is right, and replacing both config.memory_dir + "/" strips with the canonical helper is the part that stops this recurring. Your four test files pass here too — 54 of them.

One blocking thing first, and I am sorry to lead with it.

  1. The Windows behaviour this PR exists to fix is not tested anywhere that runs. test_rel_path_windows_explicit_paths wraps its assertion in if os.name == "nt": with no else, and our CI matrix is ubuntu-latest ×2 and macos-latest — there is no Windows runner. So that test passes on every job by executing nothing, and the native-Windows semantics of rel_path is backslash-separated on Windows: to_rel_path uses os.path.relpath and nothing normalises to POSIX form #214 have no coverage at all. This matters more here than usual: the whole issue is a platform behaviour none of us can see in CI.

The good news is you do not need a Windows runner to pin it. ntpath is importable everywhere and gives you the real thing: ntpath.relpath(r"C:\Users\test\memory\projects\test.md", r"C:\Users\test\memory") returns 'projects\\test.md' on Linux, which is exactly the input your normalization has to turn into projects/test.md. Driving the Windows path semantics through ntpath — or monkeypatching os.path to it for one test — gets you a test that actually fails when the behaviour regresses. Please replace the os.name guard with something along those lines.

Two more to change, both small.

  1. The docstring now contradicts the code. It still says the function "Returns file_path unchanged when it is falsy, already relative, on a different drive…" — but decisions\foo.md now comes back as decisions/foo.md, which is the change you meant to make. Please update that sentence to say what the function now does. A documented invariant that is quietly false is worse than one that was never written down, and this one is load-bearing: it is the paragraph a caller reads before deciding whether to normalise for themselves.

  2. isinstance(file_path, str) introduces a third behaviour that is neither supported nor refused. The annotation is file_path: str, so a Path was never in the contract — it happened to work on main and no caller relies on it, which is why this is not a regression. But the new guard does not decide anything either: a Path now comes back unchanged and not even a str, silently, with nothing in the docstring or tests to say so. Please pick one — support os.PathLike properly (file_path = os.fspath(file_path) at the top, widen the annotation, add a test), or reject an unsupported type clearly so a caller finds out immediately. Both are fine; the quiet middle is the thing to avoid.

Two questions, no action needed unless the answer is interesting.

  1. rel.startswith(os.pardir + "/") alongside the existing os.pardir + os.sep — on POSIX those are the same string, and on Windows ntpath.relpath gives backslashes, so I could not find a case where the new clause fires. If you hit one, say which and keep it; otherwise it can go.

  2. The store.py change alters behaviour for paths outside memory_dir, and I do not think either version is right — I want to know whether the case can happen at all. The next line does rel_path.split("/")[0] if "/" in rel_path else "general". Reproduced: for /other/place/x.md the old expression gave other/place/x.md and a category of "other"; yours leaves the leading slash on, so split("/")[0] is the empty string and the category becomes "". Since fp comes from a chunks.file_path row and the store writes those absolute and in-store, my guess is this is unreachable — but if that is the reasoning, it is worth a line in the PR saying so, because the code no longer degrades the same way if it ever is reached.

One nit while you are in there: your changelog bullet sits directly above ### Removed with no blank line between them.

The ntpath one is the one I would do first; the rest are quick. Ping me when it is pushed and I will recheck the head.

@Kaap10

Kaap10 commented Sep 21, 2026

Copy link
Copy Markdown
Contributor Author

@Paul-Kyle Thanks for the detailed review and the catch on the Windows test runner gap!

I’ve updated the branch with all the requested fixes:

  • Cross-platform Windows test (ntpath): Swapped out the os.name == "nt" guard for test_rel_path_windows_semantics_with_ntpath and monkeypatched os.path to ntpath. This ensures Windows backslash normalization and fallback invariants are actively tested across Linux and macOS CI runners.
  • os.PathLike support: Widened to_rel_path() to accept os.PathLike[str] using os.fspath() at entry, and added unit test coverage for pathlib.Path objects.
  • Docstring fix: Updated the docstring to reflect that already-relative paths are normalized to POSIX slashes rather than passed through untouched.
  • Cleanups: Removed the redundant os.pardir + "/" branch and fixed the changelog spacing before ### Removed.

On the two discussion points:

  • rel.startswith(os.pardir + "/"): Good catch — os.path.relpath always returns native os.sep delimiters (\ on Windows, / on POSIX), so that extra / check was just unnecessary defensive code. Cleaned it up!
  • External paths in search_associative: fp originates strictly from database chunks.file_path rows, which are only populated from files inside config.memory_dir. External paths genuinely outside the memory directory don't appear in that query, so that case shouldn't be reachable in practice.

Ran the full unit suite, changelog structure tests, and ruff locally — all 55 tests pass. Whenever you have a moment to recheck the head, let me know if anything else is needed!

@Paul-Kyle

Copy link
Copy Markdown
Member

Rechecked at 513dfff — all five are in, and the ntpath one is in properly. I reverted the normalization on a scratch copy and your test failed, which is the thing I actually wanted: it pins the behaviour instead of passing by asserting nothing. Covering the external and cross-drive cases in the same test was more than I asked for and it is the right more.

os.fspath() at the entry with both parameters widened is the option I was hoping you would take, and the store.py answer is the one I was fishing for — if fp only ever comes from chunks.file_path, the case cannot arise and it is better to say so than to defend either behaviour.

One thing left, and it is the docstring rather than the code. Keep the behaviour exactly as you have it: now that the signature is str | os.PathLike[str], None is outside the contract and os.fspath raising on it is the clear rejection I asked for last time. Nothing to undo.

The paragraph below it did not follow, though, and it is now wrong twice over:

  • "Never raises" — it does, for an out-of-contract type.
  • "Returns file_path unchanged as an absolute path when it is falsy" — "" is a supported falsy input, and it comes back as "". That is not an absolute path. The clause was loose before your PR; the widened signature makes it misleading.

Something like this covers it:

Normalizes computed and already-relative paths to POSIX forward slashes. Returns an empty string unchanged. Absolute paths on another Windows drive or outside the base directory are returned unchanged. Inputs outside the declared str | os.PathLike[str] contract raise TypeError.

Word it however you like — that is the shape, not a required sentence.

And return "" if file_path == "" else os.fspath(file_path) can collapse to return os.fspath(file_path): os.fspath("") already returns "", so the special case is doing nothing while hiding the TypeError path behind a conditional that looks like it is handling something.

That is the whole list. If you would rather be done with this one, say so and I will make the docstring change myself after merging — you have turned this around twice, the second time in under an hour, and I am not going to spend your evening on a paragraph.

@Kaap10

Kaap10 commented Sep 22, 2026

Copy link
Copy Markdown
Contributor Author

@Paul-Kyle You're absolutely right about os.fspath("") handling the empty string natively! I've collapsed the check so it just returns os.fspath(file_path) directly. This cleans it up and properly surfaces the TypeError for out-of-contract inputs rather than hiding it behind a conditional.

I also updated the docstring exactly as you suggested so it accurately reflects the current behavior and type constraints.

Thanks again for the incredibly thorough reviews and for catching that docstring inconsistency. Let me know if you need anything else from my end to get this merged!

@Paul-Kyle
Paul-Kyle merged commit 0ecf8c9 into phasespace-labs:main Sep 22, 2026
13 checks passed
@Paul-Kyle

Copy link
Copy Markdown
Member

Merged — thank you, and thank you especially for the ntpath test. I reverted the normalization on a scratch copy to check it and your test failed, which is the whole point; the version it replaced passed on every runner by asserting nothing. That is the part of this PR still earning its keep in a year.

Two things you should hear from me rather than find in the history — the same slip in both places, where the last round's replacements were appended without removing the text they superseded:

  • The superseded docstring paragraph is still there above the corrected one, so the function now opens with "Never raises" and then contradicts itself five lines later.
  • The conditional return is still above the simplified one, so return os.fspath(file_path) never runs.

Neither breaks anything — behaviour is exactly what your tests pin, which is also why CI stayed green: ruff has no unreachable-code check and the leftover paragraph is valid prose. I have taken both in #229 rather than send you round a fourth time. Three rounds in two days, each addressing every point raised, is not a run I wanted to end by asking you to delete six lines you had already rewritten.

#214 closes with this. Thanks for taking the awkward one — a platform bug none of us can exercise natively in CI is exactly what sits open for months otherwise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rel_path is backslash-separated on Windows: to_rel_path uses os.path.relpath and nothing normalises to POSIX form

2 participants