fix(launcher): isolate the vendored deps/ from user site-packages (#621) - #623

Summary

scripts/launcher.py put the plugin's vendored deps/ first on sys.path and left the interpreter's user site-packages behind it. That is not isolation, and it leaks in both directions.

Every package deps/ does not vendor kept resolving from user site-packages. On a Windows machine carrying a coherent CUDA torch/torchvision/torchaudio set, torch came from deps/ (CPU) while torchvision came from user site-packages built against the other torch, so importing them together aborted the MCP server during startup with RuntimeError: operator torchvision::nms does not exist. transformers re-raised that as a misleading ModuleNotFoundError: Could not import module 'PreTrainedModel', and the host surfaced it only as CONNECTION_CLOSED.

In the other direction, sys.path.insert never processes a directory's .pth files, so deps/pywin32.pth (the only route to pywintypes) was inert. Cutting user site-packages on its own with PYTHONNOUSERSITE or python -s therefore traded one import failure for another, which is what the reporter measured on the affected machine.

A new stdlib-only scripts/launcher_site.py does both halves in the order that works: site.addsitedir(deps_dir) so the .pth files run, then drop every sys.path entry that resolves to the user site-packages directory or sits under it. isolate_deps() owns the sys.path insert too, so no call site can perform the two steps out of order (site.addsitedir alone appends a directory sys.path does not carry, which would put deps/ behind system site-packages and lose the fix).

Seams it is applied to, verified against the code rather than assumed:

• scripts/launcher.py main(), the one bootstrap shared by the MCP server and all eleven lifecycle hooks (ADR-0742). It runs twice: once before ensure_deps/ensure_all_deps, so those see an isolated path, and once after, because that pip run resolves transitives (only the installers' requirements closure passes --no-deps) and can land a .pth the first call could not see. An unprocessed .pth is this issue's own bug class for a different package. The repeat costs 0.5 ms median and 0.7 ms p95 over 200 samples on a 600-entry deps directory.
• scripts/setup.py verify(), the block that produced the issue's [FAIL] sentence-transformers row. Without it the installer keeps failing on a machine whose server now starts fine.
• mcp_server/hooks/entry.py (added by feat(hooks): a console entry point runs any lifecycle hook from the wheel #605) needs nothing. It runs under uvx, which owns dependency resolution in its own environment, and it never puts deps/ on sys.path. This is one seam, not two.

A site.getusersitepackages() that cannot resolve reports the cause on stderr rather than returning None quietly, since a quiet None leaves user site-packages on sys.path and reproduces this issue's symptom with no trace. It stays silent on the normal path so eleven hooks do not each add a line.

Cross-platform safety was measured rather than assumed. Inside a virtualenv (dev clones and CI) site.ENABLE_USER_SITE is False and the user site-packages directory is already off sys.path, so the removal is a no-op there. Outside one, deps/ is self-contained by construction: launcher_deps.ensure_deps and ensure_all_deps decide what to install from .dist-info inside deps_dir, never from general importability, so cutting user site-packages cannot strand a package they thought was present.

The issue's secondary finding is fixed too. When scripts/setup.py failed, install-plugin.sh ended with "PostgreSQL must be installed and running first" whatever the run had reported, including the reported run whose four PostgreSQL checks all passed and whose only failing row was sentence-transformers. A new scripts/lib/setup_py_step.sh captures the run's output and echoes back its [FAIL] <check> rows. The PostgreSQL install guidance survives as a conditional instead of an assertion. Capturing means piping, and a pipe makes CPython block-buffer the child's stdout where a tty line-buffers it, so the run is made unbuffered: measured, a print() followed by a two-second sleep reaches the reader at t+2s through a pipe and at t+0s with PYTHONUNBUFFERED=1. Carriage returns are stripped for the same reason the message exists at all: CPython's text-mode stdout writes CRLF on Windows and a pipe preserves it byte for byte, so the extracted name arrived as sentence-transformers\r and, from column 0, the rest of the installer's line overwrote the check it had just named.

Closes #621

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would change existing behavior)
• Refactor (no functional change; rules/coding-standards.md compliance)
• Documentation only
• Audit-finding closure (cite the finding ID)

Test plan

This is a Windows-only failure mode and it was fixed on macOS, so the tests exercise the logic (which entries sys.path ends up carrying, and whether .pth processing fires) instead of requiring a real torch/torchvision mismatch.

• tests_py/scripts/test_launcher_site.py and test_launcher_site_paths.py, 18 tests across the two (split along the module's own seam to stay under the 300-line cap; the shared loader lives in _launcher_site_fixture.py and reaches them through a new tests_py/scripts/conftest.py). without_user_site is driven with a fake sys.path, including spellings that site never reports (a trailing separator, a . segment) which a raw string comparison would keep. isolate_deps runs against a temporary deps directory holding a real .pth file that names nested directories and carries an import line, exactly as deps/pywin32.pth does. site.addsitedir's .pth handling is identical on every platform, so this exercises the pywin32 half for real on macOS and Linux. Further tests cover: an absent deps_dir landing at index 0 ahead of system site-packages, idempotence across two calls, a .pth that appears between two calls being picked up, directories under user site-packages being dropped while a sibling sharing their prefix is kept, the stderr diagnostic when getusersitepackages() raises, and silence when it resolves.
• tests_py/scripts/test_setup_py_failure_message.py, 9 tests. They source the real scripts/lib/setup_py_step.sh and replay the issue's verification block with its ANSI colour codes. The failure sentence must name sentence-transformers and must not say "PostgreSQL". A real PostgreSQL failure is still named, and a run that dies before reporting any check says so instead of guessing. The unbuffered run is asserted on the mechanism, the variable the child actually receives, rather than on a clock. Two cases feed CRLF, since CPython's text-mode stdout writes it on Windows.
• Regression proof rather than a merely green suite. Reverting isolate_deps to a plain sys.path.insert body fails two of its tests (8 passed, 2 failed) and restoring it returns them. Removing PYTHONUNBUFFERED=1 fails test_the_run_is_unbuffered_so_tee_does_not_hide_progress with unbuffered=unset and restoring it returns 7 passed.
• Both test_launcher_site.py and test_launcher_site_paths.py join the "Test (Windows, SQLite backend)" leg's test list, since os.path.normcase only carries meaning there. That leg also runs bash scripts/install-plugin.sh for real, so the new shell library and the setup.py isolation call both execute on Windows on every PR.

Gates run locally after the final commit:

• pytest tests_py/scripts/: 1069 passed, 3 skipped, 360 subtests passed (rebased onto origin/main at feat(codex): full profile and the same eleven lifecycle hooks under Codex #620)

• pytest tests_py/hooks tests_py/architecture tests_py/invariants: 675 passed, 13 subtests passed

• ruff check .: All checks passed. ruff format --check .: 3410 files already formatted (ruff 0.16.6, the pinned version)

• python scripts/check_craftsmanship.py: Craftsmanship gate: OK

• python scripts/check_doc_claims.py, check_version_surfaces.py, check_ci_gate_complete.py, check_project_wiki.py, check_no_deps_invariant.py: all OK

• Rebased onto origin/main at 4400d7a (feat(codex): full profile and the same eleven lifecycle hooks under Codex #620) after it moved mid-review; the only conflict was the CHANGELOG's [Unreleased] block, resolved by keeping both sections. Every gate above was re-run on the rebased tree.

• bash -n and shellcheck -x on both changed shell scripts, plus an end-to-end drive of the failing installer path, which now prints scripts/setup.py failed. Failed check(s): sentence-transformers. Re-run manually: ...

• All existing tests pass.

• New tests added for new behavior.

• Mutation survival check: I considered what mutations would NOT be caught.

• Manual verification of any UI / CLI / MCP-tool behavior changes.

Two mutations the tests would not catch, stated rather than hidden. Swapping os.path.normcase for a no-op survives on POSIX, where normcase is the identity function, and is only killed by the Windows CI leg. And no test asserts that a real torch/torchvision mismatch now imports; that claim rests on the reporter's measurement on the affected machine, not on this repo's CI.

Audit notes

• Engineering review: the fix was derived from the primary artefacts (the issue text, scripts/launcher.py, mcp_server/hooks/entry.py, scripts/setup.py, scripts/launcher_deps.py, scripts/launcher_pip.py, scripts/launcher_pins.py) instead of applying the reporter's patch verbatim. Two of its assumptions were re-derived and one was narrowed: entry.py does not need the fix, and the removal is safe cross-platform because deps/ presence detection is dist-info-scoped to deps_dir.
• site.addsitedir semantics were measured on this machine instead of recalled. With deps_dir already at sys.path[0] it keeps that index, does not duplicate the entry, appends the directories the .pth names, and executes the .pth import line. site.ENABLE_USER_SITE and getusersitepackages() were measured inside and outside a virtualenv. The per-launch cost of addsitedir was measured rather than assumed negligible: 0.5 ms median, 0.7 ms p95, 1.3 ms max over 200 samples on a 600-entry directory, against an interpreter start of tens of milliseconds.
• A third pass found the CRLF defect above, in the code this PR itself added, on the one platform the message is for. It was reproduced against the library at this branch's head before being changed (od -c showing sentence-transformers\r.) and again after. The first draft of its fix carried a claim that BSD sed would read \r as a literal r and truncate "FlashRank reranker"; measuring macOS's /usr/bin/sed showed the opposite, so the comment now states only what was measured and rests the choice of tr on what POSIX actually defines.
• A second review pass on this diff found and fixed a regression it had introduced: routing setup.py through tee had silently turned its live progress output into block-buffered output that appeared only at exit. The fix and its measurement are described above.
• Outstanding deferred findings, two of the same shape. setup_database() and cache_embedding_model() in scripts/setup.py both hand deps/ to a child process through PYTHONPATH alone, which cannot process .pth files and cannot cut user site-packages, and the macOS/Linux twin scripts/lib/precache_embedding_model.sh does the same. cache_embedding_model and the shell twin are warn-only steps whose failure is absorbed by the lazy first-use download, which after this PR runs inside an isolated process. setup_database fails fatally, but its child imports psycopg and mcp_server.infrastructure.pg_schema, neither of which is part of the torch ecosystem split this issue reports, so nothing observed reaches it. Closing any of the three needs a child-process bootstrap API and, for the shell twin, a signature change to a library covered by scripts/setup.sh step 5/7 reports [ok] after the model cache failed #537's tests. All three now carry a comment naming the gap at the code, so it does not live only in this description; fixing them is left for the owner's call rather than folded in here.

Coding-standards compliance

• §2.2 Layer dependency direction preserved (no inward layer imports outward).
• §3.2 No any (or as any, or untyped dicts at boundaries) in production code.
• §4.1 No file > 500 lines (test files exempt).
• §4.2 No function > 50 lines (dispatch tables exempt).
• §4.4 No function with > 4 parameters.
• §7 Local reasoning preserved (no clever constructs that defeat single-function comprehension).
• §8 Every numeric constant ≥ 3 significant digits has a // source: annotation.
• §9 No dead code, no TODOs without issue references.

scripts/launcher_site.py is 118 lines and declares no numeric constants. The craftsmanship gate was verified clean on the committed diff.

Breaking changes

None for a working install. One behavioural change is worth stating: a package that was previously reachable only from user site-packages is no longer visible to the MCP server, to the lifecycle hooks, or to setup.py's verification. That is the point of the fix, since deps/ is self-contained by construction, and it means an install that was silently borrowing from user site-packages will install the package into deps/ on the next ensure_deps run.

Screenshots / logs

Installer failure message, before and after, on the reported verification block:

before: scripts/setup.py failed. PostgreSQL must be installed and running first (...)
after:  scripts/setup.py failed. Failed check(s): sentence-transformers. Re-run manually: ...

Buffering measurement behind the PYTHONUNBUFFERED=1 line, print() then a 2s sleep then a second print():

through | tee                        t+2s FIRST_LINE   t+2s SECOND_LINE
through | tee, PYTHONUNBUFFERED=1    t+0s FIRST_LINE   t+2s SECOND_LINE

Reviewer checklist

• CHANGELOG.md updated under the appropriate section.
• Documentation updated (README / SKILL.md / docs/). Nothing in docs/ or README.md documents the sys.path construction, and CHANGELOG is this repo's convention for a bug fix.
• No secrets / credentials / PII in the diff.
• CI passes on the latest commit. Not watched; this PR is handed over at the push.

Co-Authored-By: Claude Sonnet 5 [email protected]
Claude-Session: https://claude.ai/code/session_01Cz7A6goxVzfKL9NT7k4zDH

🤖 Generated with Claude Code

https://claude.ai/code/session_01Cz7A6goxVzfKL9NT7k4zDH