Skip to content

Fix pytest state isolation for direct session saves#1136

Closed
franksong2702 wants to merge 1 commit intonesquena:masterfrom
franksong2702:franksong2702/pytest-state-isolation
Closed

Fix pytest state isolation for direct session saves#1136
franksong2702 wants to merge 1 commit intonesquena:masterfrom
franksong2702:franksong2702/pytest-state-isolation

Conversation

@franksong2702
Copy link
Copy Markdown
Contributor

@franksong2702 franksong2702 commented Apr 27, 2026

Thinking Path

tests/test_sprint46.py imports api.config.SESSION_DIR at collection time and then calls Session.save() directly from the pytest main process. Before this change, tests/conftest.py only passed HERMES_WEBUI_STATE_DIR to the test server subprocess, so api.config.STATE_DIR could fall back to the real ~/.hermes/webui path in pytest itself.

What Changed

  • Publish pytest isolation env vars from tests/conftest.py at module import time, before test modules import api.config or api.models.
  • Use direct assignment for production-risk paths so an inherited shell environment cannot leave pytest pointed at the real WebUI state directory.
  • Add tests/test_pytest_state_isolation.py to lock the invariant that api.config.STATE_DIR and SESSION_DIR resolve under the isolated test state directory, not ~/.hermes/webui.
  • Update adjacent tests that intentionally exercise env-less workspace fallback or onboarding cleanup so they explicitly reset the pytest-level isolation state they depend on.

Why It Matters

Tests that save sessions directly in-process can no longer create or update files in the real WebUI session store. The server subprocess isolation and pytest main-process isolation now agree on the same test state root.

Verification

  • pytest tests/test_pytest_state_isolation.py tests/test_sprint46.py -q
  • pytest tests/test_default_workspace_fallback.py -q
  • pytest tests/test_onboarding_existing_config.py -q
  • pytest tests/test_pytest_state_isolation.py tests/test_sprint46.py tests/test_default_workspace_fallback.py tests/test_onboarding_existing_config.py -q
  • Checked the existing ~/.hermes/webui/sessions/compress_test_001.json sentinel before and after the run; size and mtime were unchanged.

Risks / Follow-ups

  • Low risk. The change is scoped to pytest configuration and a regression test.
  • Some tests that intentionally override these env vars still need to use monkeypatch or module reload patterns, as they already do today.

Model Used

GPT-5 Codex

@nesquena-hermes
Copy link
Copy Markdown
Collaborator

Thanks for this, Frank — test isolation for pytest state is a real footgun and this addresses it cleanly.

Review summary — LGTM with one absorb note

The approach is correct: publishing HERMES_WEBUI_STATE_DIR, HERMES_WEBUI_SESSION_DIR, and HERMES_WEBUI_CONFIG_DIR to os.environ at module import time in conftest.py (before api.config is loaded by test modules) is the right fix for the issue where api.config.STATE_DIR could fall back to ~/.hermes/webui in the pytest process.

Specifics that look good:

  • Direct os.environ[...] = str(...) assignment (not setdefault) means the test root cannot be overridden by an inherited shell environment.
  • test_pytest_state_isolation.py locks the invariant with clear assertions — this is exactly the kind of sentinel test that catches future regressions.
  • Scope is narrow: only conftest.py and the new regression test file. No production code touched.

One small absorb at integration time:
The SESSION_DIR env var used in the fix (HERMES_WEBUI_SESSION_DIR) needs to be verified against whatever name api.config actually reads. If api.config derives SESSION_DIR from STATE_DIR without its own env var override, setting HERMES_WEBUI_SESSION_DIR may not be picked up. We'll confirm during stage build and add a reload if needed — no action from you required.

Merge plan: Queued for the next batch release. No further changes needed from you.

@franksong2702 franksong2702 force-pushed the franksong2702/pytest-state-isolation branch from 337a6a0 to 6cd4732 Compare April 27, 2026 03:15
@nesquena-hermes nesquena-hermes added the merge soon Reviewed and queued for next release batch label Apr 27, 2026
@nesquena-hermes
Copy link
Copy Markdown
Collaborator

Clean test-infrastructure fix — the setdefault → direct assignment is the right call for production-risk paths, and the new test_pytest_state_isolation.py correctly locks the invariant. Moving to merge queue.

nesquena-hermes added a commit that referenced this pull request Apr 27, 2026
* feat: attention state for broken cron jobs + Korean i18n (#1133, @franksong2702)

* fix: pytest state isolation for direct session saves (#1136, @franksong2702)

* fix(#1095): image thumbnails in composer + lightbox in chat (#1135)

* fix(css): restore cron attention + detail-alert rules overwritten by style.css merge (absorb)

* docs: v0.50.225 release notes and version bump

---------

Co-authored-by: nesquena-hermes <[email protected]>
@nesquena-hermes
Copy link
Copy Markdown
Collaborator

Merged in v0.50.225 via PR #1137. Thank you @franksong2702 — great contribution (pytest state isolation fix)! 🎉

JKJameson pushed a commit to JKJameson/hermes-webui that referenced this pull request Apr 29, 2026
…#1137)

* feat: attention state for broken cron jobs + Korean i18n (nesquena#1133, @franksong2702)

* fix: pytest state isolation for direct session saves (nesquena#1136, @franksong2702)

* fix(nesquena#1095): image thumbnails in composer + lightbox in chat (nesquena#1135)

* fix(css): restore cron attention + detail-alert rules overwritten by style.css merge (absorb)

* docs: v0.50.225 release notes and version bump

---------

Co-authored-by: nesquena-hermes <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge soon Reviewed and queued for next release batch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants