Skip to content

fix(security): Clear-text storage of sensitive information#1004

Merged
mpawlow merged 2 commits into
mainfrom
mp/fix/GH-938-cleartext-opensearch-password-storage
Feb 23, 2026
Merged

fix(security): Clear-text storage of sensitive information#1004
mpawlow merged 2 commits into
mainfrom
mp/fix/GH-938-cleartext-opensearch-password-storage

Conversation

@mpawlow

@mpawlow mpawlow commented Feb 19, 2026

Copy link
Copy Markdown
Collaborator

Issues

Summary

Hardens .env file handling in EnvManager to prevent cleartext secrets from being exposed via insecure file permissions. All .env file writes now use os.open with 0o600 mode to restrict access to the file owner only, and adds fsync to ensure data durability. Also removes trailing whitespace throughout the file.

Security Hardening

  • Replace open() with os.open(..., 0o600) + os.fdopen() for all .env file writes, ensuring owner-only (read/write) permissions on creation
  • Add os.chmod(self.env_file, 0o600) when overwriting pre-existing .env files to retroactively restrict permissions
  • Add f.flush() + os.fsync() calls to the main save_env_file() write path to guarantee data is durably written to disk
  • Apply os.chmod(0o600) to the migrated .env after shutil.copy2 in the
    legacy migration branch (init), which previously inherited the
    source file's permissions.
  • Apply os.chmod(0o600) to the timestamped backup file created in
    save_env_file before the new .env is written, ensuring the backup is
    also protected.

Logging Improvements

  • Elevate OPENRAG_VERSION update error from logger.debug to logger.error so failures surface in standard log output

Code Cleanup

  • Remove redundant import os statement in ensure_version_in_env() (already imported at module level)
  • Strip trailing whitespace on blank lines throughout the file

Tests

  • Add tests/unit/test_env_manager.py with 168 lines of unit tests covering
    all three affected code paths:
    • TestSaveEnvFilePermissions: new file creation, overwrite of a permissive
      existing file, and backup file permissions.
    • TestEnsureOpenragVersionPermissions: update of an existing permissive
      file and creation of a new file.
    • TestLegacyMigrationPermissions: migrated file receives 0o600 after copy.
  • All tests use pytest tmp_path and unittest.mock; no running infrastructure
    required. Tests are skipped on Windows (Unix permission model only).

Build / Developer Experience

  • Add test-unit Makefile target (uv run pytest tests/unit/ -v) for running
    unit tests in isolation without triggering integration tests.
  • Register test-unit in .PHONY and add it to the help_test output.

Full Background Info on Remediation


Related PRs


Test Coverage

Test 1 — save_env_file: new file creation

  • No .env exists.
  • Call save_env_file().
  • Assert file exists and stat.S_IMODE == 0o600.

Test 2 — save_env_file: new .env when prior file exists (permissions)

  • Create a pre-existing .env with 0o644 permissions.
  • Call save_env_file().
  • Assert the new .env has 0o600.

Test 3 — save_env_file: backup file permissions

  • Create a pre-existing .env with 0o644 permissions.
  • Call save_env_file().
  • Glob for .env.backup.* in tmp_path.
  • Assert exactly one backup exists and its permissions are 0o600.

Test 4 — ensure_openrag_version: existing file update path

  • Create an .env file with 0o644 permissions containing no OPENRAG_VERSION line.
  • Mock get_current_version() to return "1.2.3".
  • Call env_manager.ensure_openrag_version().
  • Assert file permissions are 0o600 and file contains OPENRAG_VERSION='1.2.3'.

Test 5 — ensure_openrag_version: new file creation path

  • No .env exists.
  • Mock get_current_version() to return "1.2.3".
  • Call env_manager.ensure_openrag_version().
  • Assert file is created with 0o600 permissions.

Test 6 — Legacy migration path in init

  • Create a legacy_env file in tmp_path with 0o644 permissions and some content.
  • Mock get_tui_env_file() to return a path that does not exist.
  • Mock get_legacy_paths() to return {"tui_env": legacy_env}.
  • Instantiate EnvManager() (no explicit env_file).
  • Assert the migrated file exists and has 0o600 permissions.

6 tests across 3 classes:

┌───────────────────────────────────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────────────┐
│                                         Test                                          │                              Verifies                              │
├───────────────────────────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┤
│ TestSaveEnvFilePermissions::test_new_file_creation_has_secure_permissions             │ New .env created via save_env_file gets 0o600                      │
├───────────────────────────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┤
│ TestSaveEnvFilePermissions::test_overwrite_existing_file_has_secure_permissions       │ New .env written over a permissive 0o644 file gets 0o600           │
├───────────────────────────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┤
│ TestSaveEnvFilePermissions::test_backup_file_has_secure_permissions                   │ Timestamped backup of a permissive 0o644 file gets 0o600           │
├───────────────────────────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┤
│ TestEnsureOpenragVersionPermissions::test_existing_file_update_has_secure_permissions │ Updating OPENRAG_VERSION in a permissive 0o644 file enforces 0o600 │
├───────────────────────────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┤
│ TestEnsureOpenragVersionPermissions::test_new_file_creation_has_secure_permissions    │ New .env created by ensure_openrag_version gets 0o600              │
├───────────────────────────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────┤
│ TestLegacyMigrationPermissions::test_migrated_file_has_secure_permissions             │ Legacy 0o644 .env copied via __init__ migration path gets 0o600    │
└───────────────────────────────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────────┘

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request addresses a security vulnerability (issue #938) related to clear-text storage of sensitive information in .env files. Instead of removing secrets from the .env file (as suggested in PR #937), this PR takes the approach of hardening file permissions to restrict access to the file owner only.

Changes:

  • Replaces standard open() calls with os.open() using mode 0o600 for all .env file writes to ensure owner-only read/write permissions
  • Adds flush() and fsync() calls to guarantee data durability when writing configuration files
  • Elevates OPENRAG_VERSION update error logging from debug to error level for better visibility

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/tui/managers/env_manager.py
Comment thread src/tui/managers/env_manager.py
Comment thread src/tui/managers/env_manager.py
Comment thread src/tui/managers/env_manager.py
@edwinjosechittilappilly

Copy link
Copy Markdown
Collaborator

@rodageve Looking forward to your thoughts on it too?

BE side with the edge case migrations I am all good, looping you in for the Cloud scenario.

@edwinjosechittilappilly edwinjosechittilappilly left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Issues

- #938

Summary

Hardens .env file handling in EnvManager to prevent cleartext secrets from being exposed via insecure file permissions. All .env file writes now use os.open with 0o600 mode to restrict access to the file owner only, and adds fsync to ensure data durability. Also removes trailing whitespace throughout the file.

Security Hardening

- Replace `open()` with `os.open(..., 0o600)` + `os.fdopen()` for all .env
  file writes, ensuring owner-only (read/write) permissions on creation
- Add `os.chmod(self.env_file, 0o600)` when overwriting pre-existing .env
  files to retroactively restrict permissions
- Add `f.flush()` + `os.fsync()` calls to the main `save_env_file()` write
  path to guarantee data is durably written to disk

Logging Improvements

- Elevate `OPENRAG_VERSION` update error from `logger.debug` to
  `logger.error` so failures surface in standard log output

Code Cleanup

- Remove redundant `import os` statement in `ensure_version_in_env()`
  (already imported at module level)
- Strip trailing whitespace on blank lines throughout the file
Issues

- #938

Summary

Two code paths in EnvManager left .env files with uncontrolled permissions
after writing them. This commit adds the missing chmod calls so that every
path — legacy migration, backup creation, and new-file creation — always
results in owner-only (0o600) access, preventing cleartext secret exposure
to other OS users. A new unit test target is also added to the Makefile for
faster feedback on unit-only test runs.

Security Fixes

- Apply os.chmod(0o600) to the migrated .env after shutil.copy2 in the
  legacy migration branch (__init__), which previously inherited the
  source file's permissions.
- Apply os.chmod(0o600) to the timestamped backup file created in
  save_env_file before the new .env is written, ensuring the backup is
  also protected.

Tests

- Add tests/unit/test_env_manager.py with 168 lines of unit tests covering
  all three affected code paths:
  - TestSaveEnvFilePermissions: new file creation, overwrite of a permissive
    existing file, and backup file permissions.
  - TestEnsureOpenragVersionPermissions: update of an existing permissive
    file and creation of a new file.
  - TestLegacyMigrationPermissions: migrated file receives 0o600 after copy.
- All tests use pytest tmp_path and unittest.mock; no running infrastructure
  required. Tests are skipped on Windows (Unix permission model only).

Build / Developer Experience

- Add test-unit Makefile target (uv run pytest tests/unit/ -v) for running
  unit tests in isolation without triggering integration tests.
- Register test-unit in .PHONY and add it to the help_test output.
@mpawlow mpawlow force-pushed the mp/fix/GH-938-cleartext-opensearch-password-storage branch from 84dfb39 to e1e4234 Compare February 20, 2026 17:28
@mpawlow

mpawlow commented Feb 20, 2026

Copy link
Copy Markdown
Collaborator Author

Status Update

  • Added an additional commit: e1e4234
  • ✅ Addresses all Copilot feedback
    • ✅ Did addtional Security Hardening
    • ✅ Implemented 6 new Tests
    • ✅ Improved Build / Developer Experience
  • See PR description for full details

@mpawlow

mpawlow commented Feb 20, 2026

Copy link
Copy Markdown
Collaborator Author

Integration Test Failures

  • I think these failures are independent of the changes in this PR
    • i.e. Other PRs are failing with the same error
    • Problem occurred after rebasing off of the latest from main
FAILED tests/integration/test_api_endpoints.py::test_upload_and_search_endpoint[True] - AssertionError: {"error":"Failed to embed with model text-embedding-3-small"}
assert 500 == 200
 +  where 500 = <Response [500 Internal Server Error]>.status_code
FAILED tests/integration/test_api_endpoints.py::test_upload_and_search_endpoint[False] - AssertionError: {"error":"Failed to embed with model text-embedding-3-small"}
assert 500 == 200
 +  where 500 = <Response [500 Internal Server Error]>.status_code
======= 2 failed, 4 passed, 2 skipped, 66 warnings in 206.15s (0:03:26) ========
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0xf1687976b4d0>
make: *** [Makefile:709: test-ci-local] Error 1

@mpawlow

mpawlow commented Feb 20, 2026

Copy link
Copy Markdown
Collaborator Author

Status Update

  • ✅ CI Integration test failures resolved
  • OpenAI API key used in the CI system ran out of credits

@rodageve

Copy link
Copy Markdown
Collaborator

@rodageve Looking forward to your thoughts on it too?

BE side with the edge case migrations I am all good, looping you in for the Cloud scenario.

Applying 0600 file permissions to .env is a good practice but provides a low level of overall security in the context of a kubernetes environment. This is probably the minimum we should do and does provide some level of security.

@mpawlow mpawlow merged commit f66118d into main Feb 23, 2026
4 of 7 checks passed
@mpawlow mpawlow deleted the mp/fix/GH-938-cleartext-opensearch-password-storage branch February 23, 2026 16:08
@edwinjosechittilappilly

Copy link
Copy Markdown
Collaborator

@mpawlow can you try the integration test in this PR?

@mpawlow

mpawlow commented Feb 23, 2026

Copy link
Copy Markdown
Collaborator Author

@edwinjosechittilappilly

  • My apologies. I already merged this PR and deleted this branch.
  • As a result, I'm unable to re-run the integration tests in the CI system
  • Did you want me to attempt to run the integration tests in my local development environment?
    • Note: That 6 integration tests always fail / timeout when run locally

@edwinjosechittilappilly edwinjosechittilappilly added the bug 🔴 Something isn't working. label Feb 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug 🔴 Something isn't working.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Clear-text storage of sensitive information

4 participants