[Fix] Test Isolation and Path Resolution for GHA Unit Tests#24741
[Fix] Test Isolation and Path Resolution for GHA Unit Tests#24741yuneng-berri merged 2 commits intomainfrom
Conversation
…ath resolution Fix agent health check tests failing with 500 errors in parallel CI by mocking prisma_client to None. Fix documentation validation tests using CWD-relative paths that break depending on the working directory. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
Greptile SummaryThis PR fixes two categories of flaky CI failures: leaked
Confidence Score: 4/5Safe to merge after fixing the stale One P1 bug was introduced: tests/documentation_tests/test_router_settings.py — stale
|
| Filename | Overview |
|---|---|
| tests/documentation_tests/test_router_settings.py | Path resolution correctly switched to __file__-based _repo_root, but the old variable name repo_base is still referenced in the except block (line 64), causing a NameError if the docs file cannot be opened. |
| tests/test_litellm/proxy/agent_endpoints/test_endpoints.py | Adds monkeypatch.setattr("litellm.proxy.proxy_server.prisma_client", None) to prevent a leaked MagicMock from other pytest-xdist workers causing TypeError in the health-check endpoint. Clean, targeted fix that does not weaken test coverage. |
| .github/workflows/test-unit-documentation.yml | Restores the documentation validation workflow, running four scripts from the repo root with pinned action commit SHAs. Straightforward and safe. |
| docs/my-website/docs/proxy/config_settings.md | Adds ~25 new entries to the general_settings and router_settings tables, filling in previously undocumented keys that caused the documentation validation test to fail. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[pytest-xdist worker starts TestAgentHealthCheck] --> B{prisma_client leaked\nfrom another scope?}
B -- Before fix: Yes --> C["prisma_client = MagicMock()"]
C --> D["GET /v1/agents calls find_many()"]
D --> E["TypeError: MagicMock can't\nbe used in await → 500"]
B -- After fix: monkeypatch sets None --> F["prisma_client = None"]
F --> G["GET /v1/agents skips DB query"]
G --> H["200 OK — uses mock_registry only"]
subgraph doc_tests["Documentation Test Path Resolution"]
I["poetry run python ./tests/documentation_tests/test_router_settings.py"]
I --> J["_test_dir = os.path.dirname(os.path.abspath(__file__))"]
J --> K["_repo_root = os.path.abspath(join(_test_dir, '..', '..'))"]
K --> L["docs_path = _repo_root/docs/my-website/..."]
L --> M{File exists?}
M -- Yes --> N["Parse & validate docs"]
M -- No --> O["except block raises Exception\n⚠ repo_base NameError here"]
end
Comments Outside Diff (1)
-
tests/documentation_tests/test_router_settings.py, line 64 (link)repo_baseremoved but still referenced in except blockThe variable
repo_basewas deleted as part of the path-resolution fix (replaced with_repo_root), but line 64 inside theexceptblock still references the old name. If the docs file ever fails to open (e.g., the path is wrong), Python will raise aNameError: name 'repo_base' is not definedinside the exception handler, completely hiding the original error message.
Reviews (2): Last reviewed commit: "[Fix] Scope documentation workflow to ma..." | Re-trigger Greptile
…router settings Revert path fixes for documentation tests that CircleCI never ran (test_exception_types, test_general_setting_keys, test_readme_providers, test_standard_logging_payload). Update the GHA workflow to run only the 4 tests CircleCI actually executed: test_env_keys, test_router_settings, test_api_docs, test_circular_imports. Add 2 missing router_settings keys (enable_health_check_routing, health_check_staleness_threshold) and 27 missing general_settings keys to config_settings.md so test_router_settings passes. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Summary
Failure Path (Before Fix)
Agent health check tests (4 failures):
TestAgentHealthChecktests returned 500 errors when run with parallel workers (pytest-xdist --dist=loadscope). Theget_agents()endpoint importsprisma_clientfromproxy_serverat runtime — when aMagicMockleaked from another test scope on the same worker, the endpoint hitTypeError: object MagicMock can't be used in 'await' expression.Documentation validation tests:
test_router_settings.pyfound 2 undocumented router settings keys (enable_health_check_routing,health_check_staleness_threshold) that were added to the Router but never documented.Fix
Agent health check isolation: Added
monkeypatch.setattr("litellm.proxy.proxy_server.prisma_client", None)to theTestAgentHealthCheck._setupfixture so the endpoint skips DB queries regardless of leaked module-level state from other test scopes.Documentation sync: Added 2 missing router_settings keys and 27 missing general_settings keys to
config_settings.md.Documentation workflow: Re-added
test-unit-documentation.ymlrunning exactly the 4 tests that CircleCI ran:test_env_keys.py,test_router_settings.py,test_api_docs.py,test_circular_imports.py. Uses directpythonexecution matching CircleCI behavior. Path resolution updated to use__file__-based paths intest_router_settings.pyso tests work regardless of working directory.Changes
tests/test_litellm/proxy/agent_endpoints/test_endpoints.py: Mockprisma_clienttoNoneinTestAgentHealthCheckfixturetests/documentation_tests/test_router_settings.py:__file__-based path resolutiondocs/my-website/docs/proxy/config_settings.md: Add 27 general_settings + 2 router_settings keys.github/workflows/test-unit-documentation.yml: New workflow matching CircleCI's documentation testsTesting
test_env_keys.py,test_router_settings.py,test_api_docs.py,test_circular_imports.pyType
🐛 Bug Fix
📖 Documentation
🚄 Infrastructure
✅ Test