Skip to content

tests(proxy_server): surface current behavior in tests#29309

Merged
yuneng-berri merged 3 commits into
litellm_internal_stagingfrom
litellm_yj_may29
May 30, 2026
Merged

tests(proxy_server): surface current behavior in tests#29309
yuneng-berri merged 3 commits into
litellm_internal_stagingfrom
litellm_yj_may29

Conversation

@yuneng-berri

Copy link
Copy Markdown
Collaborator

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.
  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Screenshots / Proof of Fix

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

yuneng-berri and others added 3 commits May 29, 2026 15:40
* test(proxy): pin proxy_server.py forwarding-route behavior

PR2 of the proxy_server.py behavior-pinning project: fills the 12
forwarding-route test files added by the harness PR with happy + error
pins for all 52 LLM-facing routes (models, chat/completions, completions,
embeddings, moderations, audio, assistants, threads, utils, model-info,
model-metrics, queue). Every happy-path test asserts the full response
dict via normalize() so the gate enforces real shape pinning rather
than status codes.

* test(proxy): drop task-plumbing comments from PR2 test files

* test(proxy): tighten PR2 error-path status-code pins

Apply the same review feedback Greptile gave on PR1 (#28856) and PR3
(#28850) to PR2's forwarding-route tests:

- Replace permissive `>= 400` / `in (X, Y)` status assertions with the
  exact 500/405 the handler actually returns, so a regression that
  silently shifts the code now fails the pin.
- Add a body-presence check alongside each tightened status assertion
  to satisfy _pin_check.py's no-status-only rule.

---------

Co-authored-by: Claude <[email protected]>
…8856)

* test(proxy): pin proxy_server.py non-route surface behavior (PR1)

Fills the 7 PR1 placeholder files under tests/test_litellm/proxy/proxy_server/
with behavior pins for the non-route surface of proxy_server.py:
lifecycle/init/shutdown, ProxyConfig class methods, DB-overlay config scrubbers,
spend counters, background-health helpers, OpenAPI customization, exception
handlers, and streaming-generator helpers.

233 tests cover 101 pin-list symbols (1+ happy + 1+ error each). New-tests-only
coverage on litellm/proxy/proxy_server.py: 32.80% line / 20.91% branch (PR1
gate: 25% line / 18% branch). Full directory runs in ~22s with -n 4.

Plan: https://www.notion.so/Plan-Pin-proxy_server-py-behavior-2026-05-25-36c43b8acdab81ee845fd5365128a2fc

* test(proxy): address Greptile review comments on test_lifecycle.py

- test_initialize_signature_is_async_with_expected_params: hard-code
  expected_param_count so a signature change actually trips the gate
  (previously both sides of the comparison were len(sig.parameters)).
- test_check_request_disconnection_invalid_when_connected_times_out:
  patch asyncio.sleep so the test no longer spins for ~1.2 s of real
  wall-clock; timeout lowered to 0.05 s.

---------

Co-authored-by: Claude <[email protected]>
* test(proxy/proxy_server): pin misc routes (PR3, partial)

Adds happy + error tests for the misc control-plane routes:
GET /, /routes, /adaptive_router/state, /get_logo_url,
/get_image, /get_favicon.

Also gitignores .pin_list.txt (used by the pin gate).

* test(proxy/proxy_server): pin login/SSO routes (PR3, partial)

Adds happy + error tests for the 5 login/SSO control-plane routes:
GET /fallback/login, POST /login, POST /v2/login, POST /v3/login,
POST /v3/login/exchange. Mocks authenticate_user and
create_ui_token_object at their imported location.

* test(proxy/proxy_server): pin onboarding routes (PR3, partial)

Adds happy + error tests for the 2 onboarding control-plane routes:
GET /onboarding/get_token, POST /onboarding/claim_token. Wires a
MagicMock async context manager for prisma_client.db.tx() and
signs the onboarding JWT with the patched master_key.

* test(proxy/proxy_server): pin model_cost_map reload routes (PR3, partial)

Adds happy + error tests for the 5 model-cost-map control-plane routes:
POST /reload/model_cost_map, POST|DELETE|GET
/schedule/model_cost_map_reload(/status), GET /model/cost_map/source.
Attaches litellm_config to mock_prisma per-test (the table is not in
the default _PRISMA_TABLES fixture).

* test(proxy/proxy_server): pin anthropic_beta_headers reload routes (PR3, partial)

Adds happy + error tests for the 4 anthropic-beta-headers control-plane
routes: POST /reload/anthropic_beta_headers, POST|DELETE|GET
/schedule/anthropic_beta_headers_reload(/status). Stubs
db.litellm_config (not in default _PRISMA_TABLES) and monkeypatches
reload_beta_headers_config so no network calls fire.

* test(proxy/proxy_server): pin invitation routes (PR3, partial)

Adds happy + error tests for the 4 invitation control-plane routes:
POST /invitation/new, GET /invitation/info, POST /invitation/update,
POST /invitation/delete. Patches _user_has_admin_privileges /
_user_has_admin_view to avoid extensive get_user_object mocking.

* test(proxy/proxy_server): pin config CRUD routes (PR3, partial)

Adds happy + error tests for the 8 config-CRUD control-plane routes:
POST /config/update, POST|GET /config/field/update|info, GET /config/list,
POST /config/field/delete, POST /config/callback/delete,
GET /get/config/callbacks, GET /config/yaml. Attaches litellm_config
to mock_prisma per-test.

* test(proxy/proxy_server): tighten pin assertions per review

- test_routes_misc.py: `b"" in response.content` is trivially true;
  replace with `len(response.content) > 0` so an empty 405 body trips
  the gate.
- test_routes_login_sso.py: `len(response.content) >= 0` is trivially
  true; tighten to `> 0`.
- test_routes_anthropic_beta.py: replace brittle string-literal checks
  on the serialized JSON (`'"interval_hours": 12' in payload`) with
  `json.loads` + dict access so the assertion survives any serializer
  spacing.
- test_routes_config.py: `assert status_code in (404, 500)` was too
  permissive; the handler re-raises HTTPException(404) verbatim, so
  pin 404 strictly.

---------

Co-authored-by: Claude <[email protected]>
@codecov

codecov Bot commented May 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@greptile-apps

greptile-apps Bot commented May 29, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fills in the placeholder test files for proxy_server.py behavior-pinning, adding ~9000 lines of mock-based unit tests covering lifecycle helpers, ProxyConfig methods, spend counters, streaming helpers, exception handlers, background health checks, OpenAPI customization, and all major route groups.

  • All 26 test files are replaced with comprehensive happy-path + error-path pins using the shared conftest.py fixtures (normalize, auth_as, mock_prisma, mock_router); no real DB or LLM calls are made.
  • .gitignore picks up .cov_new.xml and .pin_list.txt, the latter duplicated twice in the diff (harmless but should be deduplicated).
  • A test_initialize_shared_aiohttp_session_returns_client_session test creates a live aiohttp.ClientSession/TCPConnector (real socket allocation), and a timeout-based disconnection test uses a 50 ms wall-clock budget that may be tight on loaded CI runners.

Confidence Score: 4/5

Safe to merge — all changes are test-only and do not touch production code paths.

The PR is entirely test additions. One test creates a real OS-level socket connection pool (aiohttp TCPConnector) rather than mocking it, and one timeout-driven test uses a 50 ms wall-clock budget that could produce intermittent failures on slow CI runners. Neither affects production behavior.

tests/test_litellm/proxy/proxy_server/test_lifecycle.py — the aiohttp session and timeout tests discussed above.

Important Files Changed

Filename Overview
.gitignore Adds .cov_new.xml and .pin_list.txt to .gitignore — but .pin_list.txt is added twice (lines 31 and 126), creating a harmless duplicate entry.
tests/test_litellm/proxy/proxy_server/test_lifecycle.py Behavior pins for lifecycle helpers. The timeout-based test for check_request_disconnection uses a 50ms wall-clock limit with a patched sleep; functions as intended but is a potential source of flakiness. All other tests are mock-only with no real network calls.
tests/test_litellm/proxy/proxy_server/test_proxy_config.py Comprehensive behavior pins for ProxyConfig and scrubber helpers. All happy-path and error-path branches covered with strong assertions; entirely mock-based with no real I/O besides tmp_path file operations.
tests/test_litellm/proxy/proxy_server/test_spend_counters.py Thorough behavior pins for all spend-counter helpers including Redis/in-memory interaction paths and edge cases like cache warmth checks, window counters, and invalidation.
tests/test_litellm/proxy/proxy_server/test_streaming_helpers.py Behavior pins for all streaming helper functions covering SSE serialization, chunk restamping, hook application, and generator selection paths.
tests/test_litellm/proxy/proxy_server/conftest.py Shared fixtures: normalize() helper, session-scoped app/client, mock_prisma with all tables stubbed, auth_as context manager, mock_router, and response builder utilities. Well-structured with no issues.

Reviews (1): Last reviewed commit: "test(proxy/proxy_server): pin control-pl..." | Re-trigger Greptile

Comment on lines +153 to +170
async def test_initialize_shared_aiohttp_session_returns_client_session():
from aiohttp import ClientSession

session = await _initialize_shared_aiohttp_session()
try:
observed = {
"is_client_session": isinstance(session, ClientSession),
"is_closed": session.closed,
"has_connector": session.connector is not None,
}
assert normalize(observed) == {
"is_client_session": True,
"is_closed": False,
"has_connector": True,
}
finally:
if session is not None:
await session.close()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 test_initialize_shared_aiohttp_session_returns_client_session calls the real _initialize_shared_aiohttp_session(), which creates a live aiohttp.TCPConnector and ClientSession — allocating a real OS-level socket/connection pool. While no HTTP requests are made, this is a real network resource allocation. Consider mocking aiohttp.ClientSession and TCPConnector to stay in line with the project rule that tests in this directory must not make real network calls, and to keep the test hermetic in air-gapped CI environments.

Rule Used: What: prevent any tests from being added here that... (source)

Comment on lines +198 to +223
monkeypatch.delenv("WORKER_CONFIG", raising=False)

save_worker_config(model="gpt-4", config="/tmp/c.yaml", debug=True)

payload = json.loads(os.environ["WORKER_CONFIG"])
assert normalize(payload) == {
"model": "gpt-4",
"config": "/tmp/c.yaml",
"debug": True,
}


def test_save_worker_config_invalid_no_kwargs_yields_empty(monkeypatch):
monkeypatch.delenv("WORKER_CONFIG", raising=False)

save_worker_config()
assert os.environ["WORKER_CONFIG"] == "{}"


# ---------------------------------------------------------------------------
# initialize
# ---------------------------------------------------------------------------


def test_initialize_signature_is_async_with_expected_params():
sig = inspect.signature(initialize)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 test_check_request_disconnection_invalid_when_connected_times_out relies on a 50 ms wall-clock budget (timeout=0.05). The patched asyncio.sleep reduces each iteration to a single await asyncio.sleep(0), but the event loop still needs to schedule the timeout callback and process the CancelledError. On a heavily-loaded CI runner the 50 ms budget can be exhausted before the loop even completes one full iteration, causing a spurious asyncio.TimeoutError to arrive inside a different await than expected — or the timeout machinery may race against the next loop iteration, making the test flaky. Consider raising the budget to 0.5 s or replacing the wall-clock timeout with a counter-based stop condition.

Comment thread .gitignore
Comment on lines +125 to +126
.vscode
.pin_list.txt

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 .pin_list.txt is added twice in this PR — once near the top (line 31, alongside .cov_new.xml) and again here at the end. The duplicate has no functional effect, but it's unnecessary noise.

Suggested change
.vscode
.pin_list.txt
.vscode

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@yuneng-berri yuneng-berri merged commit 12d29a3 into litellm_internal_staging May 30, 2026
115 of 118 checks passed
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
* test(proxy/proxy_server): pin forwarding routes (PR2) (BerriAI#28887)

* test(proxy): pin proxy_server.py forwarding-route behavior

PR2 of the proxy_server.py behavior-pinning project: fills the 12
forwarding-route test files added by the harness PR with happy + error
pins for all 52 LLM-facing routes (models, chat/completions, completions,
embeddings, moderations, audio, assistants, threads, utils, model-info,
model-metrics, queue). Every happy-path test asserts the full response
dict via normalize() so the gate enforces real shape pinning rather
than status codes.

* test(proxy): drop task-plumbing comments from PR2 test files

* test(proxy): tighten PR2 error-path status-code pins

Apply the same review feedback Greptile gave on PR1 (BerriAI#28856) and PR3
(BerriAI#28850) to PR2's forwarding-route tests:

- Replace permissive `>= 400` / `in (X, Y)` status assertions with the
  exact 500/405 the handler actually returns, so a regression that
  silently shifts the code now fails the pin.
- Add a body-presence check alongside each tightened status assertion
  to satisfy _pin_check.py's no-status-only rule.

---------


* test(proxy): pin proxy_server.py non-route surface behavior (PR1) (BerriAI#28856)

* test(proxy): pin proxy_server.py non-route surface behavior (PR1)

Fills the 7 PR1 placeholder files under tests/test_litellm/proxy/proxy_server/
with behavior pins for the non-route surface of proxy_server.py:
lifecycle/init/shutdown, ProxyConfig class methods, DB-overlay config scrubbers,
spend counters, background-health helpers, OpenAPI customization, exception
handlers, and streaming-generator helpers.

233 tests cover 101 pin-list symbols (1+ happy + 1+ error each). New-tests-only
coverage on litellm/proxy/proxy_server.py: 32.80% line / 20.91% branch (PR1
gate: 25% line / 18% branch). Full directory runs in ~22s with -n 4.

Plan: https://www.notion.so/Plan-Pin-proxy_server-py-behavior-2026-05-25-36c43b8acdab81ee845fd5365128a2fc

* test(proxy): address Greptile review comments on test_lifecycle.py

- test_initialize_signature_is_async_with_expected_params: hard-code
  expected_param_count so a signature change actually trips the gate
  (previously both sides of the comparison were len(sig.parameters)).
- test_check_request_disconnection_invalid_when_connected_times_out:
  patch asyncio.sleep so the test no longer spins for ~1.2 s of real
  wall-clock; timeout lowered to 0.05 s.

---------


* test(proxy/proxy_server): pin control-plane routes (PR3) (BerriAI#28850)

* test(proxy/proxy_server): pin misc routes (PR3, partial)

Adds happy + error tests for the misc control-plane routes:
GET /, /routes, /adaptive_router/state, /get_logo_url,
/get_image, /get_favicon.

Also gitignores .pin_list.txt (used by the pin gate).

* test(proxy/proxy_server): pin login/SSO routes (PR3, partial)

Adds happy + error tests for the 5 login/SSO control-plane routes:
GET /fallback/login, POST /login, POST /v2/login, POST /v3/login,
POST /v3/login/exchange. Mocks authenticate_user and
create_ui_token_object at their imported location.

* test(proxy/proxy_server): pin onboarding routes (PR3, partial)

Adds happy + error tests for the 2 onboarding control-plane routes:
GET /onboarding/get_token, POST /onboarding/claim_token. Wires a
MagicMock async context manager for prisma_client.db.tx() and
signs the onboarding JWT with the patched master_key.

* test(proxy/proxy_server): pin model_cost_map reload routes (PR3, partial)

Adds happy + error tests for the 5 model-cost-map control-plane routes:
POST /reload/model_cost_map, POST|DELETE|GET
/schedule/model_cost_map_reload(/status), GET /model/cost_map/source.
Attaches litellm_config to mock_prisma per-test (the table is not in
the default _PRISMA_TABLES fixture).

* test(proxy/proxy_server): pin anthropic_beta_headers reload routes (PR3, partial)

Adds happy + error tests for the 4 anthropic-beta-headers control-plane
routes: POST /reload/anthropic_beta_headers, POST|DELETE|GET
/schedule/anthropic_beta_headers_reload(/status). Stubs
db.litellm_config (not in default _PRISMA_TABLES) and monkeypatches
reload_beta_headers_config so no network calls fire.

* test(proxy/proxy_server): pin invitation routes (PR3, partial)

Adds happy + error tests for the 4 invitation control-plane routes:
POST /invitation/new, GET /invitation/info, POST /invitation/update,
POST /invitation/delete. Patches _user_has_admin_privileges /
_user_has_admin_view to avoid extensive get_user_object mocking.

* test(proxy/proxy_server): pin config CRUD routes (PR3, partial)

Adds happy + error tests for the 8 config-CRUD control-plane routes:
POST /config/update, POST|GET /config/field/update|info, GET /config/list,
POST /config/field/delete, POST /config/callback/delete,
GET /get/config/callbacks, GET /config/yaml. Attaches litellm_config
to mock_prisma per-test.

* test(proxy/proxy_server): tighten pin assertions per review

- test_routes_misc.py: `b"" in response.content` is trivially true;
  replace with `len(response.content) > 0` so an empty 405 body trips
  the gate.
- test_routes_login_sso.py: `len(response.content) >= 0` is trivially
  true; tighten to `> 0`.
- test_routes_anthropic_beta.py: replace brittle string-literal checks
  on the serialized JSON (`'"interval_hours": 12' in payload`) with
  `json.loads` + dict access so the assertion survives any serializer
  spacing.
- test_routes_config.py: `assert status_code in (404, 500)` was too
  permissive; the handler re-raises HTTPException(404) verbatim, so
  pin 404 strictly.

---------


---------
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants