Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

@greptileai

Greptile Summary

Fixes the _registered_pass_through_routes dict growing without bound across every 30-second scheduler reload cycle. The root cause was that the stale-entry cleanup called remove_endpoint_routes(endpoint_key), which matched by endpoint_id inside the dict values — never by the route key itself — so nothing was ever deleted; the fix replaces that call with a direct _registered_pass_through_routes.pop(endpoint_key, None).

• Production change (1 line): In initialize_pass_through_endpoints, the stale-route loop now pops the exact key from the registry in O(1) rather than scanning values by endpoint_id, which never matched because route keys encode a per-cycle UUID. LiteLLMRoutes.openai_routes is intentionally left alone: its append is path-deduped so it cannot grow unboundedly, and on a reload the path is still owned by the freshly re-registered live endpoint.
• Tests (3 new async tests): TestStaleRouteCleanupOnReload drives initialize_pass_through_endpoints across 25 cycles to assert the registry stays at 2 entries, confirms a departed endpoint is evicted on the next reload, and verifies the live route remains resolvable after repeated reloads. All network/IO is mocked via ExitStack patches.

Confidence Score: 5/5

Safe to merge — the change is a single-line targeted fix to the stale-route cleanup loop, backed by three regression tests that would fail on the unfixed code.

The production change is minimal and precise: one line replaces a call that silently did nothing with a direct dict pop. The root cause analysis in the PR description is accurate and traceable in the code. The three new tests exercise the exact failure scenarios (bounded growth, departed-endpoint removal, live-route survivability) using real function calls against properly mocked I/O, and would catch a regression. No existing tests are modified or weakened. The deliberate decision to leave openai_routes un-cleaned is sound — that list is append-deduped so it cannot grow, and removing the path for a same-cycle re-registration would break the live endpoint.

No files require special attention.

_{Reviews (1): Last reviewed commit: "fix(pass-through): remove stale routes b..." | Re-trigger Greptile}

Greptile Summary

This PR fixes a CPU-sink regression where _registered_pass_through_routes grew by two entries every 30-second reload cycle for any id-less pass-through endpoint, because the stale-key cleanup called remove_endpoint_routes(route_key) — a function that matches on endpoint_id, not on a full route key — so it never matched anything.

• Core fix (one line change): the cleanup loop now calls _registered_pass_through_routes.pop(endpoint_key, None) directly, which is O(1) and correct. The LiteLLMRoutes.openai_routes cleanup is intentionally skipped because that path is append-deduped and is still valid for the live endpoint re-registered in the same cycle.
• Tests: three new async regression tests drive initialize_pass_through_endpoints through multiple reload cycles and assert the registry stays bounded, stale entries are evicted, and live routes remain resolvable. All tests use mocks and make no real network calls.

Confidence Score: 5/5

Safe to merge. The change is minimal and surgical, touching only the stale-key eviction path in the 30-second reload loop and adding purely additive tests.

The root cause analysis is accurate, the one-line fix is correct, and the three new tests directly exercise the broken path and would fail against the pre-fix code. The intentional omission of openai_routes cleanup for departed auth-required endpoints is well-documented and represents a pre-existing gap that was already broken (the old call never matched). No backward-incompatible behavior changes, no security implications, no new DB queries in the critical path.

No files require special attention. Both changed files are straightforward and well-reasoned.

_{Reviews (2): Last reviewed commit: "fix(pass-through): remove stale routes b..." | Re-trigger Greptile}

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!