Skip to content

litellm_fix(perf): skip global router_settings in per-request Router creation#20133

Closed
ghost wants to merge 1 commit into
mainfrom
unknown repository
Closed

litellm_fix(perf): skip global router_settings in per-request Router creation#20133
ghost wants to merge 1 commit into
mainfrom
unknown repository

Conversation

@ghost

@ghost ghost commented Jan 31, 2026 •
edited by ghost
Loading

Copy link
Copy Markdown

Problem

┌─────────────────────────────────────────────────────────────┐
│                     STARTUP                                 │
│  Global router_settings → Applied to shared llm_router ✅   │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                   EVERY REQUEST                             │
│  _get_hierarchical_router_settings() returns global settings│
│           ↓                                                 │
│  Creates NEW Router (redundant - already on shared!) ❌     │
└─────────────────────────────────────────────────────────────┘

Issue: Global router_settings in DB trigger per-request Router creation, but they're already applied to the shared llm_router at startup.

Fix

Remove global settings lookup from _get_hierarchical_router_settings():

- # 3. Try global router_settings
- db_router_settings = await prisma_client.db.litellm_config.find_first(...)
- return db_router_settings.param_value
+ # Skip global - already on shared router from startup
+ return None

Now only key/team-specific settings create per-request Routers.

Related

@vercel

vercel Bot commented Jan 31, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Error Error Jan 31, 2026 8:41am

Request Review

@CLAassistant

CLAassistant commented Jan 31, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@ghost ghost mentioned this pull request Jan 31, 2026
Closed
1 task
litellm_fix(perf): skip global router_settings lookup
20aad2e 
Global settings already on shared router from startup.
Removes redundant per-request Router creation.

Issue #19921
@ghost ghost closed this Jan 31, 2026
@ghost

ghost commented Jan 31, 2026

Copy link
Copy Markdown
Author

Closing - need deeper investigation first. The fix may be incomplete (DB query still happens on every request). Will reopen with proper fix after full analysis.

cursor Bot pushed a commit that referenced this pull request Feb 4, 2026
@cursoragent @ishaan-jaff
docs: Add troubleshooting guide for DB connection failures (#19921)
4c9eab7 
This documents the root causes and solutions for the 'All connection attempts failed'
error that users experienced after upgrading to v1.81.x:

1. Per-request Router creation with expensive Prometheus initialization (fixed in #20087)
2. Redundant Router creation for global settings (fixed in #20133)
3. DB connection pool exhaustion with IAM auth (fixed in #13140)
4. Connection pool configuration recommendations

Includes SQL queries to diagnose/clean router_settings and config recommendations.

Co-authored-by: Ishaan Jaff <[email protected]>
@yassin-berriai yassin-berriai mentioned this pull request Jun 25, 2026
Merged
4 tasks
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@CLAassistant