Repository navigation
[FEATURE]: Add LiteLLM proxy as a built-in provider #29935
Description
Feature hasn't been suggested before.
- I have verified this feature I'm about to request hasn't been suggested before.
Describe the enhancement you want to request
LiteLLM is an open-source proxy that provides a unified OpenAI-compatible API in front of 100+ LLM providers, with load balancing, spend tracking, and rate limiting.
Users can already use LiteLLM via the custom provider flow, but that requires manually adding each model. Since LiteLLM exposes a /v1/models endpoint, a built-in provider could
auto-discover all available models from the proxy at startup.
The integration would work by setting LITELLM_BASE_URL and optionally LITELLM_API_KEY env vars. LiteLLM is a gateway/proxy, so it doesn't fit the models.dev pattern - the available models depend on what the user has configured on their proxy.
I have a working implementation ready to submit as a PR.
Reacted by El AI, Martin Grotzke, Oliver, Pol Montanera, cwatzl and Hinrich B. Winther
Metadata
Metadata
Assignees
Labels
No labels
Activity
github-actions commentedon May 29, 2026
This issue might be a duplicate of existing issues. Please check:
/v1/modelsautoloadoption #13891: [FEATURE]: Auto-load available models from LiteLLM proxy withautoloadoption — same concept of fetching models from LiteLLM's/modelsendpoint at startup (closed as discussion)github-actions commentedon Jul 29, 2026
To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.
stefanlasiewski commentedon Jul 30, 2026
Also see this related issue, also closed: #13891