feat(core): add LiteLLM provider by misbahsy · Pull Request #54558 · anomalyco/opencode

misbahsy · 2026-10-11T17:07:56Z

Issue for this PR

Type of change

Bug fix
New feature
Refactor / code improvement
Documentation

What does this PR do?

I add LiteLLM as a built-in provider with the id litellm, with tests and docs. It reuses the local-provider code that vLLM and Ollama use.

OpenCode reads /v1/models from http://127.0.0.1:4000 in the background. A user can set providers.litellm.settings.baseURL and an optional apiKey. It sends the key as a Bearer token. I do not hardcode a model catalog.

#48938 asks for this. PR #29937 closed because of the V2 move, so this change targets V2.

The models.dev entry is a separate PR. This change does not need it. Gateway cost (#43818, #52127) is out of scope.

A proxy with auth needs apiKey in config. /v1/models does not report tool or image support, so discovered models start with both off.

How did you verify your code works?

bun test test/plugin/provider-litellm.test.ts test/plugin/provider-vllm.test.ts in packages/core: 7 pass.
bun run typecheck in packages/core: errors only in unrelated files.
prettier --check passes on the new TypeScript files. oxlint with default rules finds nothing. I did not run the repo oxlint config.
The tests use a mock server. I did not run a real LiteLLM proxy.

Screenshots / recordings

No UI change.

Checklist

I have tested my changes locally
I have not included unrelated changes in this PR

If you do not follow this template your PR will be automatically rejected.

Added by the opencode bot (AI-generated).

Fixes #48938

Related: #53235, #47480, #52127, #43818

Public API: adds the built-in litellm provider (plugin opencode.provider.litellm, providers.litellm.settings.baseURL/apiKey), which auto-discovers models from 127.0.0.1:4000 (backwards compatible)

github-actions · 2026-10-11T17:08:21Z

The following comment was made by an LLM, it may be inaccurate:

opencode-agent

This works as described. I ran a mock LiteLLM proxy on 127.0.0.1:4000 and ran opencode models from source. On the base, no litellm models appear. With this PR, chat models are discovered, the embedding model is left out, and max_input_tokens becomes the context limit. I also tried a config that already defines a custom providers.litellm provider (headers, apiKey, models with tools/image capabilities, and a model the proxy doesn't list). All of those settings were kept, and discovery sent the Bearer key. The LiteLLM and vLLM tests pass, and typecheck in packages/core is clean.

Public API: this adds the built-in provider ID litellm and the plugin ID opencode.provider.litellm. Settings are providers.litellm.settings.baseURL/apiKey, and discovery can be turned off with -opencode.provider.litellm. OpenCode now also checks 127.0.0.1:4000 by default. It is backwards compatible: existing custom litellm provider configs keep working, and I tested this.

Two optional suggestions, noted inline: the default check on port 4000 could confirm the server is actually LiteLLM, and the docs could explain how to turn tools and images back on.

Fixes #48938
Related: #53235, #47480, #52127, #43818

opencode-agent · 2026-10-11T17:12:21Z

+  discover: (client) =>
+    Effect.gen(function* () {
+      const response = yield* client.getJson("/v1/models", Response)
+      return response.data


This now checks 127.0.0.1:4000 every 30s for every user, and port 4000 is a common dev-server port (Phoenix, Jekyll, …). The vLLM plugin checks /health and only accepts models where owned_by === "vllm", so other servers aren't mistaken for it. Here, any OpenAI-compatible server on port 4000 would appear as "LiteLLM". Could you first check something only LiteLLM serves, such as the unauthenticated /health/liveliness endpoint, before reading /v1/models?

opencode-agent · 2026-10-11T17:12:22Z

+OpenCode refreshes `/v1/models` in the background. It uses `max_input_tokens` as the context limit when the proxy
+reports it, and it excludes models whose `mode` is not `chat`.
+
+Discovered models advertise text input and output, but not vision or tools. `/v1/models` does not report what the model


Most LiteLLM proxies sit in front of models that support tools and images (Claude, GPT, …). With input: ["text"], pasted images are replaced with a text placeholder (see #47480). Overriding this per model in providers.litellm.models.<id>.capabilities works: I tested it, and the override is kept after discovery. A short example of that override here would save users from getting stuck.

misbahsy · 2026-10-20T14:59:06Z

Thanks for the review. I changed both points:

Port 4000 check. Before it reads /v1/models, the plugin now checks /health/liveliness. This endpoint does not need a key, and only LiteLLM returns "I'm alive!" there. If another server on port 4000 gives a different response, OpenCode does not show it as LiteLLM. A new test covers a server that returns 200 for all paths and also serves /v1/models.
Tools and images. The LiteLLM docs now show how to set capabilities for a discovered model. Discovery keeps this setting.

The LiteLLM and vLLM tests pass, and typecheck in packages/core is clean.

old-square-eyes · 2026-10-21T09:48:25Z

Thanks for implementing this. One missing piece for local coding-agent use is enriching discovery with LiteLLM’s /model/info metadata.
With aliases such as primary and default, /v1/models may provide only an ID. That means:

max_input_tokens is absent, so OpenCode uses a fallback rather than the deployed context limit.
Tool support remains disabled, even when the backend supports function calling.
Output limits and reasoning capabilities aren’t discovered.
For example, my backend serves Qwen3.8-27B with 32K context, 4K maximum output and working tool calls. Discovering its alias alone doesn’t give OpenCode enough information to use it correctly as a coding agent.
Could this also read /model/info, matching model_name to the discovered ID, and map:
max_input_tokens / max_output_tokens
supports_function_calling
supports_reasoning
Supported input/output modalities
Ideally, discovery would reuse saved provider credentials, with /model/info enrichment optional when the client key lacks access—without requiring a LiteLLM master key or losing the basic model list.

Another OpenAI-compatible server on port 4000 is no longer shown as LiteLLM. The docs show how to set tools and image input for a discovered model.

old-square-eyes · 2026-11-06T08:25:30Z

Any comments on my suggestion? @misbahsy

github-actions · 2026-11-11T22:21:43Z

Automated PR Cleanup

Thank you for contributing to opencode.

Due to the high volume of PRs from users and AI agents, we periodically close older PRs using automated criteria so maintainers can focus review time on the most active and community-supported contributions.

This PR was closed because it matched the following cleanup criteria:

The PR was created more than 1 month ago
The PR had fewer than 2 positive reactions
Positive reactions are counted as thumbs-up, heart, celebration, or rocket reactions on the PR

PRs created within the last month are not affected by this cleanup.

If you believe this PR was closed incorrectly, or if you are still actively working on it, please leave a comment explaining why it should be reopened. A maintainer can review and reopen it if appropriate.

Thanks again for taking the time to contribute.

opencode-agent Bot reviewed Oct 11, 2026

View reviewed changes

This was referenced Oct 14, 2026

feat(core): discover gateway model metadata and request defaults #53713

Closed

[FEATURE]: Connect OpenCode to centrally managed gateway models and settings #55354

Open

opencode-agent Bot mentioned this pull request Oct 19, 2026

feat(core): discover models for configured OpenAI-compatible providers #57161

Open

4 of 6 tasks

opencode-agent Bot mentioned this pull request Oct 20, 2026

providers: vLLM discovery silently drops every model whose owned_by is not "vllm" #57460

Open

misbahsy added 2 commits November 6, 2026 00:13

feat(core): add LiteLLM provider

bfd63ed

fix(core): check LiteLLM liveliness before model discovery

348de8b

Another OpenAI-compatible server on port 4000 is no longer shown as LiteLLM. The docs show how to set tools and image input for a discovered model.

misbahsy force-pushed the litellm-provider branch from 7709342 to 348de8b Compare November 6, 2026 07:13

test(core): pass plugin ID to PluginHost.make in LiteLLM tests

054a84b

github-actions Bot closed this Nov 11, 2026

github-actions Bot added the automated-pr-cleanup label Nov 11, 2026

Navigation Menu

Repository navigation

feat(core): add LiteLLM provider - #54558

feat(core): add LiteLLM provider#54558
misbahsy wants to merge 3 commits into
anomalyco:devfrom
misbahsy:litellm-provider

misbahsy commented Oct 11, 2026 •

edited by opencode-agent Bot

Loading

Uh oh!

github-actions Bot commented Oct 11, 2026

Uh oh!

opencode-agent Bot left a comment

Uh oh!

opencode-agent Bot Oct 11, 2026

Uh oh!

opencode-agent Bot Oct 11, 2026

Uh oh!

misbahsy commented Oct 20, 2026

Uh oh!

old-square-eyes commented Oct 21, 2026

Uh oh!

old-square-eyes commented Nov 6, 2026

Uh oh!

github-actions Bot commented Nov 11, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

misbahsy commented Oct 11, 2026 • edited by opencode-agent Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue for this PR

Type of change

What does this PR do?

How did you verify your code works?

Screenshots / recordings

Checklist

Uh oh!

github-actions Bot commented Oct 11, 2026

Uh oh!

opencode-agent Bot left a comment

Choose a reason for hiding this comment

Uh oh!

opencode-agent Bot Oct 11, 2026

Choose a reason for hiding this comment

Uh oh!

opencode-agent Bot Oct 11, 2026

Choose a reason for hiding this comment

Uh oh!

misbahsy commented Oct 20, 2026

Uh oh!

old-square-eyes commented Oct 21, 2026

Uh oh!

old-square-eyes commented Nov 6, 2026

Uh oh!

github-actions Bot commented Nov 11, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

misbahsy commented Oct 11, 2026 •

edited by opencode-agent Bot

Loading