Skip to content

providers: vLLM discovery silently drops every model whose owned_by is not "vllm" #57460

Open

Description

@fede-ciliberti

Summary

The built-in vllm provider discovers models from /health + /v1/models, then silently drops every entry whose owned_by is not exactly vllm or that carries no max_model_len. Any OpenAI-compatible gateway that reports its real owners (LiteLLM, Portkey, GoModel, a plain nginx-fronted upstream) is therefore invisible, with no error in the logs, and the operator has to write the model list by hand.

Environment

  • opencode version: 2.0.32
  • OS: Linux 6.18.51-brainykernel-zen3 (linux x64)
  • Terminal: TERM_PROGRAM=ghostty, TERM=xterm-256color, COLORTERM=truecolor
  • Shell: /usr/bin/zsh
  • Install/channel: latest
  • Active plugins: oh-my-openagent-v2, opencode-skill-creator (unrelated to the failure; it reproduces with an isolated project config)

Reproduction

  1. Serve an OpenAI-compatible /v1/models on http://localhost:8090/v1 whose entries look like a normal gateway:
{"object":"list","data":[
  {"id":"my-model","object":"model","owned_by":"anthropic","created":1792503066,
   "metadata":{"context_window":1000000}}
]}
  1. Serve GET /health returning {"status":"ok"} on the same host.
  2. Point OpenCode at it in a project opencode.json:
{
  "$schema": "https://opencode.ai/config.json",
  "providers": { "vllm": { "settings": { "baseURL": "http://localhost:8090/v1" } } }
}
  1. Run opencode models.

Expected Behavior

The discovered model appears as vllm/my-model. The runtime already reached the endpoint successfully: with request logging on, OpenCode performs GET /health and GET /v1/models, both 200.

Actual Behavior

No vllm/* model is listed and nothing is logged as an error. The discovery result is discarded silently.

Bisecting the payload one field at a time, with everything else held constant:

/v1/models entry Listed
owned_by: "anthropic" (gateway's real owner) no
owned_by: "gomodel" no
owned_by: "vllm", no max_model_len no
owned_by: "vllm" + max_model_len yes
owned_by: "vllm" + max_model_len, keeping the extra metadata object yes

So both conditions are required, and metadata, created and object are irrelevant.

Additional Context

  • owned_by is informational in the OpenAI schema. Requiring the literal vllm makes the runtime unusable against every gateway that reports a real owner, which is the common case: LiteLLM reports the provider, Portkey reports the upstream, GoModel reports the provider instance. Serving owned_by: "vllm" is a lie only a gateway maintainer can opt into.
  • Requiring max_model_len is reasonable for a real vLLM server, but a gateway that has no such field still serves the models; treating it as a hard filter hides them.
  • Workaround used meanwhile: the gateway emits owned_by: "vllm" and fills max_model_len from its model catalog behind an opt-in flag. Once this filter is relaxed, that flag becomes unnecessary. Verified end to end: with the two fields present, OpenCode lists the model and an agent runs shell tools through it.
  • Related: V2: Reconcile provider catalogs with /models endpoints #40524 (reconcile provider catalogs with /models), Feature: Dynamic context length detection for OpenAI-compatible providers #40908 (dynamic context length detection), Built-in vLLM discovery causes recurring 401 requests to oMLX on port 8000 #54347 (vLLM discovery probing port 8000).

A suggestion: when discovery receives entries but publishes none, log a single debug/warn line explaining why (e.g. skipped N models: owned_by is not "vllm"). Silent discard is what made this take a full payload bisect to diagnose.

Activity

  1. opencode-agent commented on Oct 20, 2026

    @opencode-agent
    Contributor

    Thanks for the detailed report and the field-by-field bisect, it made this easy to trace.

    The built-in vllm provider is written for real vLLM servers, which always report owned_by: "vllm" and max_model_len. Its discovery (packages/core/src/plugin/provider/vllm.ts) checks the owner on purpose and treats max_model_len as required. The gap you hit is that gateways reporting their real owners can't use it, and nothing is logged when their models are dropped. I've filed this as a feature request: relax those checks or make them optional, and log a line when discovered entries are skipped. Maintainers will consider it.

    The more general fix for gateways is discovery for any configured OpenAI-compatible provider. These are related:

    Until then, your gateway workaround (owned_by: "vllm" plus max_model_len) or a hand-written model list in a custom OpenAI-compatible provider are the options.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions