Repository navigation
Conversation
|
The following comment was made by an LLM, it may be inaccurate: |
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Works for the case it targets. I ran serve from source against a local /models stub. On the base, a custom openai-compatible model keeps the 200000/32000 defaults. With this PR it picks up the advertised context_length / max_output_tokens, and configured limit fields still win. The new tests pass and typecheck is clean.
Public API: unchanged (no new config keys, endpoints, events or plugin API). It does add a default-on outbound GET {baseURL}/models for every available provider whose package is @opencode/ai/providers/openai-compatible.
Concerns, most important first (details inline):
- Stored credentials are ignored. Auth comes only from
settings.apiKeyor anauthorizationheader in config. Keys connected throughopencode auth/ integrations (resolved at request time, as the Modal plugin does withctx.integration.connection) are never sent. Most authenticated custom gateways (LiteLLM, OpenRouter-style proxies) will return 401, and discovery silently does nothing. - Not limited to custom providers. Any available provider on this package qualifies, including the built-in OpenCode Zen provider (it now fetches
https://opencode.ai/zen/v1/modelson every startup) and models.dev providers mapped to the generic package. - Real limits can be overwritten. A limit counts as a placeholder whenever it equals the
Model.Infodefault (200000 / 32000). Those are common real values: Zen'smimo-v2.6-flash-freehas exactly{context: 200000, output: 32000}from the catalog. They would then be replaced by any advertisement, which contradicts the precedence stated in the docs and the code comment.
Fixes #40908
|
Pre-landing gate run on head openai-compatible-limits against dev (contributor-side; supersedes closed #53236; leaving review/merge to maintainers). Verdict: ready. No blocking findings. Ponytail pass: nothing to delete (no new config knobs, shared HttpClient effect rather than hand-rolled HTTP, no duplication of the models.dev path, plugin gating correctly scoped to custom openai-compatible providers). Verified: fail-open on discovery errors (4s timeout, keep-last-inventory, 60s failure TTL); 5-minute in-memory success cache shared per endpoint; reload loop converges; transform ordering correct (ProviderPlugins pre, ConfigProviderPlugin post); fill-only semantics never overwrite real models.dev limits and a manual config limit still wins. Tests: new suite 9/9 pass (7 unit + 2 live Bun.serve integration); typecheck clean. Notes for the thread:
|
…iders
Custom OpenAI-compatible providers are not in the models.dev catalog, so
their models keep the built-in placeholder limits and auto-compaction
fires against a guess instead of the real window. Request
GET {baseURL}/models once per endpoint and fill context, input, and
output from the advertised context_length, max_input_tokens, and
max_output_tokens when those fields are still placeholders.
Configured limits win per field through the config provider transform
that runs after this plugin, and limits from models.dev or dedicated
discovery plugins are never replaced. Discovery fails open: an
unreachable or malformed endpoint keeps the last known limits and never
blocks the catalog.
Ports #53236 to V2.
…via connections Discovery matched every provider on the openai-compatible package, so bundled catalog providers (zen free models, models.dev entries mapped to the generic package) were probed on every startup. It also authorized only from settings.apiKey or a configured header, but v2 keys live in connections and resolve at request time, so connection-authenticated gateways got a 401 and never filled limits. Eligible models are now those synthesized without a catalog entry: no release date and no cost rows. Catalog-backed models keep their real values. This also retires the placeholder-value comparison as the only guard: 200000/32000 is a real catalog value for some models, so an equality check cannot tell a default from data. Authorization resolves the provider's active connection credential through ctx.integration.connection, behind a configured authorization header and ahead of settings.apiKey.
acedb72 to
4312e23
Compare
An empty settings.apiKey produced a bare "Bearer " authorization header on the /models probe. Treat it as absent, as the local provider does, and cover it with a live test.
|
The |
There was a problem hiding this comment.
My three earlier concerns are resolved:
- Stored credentials: the probe now uses the provider's active connection credential.
- Bundled and models.dev providers: they are no longer probed.
- Real catalog limits: values that equal the default can no longer be overwritten. Catalog-backed models are skipped before the placeholder comparison runs.
I checked this by running serve from source with a custom proxy provider whose only key was added through opencode auth import, against a /models stub that rejects unauthenticated requests:
- The stub received
Bearer stored, and the model's limits went from 200000/32000 to the advertised 1000000/65536. - OpenCode Zen models with real 200000/32000 limits (e.g.
mimo-v2.6-flash-free) kept their catalog values. - The 10 tests in
provider-openai-compatible-limits.test.tspass, andtypecheckinpackages/coreis clean.
Public API: unchanged (no config keys, endpoints, events or plugin API). It still adds a default-on GET {baseURL}/models request, now only for openai-compatible models that are not in the catalog.
One smaller point: the two new behaviours this round (auth from a stored connection credential, and skipping catalog-backed models) have no tests yet. One small plugin-level test for each would guard them against regressions.
Fixes #40908
Extend the discovery lifecycle from anomalyco#57161 so native proxy endpoints do not inherit unsupported models from the shared catalog. Preserve catalog metadata and explicit configuration, without coupling availability to routing or request-budget changes. Based on anomalyco#57161; availability and effective-connection handling informed by anomalyco#53713 and anomalyco#54398. Co-authored-by: Gr33nbl00d <[email protected]> Co-authored-by: J.H.S. Schlepers <[email protected]> Co-authored-by: Lance Woodson <[email protected]>
Extend the discovery lifecycle from anomalyco#57161 so native proxy endpoints do not inherit unsupported models from the shared catalog. Preserve catalog metadata and explicit configuration, without coupling availability to routing or request-budget changes. Based on anomalyco#57161; availability and effective-connection handling informed by anomalyco#53713 and anomalyco#54398. Co-authored-by: Gr33nbl00d <[email protected]> Co-authored-by: J.H.S. Schlepers <[email protected]> Co-authored-by: Lance Woodson <[email protected]>
Extend the discovery lifecycle from anomalyco#57161 so native proxy endpoints do not inherit unsupported models from the shared catalog. Preserve catalog metadata and explicit configuration, without coupling availability to routing or request-budget changes. Based on anomalyco#57161; availability and effective-connection handling informed by anomalyco#53713 and anomalyco#54398. Co-authored-by: Gr33nbl00d <[email protected]> Co-authored-by: J.H.S. Schlepers <[email protected]> Co-authored-by: Lance Woodson <[email protected]>
Extend the discovery lifecycle from anomalyco#57161 so native proxy endpoints do not inherit unsupported models from the shared catalog. Preserve catalog metadata and explicit configuration, without coupling availability to routing or request-budget changes. Based on anomalyco#57161; availability and effective-connection handling informed by anomalyco#53713 and anomalyco#54398. Co-authored-by: Gr33nbl00d <[email protected]> Co-authored-by: J.H.S. Schlepers <[email protected]> Co-authored-by: Lance Woodson <[email protected]>
Issue for this PR
Closes #53235
Type of change
What does this PR do?
Ports #53236 to V2, following the invitation in that PR to reopen against the V2 line.
Custom OpenAI-compatible providers are not in models.dev, so their models keep the built-in placeholder limits and auto-compaction fires against a guess. A new provider plugin requests
GET {baseURL}/modelsonce per provider endpoint (4s timeout, 5 minute success cache, 60 second failure retry) and fillscontext,input, andoutputfrom advertisedcontext_length,max_input_tokens, andmax_output_tokens.Adaptations for V2:
apirecords to providerpackageplus merged model overlays. Discovery now applies to models whose effective package is@opencode/ai/providers/openai-compatible, with the base URL and auth resolved from provider and model settings the same way the model snapshot merges them.Model.Infodefault limits rather than zero, so a field is filled when it is absent, zero, or still equal to that default placeholder. Real limits from models.dev and the dedicated LM Studio, Ollama, vLLM, and Modal plugins are never replaced. Configuredlimitfields win per field through the config provider transform, which is registered after this plugin.settings.apiKey; a configuredauthorizationheader still wins.How did you verify your code works?
packages/core/test/plugin/provider-openai-compatible-limits.test.ts: parsing and merge rules, plus plugin-level tests against Bun-served endpoints covering fills, bearer auth, configured limits winning, and fail-open behavior on an erroring endpoint.bun run testin packages/core: 6601 pass, 2 fail, both reproduced on a clean upstream/dev checkout (session-diff and supervisor-reload flakes, unrelated to this change).bun run checkfrom the repository root: clean.bun typecheckin packages/core: clean.Screenshots / recordings
N/A (no UI change)
Checklist
Added by the opencode bot (AI-generated).
Fixes #40908