Skip to content

fix: support local providers (ollama, new-api, aihubmix) for extended thinking and API server#12796

Merged
kangfenmao merged 6 commits intomainfrom
DeJeune/fix-extended-thinking
Mar 12, 2026
Merged

fix: support local providers (ollama, new-api, aihubmix) for extended thinking and API server#12796
kangfenmao merged 6 commits intomainfrom
DeJeune/fix-extended-thinking

Conversation

@DeJeune
Copy link
Copy Markdown
Collaborator

@DeJeune DeJeune commented Feb 7, 2026

What this PR does

Before this PR:

  • Ollama was not recognized as an Anthropic-compatible provider for extended thinking
  • Local providers (ollama, lmstudio) without API keys would fail agent model validation
  • The API server only exposed OpenAI and Anthropic provider types, excluding ollama and new-api
  • aihubmix's Anthropic model checker was restricted to Claude-only models (m.id.includes('claude'))

After this PR:

  • Ollama is supported as an Anthropic-compatible provider with anthropicApiHost configuration
  • Local providers automatically use a placeholder API key if none is provided
  • The API server now exposes ollama and new-api provider types alongside OpenAI and Anthropic
  • aihubmix falls through to the default checker (() => true), allowing all its Anthropic-compatible models

Fixes: #13247

Why we need it and why it was done in this way

Ollama extended thinking: Local AI models through Ollama should be able to access extended thinking features like other Anthropic-compatible providers. The implementation adds ollama to the Anthropic-compatible provider list and configures it with an anthropicApiHost pointing to the local Ollama server.

API server provider expansion: new-api already had Anthropic model support via getProviderAnthropicModelChecker but was missing from supportedTypes, so its models weren't exposed through the API server. Similarly, ollama needs to be in the supported providers list for validateModelId to resolve ollama models for agents.

aihubmix model checker: aihubmix now supports all LLM models as Anthropic-compatible (not just Claude models), so the previous Claude-only filter was overly restrictive.

API key bypass: Local providers (ollama, lmstudio) don't require real API keys, but downstream SDK calls reject empty keys. The validation method now sets a placeholder key for these providers, documented via JSDoc.

Breaking changes

None — this is a backwards-compatible enhancement.

Special notes for your reviewer

  • The migration (v196) safely initializes anthropicApiHost for existing ollama configs, with a fallback to http://localhost:11434 if apiHost is empty
  • The validateAgentModels JSDoc now documents the side-effect of mutating provider.apiKey for local providers
  • localProvidersWithoutApiKey is hoisted outside the loop to avoid per-iteration allocation
  • All tests pass (lint, type-check, unit tests)

Checklist

  • Code follows established patterns in the codebase
  • Tests added and passing (API key validation bypass, provider compatibility)
  • Build check passed (lint, test, typecheck)
  • Documentation: No user-facing feature (internal provider configuration)

Release note

NONE

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge This PR has sufficient reviewer approvals but is awaiting code owner approval.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: The code tool cannot recognize the New API

4 participants