Skip to content

Commit 591dd2f

Browse files
authored
docs: swap Ollama example for Infron LLM gateway in custom LLM provider doc (anomalyco#1477)
* docs: swap Ollama functional example for Infron LLM gateway Introduces the LLM gateway concept (single OpenAI-compatible endpoint, unified billing, fallbacks), replaces the Ollama walkthrough with an Infron config, and links out to Infron's signup + quickstart per their reciprocal doc in their OpenWork page. * docs: frame Infron as hosted gateway vs LiteLLM starting point Link to Infron's LiteLLM integration page as the canonical open-source gateway reference, and position Infron as the OpenRouter-style hosted gateway that sits on top: one key, unified billing, provider fallbacks. * docs: drop OpenRouter comparison, pitch Infron directly for dev ergonomics * docs: drop em dashes, loosen Infron pitch to devrel voice * docs: add Infron model picker + active session screenshots
1 parent 85ab73b commit 591dd2f

3 files changed

Lines changed: 28 additions & 27 deletions

File tree

‎packages/docs/cloud/share-with-your-team/custom-llm-provider.mdx‎

Lines changed: 28 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -68,63 +68,64 @@ The JSON must include `id`, `name`, `npm`, `env`, `doc`, and `models`. `api` is
6868
3. Under `Cloud providers`, click `Import`.
6969
4. Reload the workspace when OpenWork asks.
7070

71-
### Functional example: Ollama (Qwen3 8B)
71+
### Functional example: an LLM gateway (Infron)
7272

73-
This setup a local [Ollama](https://docs.ollama.com/api/openai-compatibility) instance running `qwen3:8b`.
74-
Ollama's OpenAI-compatible endpoint requires an API key but ignores the value, so paste anything (for example `ollama`) into the `API key / credential` field when creating the provider.
73+
An LLM gateway is one OpenAI-compatible endpoint that fans out to many model providers. If you want to run the routing yourself, [LiteLLM](https://infron.ai/docs/frameworks-and-integrations/litellm) is a good starting point. [Infron](https://infron.ai) is the hosted option: one API key gets you every model in its [marketplace](https://infron.ai/models) with automatic provider fallbacks and a single invoice, so adding a new LLM to OpenWork is just another entry under `models`.
7574

76-
<Frame>
77-
![Config to add Ollama as a custom provider](/images/ollama-custom-provider.png)
78-
</Frame>
75+
Grab a key from the [API Keys dashboard](https://infron.ai/dashboard/apiKeys). If you don't have an account yet, sign up at [infron.ai/login](https://infron.ai/login) and their [quickstart](https://infron.ai/docs/overview/quickstart/text) walks you through your first request.
76+
77+
Paste the key into `API key / credential` and use this JSON:
7978

8079
```json
8180
{
82-
"id": "ollama",
83-
"name": "Ollama",
81+
"id": "infron",
82+
"name": "Infron",
8483
"npm": "@ai-sdk/openai-compatible",
8584
"env": [
86-
"OLLAMA_API_KEY"
85+
"INFRON_API_KEY"
8786
],
88-
"doc": "https://docs.ollama.com/api/openai-compatibility",
89-
"api": "http://localhost:11434/v1",
87+
"doc": "https://infron.ai/docs/frameworks-and-integrations/openwork",
88+
"api": "https://llm.onerouter.pro/v1",
9089
"models": [
9190
{
92-
"id": "qwen3:8b",
93-
"name": "Qwen3 8B",
91+
"id": "deepseek/deepseek-v3.2",
92+
"name": "DeepSeek V3.2",
9493
"attachment": false,
9594
"reasoning": true,
9695
"tool_call": true,
9796
"structured_output": true,
9897
"temperature": true,
99-
"release_date": "2025-04-28",
100-
"last_updated": "2025-04-28",
98+
"release_date": "2025-09-29",
99+
"last_updated": "2025-09-29",
101100
"open_weights": true,
102101
"limit": {
103-
"context": 32768,
104-
"input": 32768,
102+
"context": 128000,
103+
"input": 128000,
105104
"output": 8192
106105
},
107106
"modalities": {
108-
"input": [
109-
"text"
110-
],
111-
"output": [
112-
"text"
113-
]
107+
"input": ["text"],
108+
"output": ["text"]
114109
}
115110
}
116111
]
117112
}
118113
```
119114

115+
Once imported, Infron models show up in the Chat model picker alongside everything else:
116+
120117
<Frame>
121-
![Ollama added as a provider](/images/ollama-added-as-provider.png)
118+
![DeepSeek V3.2 via Infron in the OpenWork Chat model picker](/images/infron-model-picker.png)
122119
</Frame>
123120

124-
Pull the model first with `ollama pull qwen3:8b`, then make sure the Ollama server is reachable at `http://localhost:11434` with `ollama serve`
121+
Pick the model and it's live in the session footer:
122+
123+
<Frame>
124+
![Infron · DeepSeek V3.2 selected in an OpenWork session](/images/infron-model-active.png)
125+
</Frame>
125126

126-
Import it in your desktop, change the provider name and voila! You have an fully local LLM running in your machine.
127+
Add more entries under `models` to expose other routes the gateway supports (e.g. `openai/gpt-5.4`, `google/gemini-2.5-flash`).
127128

128129
## When to use a cloud provider
129130

130-
We recommend using Cloud-bsaed provider when the setup is meant to be shared across an org or team. Otherwise, doing it directly in desktop is easier.
131+
Use a Cloud provider when the setup is meant to be shared across an org or team. For solo use, configuring it directly in the desktop app is simpler.
384 KB
Loading
487 KB
Loading

0 commit comments

Comments
 (0)