Skip to content

Commit 1fef999

Browse files
steipetesteipete
authored
feat(nodes): add auto-discovered Ollama inference (#99234)
* feat(nodes): add local Ollama inference * fix(gateway): preserve plugin node runtime for agent turns * feat(ollama): add node inference opt-out * test(security): preserve plugin runtime exports * test(security): preserve plugin runtime exports * test(security): preserve plugin runtime exports * fix(ci): raise artifact build heap
1 parent 5e61da3 commit 1fef999

23 files changed

Lines changed: 1233 additions & 52 deletions

‎.github/workflows/ci-build-artifacts-testbox.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -156,7 +156,7 @@ jobs:
156156
- name: Build dist on cache miss
157157
if: steps.dist-cache.outputs.cache-hit != 'true'
158158
env:
159-
NODE_OPTIONS: --max-old-space-size=8192
159+
NODE_OPTIONS: --max-old-space-size=12288
160160
run: pnpm build:ci-artifacts
161161

162162
- name: Build Control UI on cache miss

‎docs/docs_map.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4712,6 +4712,7 @@ Do not edit it by hand; run `pnpm docs:map:gen`.
47124712
- H3: Pair + name
47134713
- H3: Allowlist the commands
47144714
- H3: Point exec at the node
4715+
- H3: Local model inference
47154716
- H2: Invoking commands
47164717
- H2: Command policy
47174718
- H2: Config (openclaw.json)
@@ -7613,6 +7614,7 @@ Do not edit it by hand; run `pnpm docs:map:gen`.
76137614
- H2: Getting started
76147615
- H2: Cloud models
76157616
- H2: Model discovery (implicit provider)
7617+
- H2: Node-local inference
76167618
- H2: Vision and image description
76177619
- H2: Configuration
76187620
- H2: Common recipes

‎docs/nodes/index.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -184,6 +184,14 @@ Related:
184184
- [Exec tool](/tools/exec)
185185
- [Exec approvals](/tools/exec-approvals)
186186

187+
### Local model inference
188+
189+
A desktop or server node can expose chat-capable models from an Ollama server
190+
running on that node. Agents use the Ollama plugin's `node_inference` tool to
191+
discover installed models and run a bounded prompt remotely; the Gateway does
192+
not need direct network access to Ollama. See [Ollama node-local inference](/providers/ollama#node-local-inference)
193+
for setup, model filtering, and direct verification commands.
194+
187195
## Invoking commands
188196

189197
Low-level (raw RPC):

‎docs/providers/ollama.md‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -304,6 +304,107 @@ The new model will be automatically discovered and available to use.
304304
If you set `models.providers.ollama` explicitly, or configure a custom remote provider such as `models.providers.ollama-cloud` with `api: "ollama"`, auto-discovery is skipped and you must define models manually. Loopback custom providers such as `http://127.0.0.2:11434` are still treated as local. See the explicit config section below.
305305
</Note>
306306

307+
## Node-local inference
308+
309+
Agents can delegate a short task to an Ollama model installed on a paired
310+
desktop or server node. The prompt and response cross the existing authenticated
311+
Gateway/node connection; the model request runs on the selected node against
312+
its standard loopback Ollama endpoint (`http://127.0.0.1:11434`).
313+
314+
<Steps>
315+
<Step title="Start Ollama on the node">
316+
Pull at least one chat model and keep Ollama running:
317+
318+
```bash
319+
ollama pull qwen3:0.6b
320+
ollama list
321+
```
322+
323+
</Step>
324+
<Step title="Connect the node host">
325+
On the same machine as Ollama, connect a node host to the Gateway:
326+
327+
```bash
328+
openclaw node run \
329+
--host <gateway-host> \
330+
--port 18789 \
331+
--display-name "Local inference"
332+
```
333+
334+
Approve the new device and its declared node commands on the Gateway host,
335+
then verify the node:
336+
337+
```bash
338+
openclaw devices list
339+
openclaw devices approve <deviceRequestId>
340+
openclaw nodes pending
341+
openclaw nodes approve <nodeRequestId>
342+
openclaw nodes status --connected
343+
```
344+
345+
A first connection and an upgrade that adds the Ollama commands can both
346+
trigger node-command approval. If the node connects without advertising
347+
`ollama.models` and `ollama.chat`, check `openclaw nodes pending` again.
348+
349+
</Step>
350+
<Step title="Ask an agent to use local inference">
351+
The bundled Ollama plugin exposes the `node_inference` tool. Agents first
352+
use `action: "discover"`, then `action: "run"` with a returned node and
353+
model. If exactly one capable node is connected, `run` can omit the node.
354+
355+
For example: “Discover the Ollama models on my nodes, then use the fastest
356+
loaded model to summarize this text.”
357+
358+
</Step>
359+
</Steps>
360+
361+
Discovery reads `/api/tags`, checks `/api/show` capabilities, and uses `/api/ps`
362+
when available to rank already-loaded models first. It returns only local
363+
chat-capable models: Ollama Cloud rows and embedding-only models are excluded.
364+
Each run asks Ollama to disable model thinking and caps output at 512 tokens
365+
unless the tool call requests a different `maxTokens` value. Some models, such
366+
as GPT-OSS, do not support disabling thinking and may still use reasoning tokens.
367+
368+
To keep Ollama running on a node without making it available to agents, set the
369+
following in the config used by that node host:
370+
371+
```bash
372+
openclaw config set plugins.entries.ollama.config.nodeInference.enabled false
373+
```
374+
375+
If the node uses the foreground `openclaw node run` command from the setup
376+
above, stop that process and run the command again. If it uses an installed node
377+
service, run `openclaw node restart`.
378+
379+
The node stops advertising `ollama.models` and `ollama.chat`; Ollama itself and
380+
the Gateway's Ollama provider remain unchanged. Set the value to `true` and
381+
restart the node to advertise local inference again. A changed command surface
382+
may require approval through `openclaw nodes pending` after reconnect.
383+
384+
You can verify the same node commands without an agent turn:
385+
386+
```bash
387+
openclaw nodes invoke \
388+
--node "Local inference" \
389+
--command ollama.models \
390+
--params '{}' \
391+
--invoke-timeout 90000 \
392+
--timeout 100000
393+
394+
openclaw nodes invoke \
395+
--node "Local inference" \
396+
--command ollama.chat \
397+
--params '{"model":"qwen3:0.6b","prompt":"Reply with exactly: pong","maxTokens":32,"timeoutMs":120000}' \
398+
--invoke-timeout 130000 \
399+
--timeout 140000
400+
```
401+
402+
Node-local inference intentionally does not reuse a remote or cloud
403+
`models.providers.ollama.baseUrl`. Start Ollama on the node's standard loopback
404+
endpoint. The node commands are available by default on macOS, Linux, and
405+
Windows node hosts and remain subject to the normal node pairing and command
406+
policy.
407+
307408
## Vision and image description
308409

309410
The bundled Ollama plugin registers Ollama as an image-capable media-understanding provider. This lets OpenClaw route explicit image-description requests and configured image-model defaults through local or hosted Ollama vision models.

‎extensions/ollama/index.test.ts‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,57 @@ function captureWrappedOllamaPayload(
178178
}
179179

180180
describe("ollama plugin", () => {
181+
it("registers node-local inference commands, policy, and agent tool", () => {
182+
const registerNodeHostCommand = vi.fn();
183+
const registerNodeInvokePolicy = vi.fn();
184+
const registerTool = vi.fn();
185+
186+
plugin.register(
187+
createTestPluginApi({
188+
id: "ollama",
189+
name: "Ollama",
190+
source: "test",
191+
registerNodeHostCommand,
192+
registerNodeInvokePolicy,
193+
registerTool,
194+
}),
195+
);
196+
197+
expect(registerNodeHostCommand.mock.calls.map(([entry]) => entry.command)).toEqual([
198+
"ollama.models",
199+
"ollama.chat",
200+
]);
201+
expect(registerNodeInvokePolicy).toHaveBeenCalledWith(
202+
expect.objectContaining({
203+
commands: ["ollama.models", "ollama.chat"],
204+
defaultPlatforms: ["macos", "linux", "windows"],
205+
}),
206+
);
207+
expect(registerTool).toHaveBeenCalledWith(expect.objectContaining({ name: "node_inference" }));
208+
});
209+
210+
it("keeps the agent tool but does not advertise node inference when disabled locally", () => {
211+
const registerNodeHostCommand = vi.fn();
212+
const registerNodeInvokePolicy = vi.fn();
213+
const registerTool = vi.fn();
214+
215+
plugin.register(
216+
createTestPluginApi({
217+
id: "ollama",
218+
name: "Ollama",
219+
source: "test",
220+
pluginConfig: { nodeInference: { enabled: false } },
221+
registerNodeHostCommand,
222+
registerNodeInvokePolicy,
223+
registerTool,
224+
}),
225+
);
226+
227+
expect(registerNodeHostCommand).not.toHaveBeenCalled();
228+
expect(registerNodeInvokePolicy).toHaveBeenCalledOnce();
229+
expect(registerTool).toHaveBeenCalledWith(expect.objectContaining({ name: "node_inference" }));
230+
});
231+
181232
it("does not preselect a default model during provider auth setup", async () => {
182233
const provider = registerProvider();
183234

‎extensions/ollama/index.ts‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,11 @@ import {
5858
} from "./src/embedding-provider.js";
5959
import { ollamaMediaUnderstandingProvider } from "./src/media-understanding-provider.js";
6060
import { ollamaMemoryEmbeddingProviderAdapter } from "./src/memory-embedding-adapter.js";
61+
import {
62+
createOllamaNodeHostCommands,
63+
createOllamaNodeInferenceTool,
64+
createOllamaNodeInvokePolicy,
65+
} from "./src/node-inference.js";
6166
import { readProviderBaseUrl } from "./src/provider-base-url.js";
6267
import {
6368
createConfiguredOllamaCompatStreamWrapper,
@@ -435,12 +440,19 @@ export default definePluginEntry({
435440
name: "Ollama Provider",
436441
description: "Bundled Ollama provider plugin",
437442
register(api: OpenClawPluginApi) {
443+
const startupPluginConfig = (api.pluginConfig ?? {}) as OllamaPluginConfig;
438444
if (api.registrationMode === "full") {
439445
void checkWsl2CrashLoopRisk(api.logger);
440446
}
441447
api.registerMemoryEmbeddingProvider(ollamaMemoryEmbeddingProviderAdapter);
442448
api.registerMediaUnderstandingProvider(ollamaMediaUnderstandingProvider);
443-
const startupPluginConfig = (api.pluginConfig ?? {}) as OllamaPluginConfig;
449+
if (startupPluginConfig.nodeInference?.enabled !== false) {
450+
for (const command of createOllamaNodeHostCommands()) {
451+
api.registerNodeHostCommand(command);
452+
}
453+
}
454+
api.registerNodeInvokePolicy(createOllamaNodeInvokePolicy());
455+
api.registerTool(createOllamaNodeInferenceTool(api));
444456
const resolveCurrentPluginConfig = (config?: OpenClawConfig): OllamaPluginConfig => {
445457
const runtimePluginConfig = resolvePluginConfigObject(config, "ollama");
446458
if (runtimePluginConfig) {

‎extensions/ollama/openclaw.plugin.json‎

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"id": "ollama",
33
"icon": "https://cdn.simpleicons.org/ollama",
44
"activation": {
5-
"onStartup": false
5+
"onStartup": true
66
},
77
"enabledByDefault": true,
88
"providers": ["ollama", "ollama-cloud"],
@@ -155,6 +155,7 @@
155155
},
156156
"contracts": {
157157
"memoryEmbeddingProviders": ["ollama"],
158+
"tools": ["node_inference"],
158159
"webSearchProviders": ["ollama"]
159160
},
160161
"configSchema": {
@@ -167,6 +168,13 @@
167168
"properties": {
168169
"enabled": { "type": "boolean" }
169170
}
171+
},
172+
"nodeInference": {
173+
"type": "object",
174+
"additionalProperties": false,
175+
"properties": {
176+
"enabled": { "type": "boolean" }
177+
}
170178
}
171179
}
172180
},
@@ -178,6 +186,14 @@
178186
"discovery.enabled": {
179187
"label": "Enable Discovery",
180188
"help": "When false, OpenClaw keeps the Ollama plugin available but skips implicit startup discovery of ambient local or remote Ollama models."
189+
},
190+
"nodeInference": {
191+
"label": "Node Inference",
192+
"help": "Controls whether this node host advertises its local Ollama models to agents."
193+
},
194+
"nodeInference.enabled": {
195+
"label": "Enable Node Inference",
196+
"help": "When false, this node host does not advertise or accept Ollama node-inference commands."
181197
}
182198
}
183199
}

‎extensions/ollama/src/discovery-shared.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,9 @@ export type OllamaPluginConfig = {
2424
discovery?: {
2525
enabled?: boolean;
2626
};
27+
nodeInference?: {
28+
enabled?: boolean;
29+
};
2730
};
2831

2932
type OllamaDiscoveryContext = {

0 commit comments

Comments
 (0)