You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit b79b7e8
Browse filesBrowse the repository at this point in the historyBrowse files
docs: add local Ollama inference setup section (#822)
* docs: add local Ollama inference setup section
Includes changes done in PR #735.
Fixes#314, #385
Signed-off-by: Paritosh Dixit <[email protected]>
* docs: fix style and remove stale experimental gate
Remove numbered section titles and colons from the Ollama setup section
in spark-install.md per doc style guide.
Remove the NEMOCLAW_EXPERIMENTAL=1 gate from inference-profiles.md —
local providers are GA and the gate no longer exists in source.
Co-authored-by: Claude Opus 4.6 <[email protected]>
Signed-off-by: Paritosh Dixit <[email protected]>
---------
Signed-off-by: Paritosh Dixit <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
> **WIP** — This page is actively being updated as we work through Spark installs. Expect changes.
4
4
5
+
## Prerequisites
6
+
7
+
-**Docker** (pre-installed, v28.x)
8
+
-**Node.js 22** (installed by the install.sh)
9
+
-**OpenShell CLI** (installed via the Quick Start steps below)
10
+
-**NVIDIA API Key** from [build.nvidia.com](https://build.nvidia.com) — prompted on first run
11
+
5
12
## Quick Start
6
13
7
14
```bash
8
-
# Clone and install
15
+
# Install OpenShell:
16
+
curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh
Download Nemotron 3 Super 120B (~87 GB; may take several minutes):
81
+
82
+
```bash
83
+
ollama pull nemotron-3-super:120b
84
+
```
85
+
86
+
Run it briefly to pre-load weights into unified memory, then exit:
87
+
88
+
```bash
89
+
ollama run nemotron-3-super:120b
90
+
# type /bye to exit
91
+
```
92
+
93
+
### Configure Ollama to Listen on All Interfaces
94
+
95
+
By default Ollama binds to `127.0.0.1`, which is not reachable from inside the sandbox container. Configure it to listen on all interfaces:
96
+
97
+
> **Note:**`OLLAMA_HOST=0.0.0.0` exposes Ollama on your network. If you're not on a trusted LAN, restrict access with host firewall rules (`ufw`, `iptables`, etc.).
# Expected: JSON response listing the configured model
138
+
# Exits non-zero on HTTP errors (403, 503, etc.) — failure here indicates a proxy routing regression
139
+
```
140
+
141
+
Then talk to the agent:
142
+
143
+
```bash
144
+
openclaw agent --agent main --local -m "Which model and GPU are in use?" --session-id test
145
+
```
18
146
19
147
## What's Different on Spark
20
148
@@ -42,28 +170,6 @@ Failed to start ContainerManager: failed to initialize top level QOS containers
42
170
43
171
**Fix**: `setup-spark` sets `"default-cgroupns-mode": "host"` in `/etc/docker/daemon.json` and restarts Docker. This makes all containers use the host cgroup namespace, which is what k3s needs.
-**NVIDIA API Key** from [build.nvidia.com](https://build.nvidia.com) — prompted on first run
0 commit comments