Skip to content

Commit 3137110

Browse files
osolmazosolmaz
authored
fix(memory): move local llama.cpp runtime to provider plugin
* fix(memory): move local llama.cpp runtime to provider plugin * chore: ignore llama cpp dynamic dependency * test: remove invalid local provider alias fixture * chore: refresh llama cpp shrinkwrap * chore: drop stale memory embedding defaults facade
1 parent 4c98a54 commit 3137110

39 files changed

Lines changed: 3267 additions & 179 deletions

‎.github/labeler.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -293,6 +293,10 @@
293293
- changed-files:
294294
- any-glob-to-any-file:
295295
- "extensions/lobster/**"
296+
"extensions: llama-cpp":
297+
- changed-files:
298+
- any-glob-to-any-file:
299+
- "extensions/llama-cpp/**"
296300
"extensions: memory-core":
297301
- changed-files:
298302
- any-glob-to-any-file:

‎config/knip.config.ts‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -217,6 +217,16 @@ const config = {
217217
entry: ["index.js!", "scripts/postinstall.js!"],
218218
project: ["index.js!", "scripts/**/*.js!"],
219219
},
220+
[`${BUNDLED_PLUGIN_ROOT_DIR}/llama-cpp`]: {
221+
entry: bundledPluginEntries,
222+
project: ["index.ts!", "src/**/*.{js,mjs,ts}!"],
223+
ignoreDependencies: [
224+
// The provider resolves node-llama-cpp from its own package at runtime
225+
// so local embeddings use the plugin-owned native dependency.
226+
"node-llama-cpp",
227+
...bundledPluginIgnoredRuntimeDependencies,
228+
],
229+
},
220230
[`${BUNDLED_PLUGIN_ROOT_DIR}/*`]: {
221231
// Bundled plugins often load their public surface via string specifiers in
222232
// `index.ts` contracts, so Knip needs these convention-based entry files.

‎docs/concepts/memory-builtin.md‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -39,9 +39,12 @@ To set a provider explicitly:
3939

4040
Without an embedding provider, only keyword search is available.
4141

42-
To force the built-in local embedding provider, install the optional
43-
`node-llama-cpp` runtime package next to OpenClaw, then point `local.modelPath`
44-
at a GGUF file:
42+
To force local GGUF embeddings, install the official llama.cpp provider plugin,
43+
then point `local.modelPath` at a GGUF file:
44+
45+
```bash
46+
openclaw plugins install @openclaw/llama-cpp-provider
47+
```
4548

4649
```json5
4750
{
@@ -67,7 +70,7 @@ at a GGUF file:
6770
| DeepInfra | `deepinfra` | Default: `BAAI/bge-m3` |
6871
| Gemini | `gemini` | Supports multimodal (image + audio) |
6972
| GitHub Copilot | `github-copilot` | Uses Copilot subscription |
70-
| Local | `local` | Optional `node-llama-cpp` runtime |
73+
| Local | `local` | `@openclaw/llama-cpp-provider` |
7174
| Mistral | `mistral` | |
7275
| Ollama | `ollama` | Local/self-hosted |
7376
| OpenAI | `openai` | Default: `text-embedding-3-small` |

‎docs/concepts/memory-qmd.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ binary, and can index content beyond your workspace memory files.
1515
- **Reranking and query expansion** for better recall.
1616
- **Index extra directories** -- project docs, team notes, anything on disk.
1717
- **Index session transcripts** -- recall earlier conversations.
18-
- **Fully local** -- runs with the optional node-llama-cpp runtime package and
18+
- **Fully local** -- runs with the official llama.cpp provider plugin and
1919
auto-downloads GGUF models.
2020
- **Automatic fallback** -- if QMD is unavailable, OpenClaw falls back to the
2121
builtin engine seamlessly.

‎docs/concepts/memory-search.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,8 @@ For multi-endpoint setups with memory-specific providers, `provider` can also
3232
be a custom `models.providers.<id>` entry, such as `ollama-5080`, when that
3333
provider sets `api: "ollama"` or another memory embedding adapter owner.
3434

35-
For local embeddings with no API key, set `provider: "local"`. Source checkouts
35+
For local embeddings with no API key, install
36+
`@openclaw/llama-cpp-provider` and set `provider: "local"`. Source checkouts
3637
may still require native build approval: `pnpm approve-builds` then
3738
`pnpm rebuild node-llama-cpp`.
3839

‎docs/docs.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1249,6 +1249,7 @@
12491249
"plugins/admin-http-rpc",
12501250
"plugins/voice-call",
12511251
"plugins/memory-wiki",
1252+
"plugins/llama-cpp",
12521253
"plugins/memory-lancedb",
12531254
"plugins/oc-path",
12541255
"plugins/zalouser"

‎docs/plugins/llama-cpp.md‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
---
2+
summary: "Install the official llama.cpp provider for local GGUF memory embeddings"
3+
read_when:
4+
- You want memory search embeddings from a local GGUF model
5+
- You are configuring memorySearch.provider = "local"
6+
- You need the OpenClaw plugin that owns the node-llama-cpp runtime
7+
title: "llama.cpp Provider"
8+
sidebarTitle: "llama.cpp Provider"
9+
---
10+
11+
`llama-cpp` is the official external provider plugin for local GGUF embeddings.
12+
It owns the `node-llama-cpp` runtime dependency used by
13+
`memorySearch.provider: "local"`.
14+
15+
Install it before using local memory embeddings:
16+
17+
```bash
18+
openclaw plugins install @openclaw/llama-cpp-provider
19+
```
20+
21+
The main `openclaw` npm package does not include `node-llama-cpp`. Keeping the
22+
native dependency in this plugin prevents normal OpenClaw npm updates from
23+
deleting a manually installed runtime inside the OpenClaw package directory.
24+
25+
## Configuration
26+
27+
Set the memory search provider to `local`:
28+
29+
```json5
30+
{
31+
agents: {
32+
defaults: {
33+
memorySearch: {
34+
provider: "local",
35+
local: {
36+
modelPath: "hf:ggml-org/embeddinggemma-300m-qat-q8_0-GGUF/embeddinggemma-300m-qat-Q8_0.gguf",
37+
},
38+
},
39+
},
40+
},
41+
}
42+
```
43+
44+
The default model is `embeddinggemma-300m-qat-Q8_0.gguf`. You can also point
45+
`local.modelPath` at a local `.gguf` file.
46+
47+
## Native Runtime
48+
49+
Use Node 24 for the smoothest native install path. Source checkouts using pnpm
50+
may need to approve and rebuild the native dependency:
51+
52+
```bash
53+
pnpm approve-builds
54+
pnpm rebuild node-llama-cpp
55+
```
56+
57+
For lower-friction local embeddings, use a local service provider such as
58+
Ollama or LM Studio instead.

‎docs/plugins/plugin-inventory.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -137,7 +137,7 @@ Each entry lists the package, distribution route, and description.
137137

138138
- **[mattermost](/plugins/reference/mattermost)** (`@openclaw/mattermost`) - included in OpenClaw. Adds the Mattermost channel surface for sending and receiving OpenClaw messages.
139139

140-
- **[memory-core](/plugins/reference/memory-core)** (`@openclaw/memory-core`) - included in OpenClaw. Adds memory embedding provider support. Adds agent-callable tools.
140+
- **[memory-core](/plugins/reference/memory-core)** (`@openclaw/memory-core`) - included in OpenClaw. Adds file-backed memory search tools.
141141

142142
- **[memory-wiki](/plugins/reference/memory-wiki)** (`@openclaw/memory-wiki`) - included in OpenClaw. Persistent wiki compiler and Obsidian-friendly knowledge vault for OpenClaw.
143143

@@ -235,7 +235,7 @@ Each entry lists the package, distribution route, and description.
235235

236236
## Official external packages
237237

238-
34 plugins
238+
35 plugins
239239

240240
- **[acpx](/plugins/reference/acpx)** (`@openclaw/acpx`) - npm; ClawHub. OpenClaw ACP runtime backend with plugin-owned session and transport management.
241241

@@ -267,6 +267,8 @@ Each entry lists the package, distribution route, and description.
267267

268268
- **[googlechat](/plugins/reference/googlechat)** (`@openclaw/googlechat`) - npm; ClawHub. OpenClaw Google Chat channel plugin for spaces and direct messages.
269269

270+
- **[llama-cpp](/plugins/reference/llama-cpp)** (`@openclaw/llama-cpp-provider`) - npm; ClawHub. OpenClaw llama.cpp embedding provider plugin.
271+
270272
- **[line](/plugins/reference/line)** (`@openclaw/line`) - npm; ClawHub. OpenClaw LINE channel plugin for LINE Bot API chats.
271273

272274
- **[lobster](/plugins/reference/lobster)** (`@openclaw/lobster`) - npm; ClawHub. Lobster workflow tool plugin for typed pipelines and resumable approvals.

‎docs/plugins/reference.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,5 +15,5 @@ This page is generated from `extensions/*/package.json` and
1515
pnpm plugins:inventory:gen
1616
```
1717

18-
Use [Plugin inventory](/plugins/plugin-inventory) to browse all 126
18+
Use [Plugin inventory](/plugins/plugin-inventory) to browse all 127
1919
generated plugin reference pages by distribution, package, and description.

‎docs/plugins/reference/llama-cpp.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
---
2+
summary: "OpenClaw llama.cpp embedding provider plugin."
3+
read_when:
4+
- You are installing, configuring, or auditing the llama-cpp plugin
5+
title: "llama-cpp plugin"
6+
---
7+
8+
# llama-cpp plugin
9+
10+
OpenClaw llama.cpp embedding provider plugin.
11+
12+
## Distribution
13+
14+
- Package: `@openclaw/llama-cpp-provider`
15+
- Install route: npm; ClawHub
16+
17+
## Surface
18+
19+
contracts: embeddingProviders
20+
21+
## Related docs
22+
23+
- [llama.cpp Provider](/plugins/llama-cpp)

0 commit comments

Comments
 (0)