-
-
Notifications
You must be signed in to change notification settings - Fork 68.9k
memory-lancedb: support OpenAI-compatible local embeddings (baseUrl + dimensions) and fix vector-size mismatch #17564
Copy link
Copy link
Closed
Labels
close:already-fixedclaw-marshal labelclaw-marshal labeldedupe:childDuplicate issue/PR child in dedupe clusterDuplicate issue/PR child in dedupe cluster
Description
Issue Prompt: memory-lancedb local embeddings support (LM Studio)
Hello!
I want to enable my local mac LM Studio embedings model, and asked LLM to solve it and prepare as a prompt, so you can generate the solution yourself too. It was tested to work and after here its AI written. Let me know what you think about such approach, I can participate more.
Title
memory-lancedb: support OpenAI-compatible local embeddings (baseUrl + dimensions) and fix vector-size mismatch
Problem
memory-lancedb is hard to use with local OpenAI-compatible embedding endpoints (for example LM Studio/Ollama proxies).
Before fix:
- Config only allowed OpenAI model enum values.
- No
embedding.baseUrlfor custom endpoints. - No
embedding.dimensionsoverride for non-OpenAI models. - Using SDK embedding calls against LM Studio could yield vector-size mismatch vs LanceDB schema in practice.
Reproduction
- Configure
plugins.slots.memory = "memory-lancedb". - Set embedding model to local nomic GGUF (
text-embedding-nomic-embed-text-v1.5@q8_0) on LM Studio. - Run memory tools (
memory_storethenmemory_recall). - Observe LanceDB query errors due vector dimension mismatch.
Expected behavior
- Plugin accepts any OpenAI-compatible model id.
- Plugin can target custom embedding endpoint via
embedding.baseUrl. - Plugin can pin vector size via
embedding.dimensions. - Stored/search vectors use consistent dimension and work end-to-end with local providers.
Proposed solution
- Extend plugin config schema:
embedding.baseUrl?: stringembedding.dimensions?: integer- remove hardcoded
embedding.modelenum restriction
- Use configured dimensions when initializing LanceDB vector schema.
- Replace the
openaiSDK with raw HTTPfetch()to/embeddingsfor broader local-provider compatibility. AddAbortSignal.timeout(30_000)to prevent indefinite hangs when a local server is unresponsive. Remove theopenaidependency frompackage.json. - Split embedding dimensions into two concerns:
requestDimensions: only sent in the API request body when the user explicitly setsembedding.dimensionsin config. This avoids breaking local providers that reject unknown request fields.expectedDimensions: always validated against the response vector length (derived from LanceDB schema). This catches dimension mismatches early instead of surfacing as cryptic LanceDB errors.
- Support
${ENV_VAR}substitution inembedding.baseUrl(same pattern asembedding.apiKey). - Validate
embedding.dimensionsrange (1..32768) and reject empty/whitespacebaseUrl. - Update plugin manifest (
openclaw.plugin.json) UI hints and JSON schema. - Add tests for:
- custom model + baseUrl + dimensions
- env var resolution for baseUrl
- unknown model requiring dimensions
- invalid dimensions (type, range: 0, fractional)
- empty baseUrl rejection
Files touched (fork reference)
extensions/memory-lancedb/config.tsextensions/memory-lancedb/index.tsextensions/memory-lancedb/openclaw.plugin.jsonextensions/memory-lancedb/index.test.tsextensions/memory-lancedb/package.json(removeopenaidependency)pnpm-lock.yaml(lockfile cleanup)
Acceptance criteria
memory-lancedbcan run with LM Studio embeddings.memory_store,memory_recall,memory_forgetwork with local endpoint.- Plugin config validates correctly for both OpenAI hosted and local OpenAI-compatible providers.
- Existing OpenAI usage remains backward compatible (no
dimensionssent in request body for default models).
Validation performed (fork)
- Unit tests:
pnpm vitest run extensions/memory-lancedb/index.test.ts— 15/15 passed - Live smoke (LM Studio on Tailscale,
text-embedding-nomic-embed-text-v1.5@q8_0, 768 dims):memory_store-> createdmemory_recall-> found stored memorymemory_forget-> deleted stored memory- Auto-recall hook:
memory-lancedb: injecting 3 memories into context - Graceful degradation: app kept running when embedding server had no model loaded, auto-recovered once model was available
- Full suite:
pnpm build && pnpm check && pnpm test— pass (8 pre-existing upstream failures only)
Risk notes
- Local providers may vary in
/embeddingsbehavior; dimension enforcement surfaces misconfiguration early. - For custom models,
embedding.dimensionsshould be explicitly set. - The
openaiSDK included automatic retry logic for transient errors (429, 500). The rawfetchreplacement does not retry, but for a memory plugin making ~1 call per message, rate limiting is unlikely. Failures degrade gracefully via the existing try/catch in lifecycle hooks.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
close:already-fixedclaw-marshal labelclaw-marshal labeldedupe:childDuplicate issue/PR child in dedupe clusterDuplicate issue/PR child in dedupe cluster
Type
Fields
Give feedbackNo fields configured for issues without a type.