Skip to content

[Bug]: Memory index fails with "input length exceeds context length" — no chunking before embedding #28919

Description

@minerva-care

Summary

openclaw memory index fails for all agents with openai embeddings failed: 400 {"error":{"message":"the input length exceeds the context length"}}. The index never builds; zero chunks are stored.

Steps to reproduce

  1. Configure memorySearch with provider: "openai", remote.baseUrl: "http://localhost:11434/v1/", remote.apiKey: "ollama", model: "nomic-embed-text"
  2. Have one or more memory files larger than ~6,000 tokens (e.g. a daily notes file with multiple appended sessions)
  3. Run openclaw memory index or openclaw memory index --force

Expected behavior

Large files should be chunked into ~400 token segments before being sent to the embedding API. The indexer should never send a document larger than the model's context window in a single call.

Actual behavior

All agents fail immediately. openclaw memory status shows Indexed: 0/3563 files · 0 chunks.

OpenClaw version

2026.2.25 / 2026.2.26

Operating system

macOS 15.3 (Darwin 25.3.0, arm64)z

Install method

openclaw update

Logs, screenshots, and evidence

## Environment

- **OS:** macOS 15.3 (Darwin 25.3.0, arm64)
- **Embedding provider:** openai (Ollama OpenAI-compatible endpoint)
- **Embedding model:** nomic-embed-text (via Ollama at localhost:11434)
- **nomic-embed-text context window:** 8,192 tokens
- **Memory sources:** MEMORY.md + memory/*.md + 8 Obsidian extra paths

Impact and severity

High — semantic memory search is completely non-functional

Additional information

Root Cause

OpenClaw appears to send entire files to the embedding API without pre-chunking. When any file exceeds the model's context limit, the entire agent index fails — not just that file. The verbose output shows embeddings: batch start lines, suggesting batching infrastructure exists but chunking is not applied before the API call.

Workaround

Manually split large memory files so no single file exceeds ~5,000 tokens. Not sustainable for daily notes that grow throughout the day.

Suggested Fix

  1. Pre-chunk all source files before embedding, regardless of size
  2. Expose a chunkSize / maxChunkTokens config option in memorySearch
  3. Handle per-file failures gracefully — skip and warn, don't fail the entire agent index

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingstaleMarked as stale due to inactivity

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions