Skip to content

Per-message metadata in system prompt invalidates llama.cpp KV cache every turn #20430

Description

@shem-aleph

Problem

When using a local llama.cpp server (via openai-completions provider), the KV cache is fully invalidated on every message because the system prompt changes between turns.

Root Cause

buildInboundMetaSystemPrompt() injects per-message metadata into the system prompt (via extraSystemPrompt), including message_id which changes every message:

{
  "schema": "openclaw.inbound_meta.v1",
  "message_id": "2959",
  "sender_id": "414434471",
  "chat_id": "telegram:414434471",
  ...
}

For Anthropic/OpenAI this is fine — they handle caching server-side. But for llama.cpp, the KV cache works by matching the token prefix byte-for-byte. Any change in the system prompt (first message in the conversation) shifts the entire token sequence and forces a full reprocessing of the entire context.

Additionally, buildInboundUserContextPrefix() prepends changing metadata (including message_id) to each user message.

Impact

With a 196K context window model like MiniMax-M2.5, every message requires reprocessing the full conversation history from scratch, making the agent progressively slower and wasting compute.

Suggested Fix

One or more of:

  1. Strip volatile fields (message_id) from inbound meta for openai-completions providers
  2. Move per-message metadata out of the system prompt and into the last user message only, so the system prefix stays stable and cacheable
  3. Add a provider-level config like "stableSystemPrompt": true that suppresses per-message metadata injection into the system prompt
  4. Move the inbound meta block entirely to user messages instead of appending to the system prompt

Option 2 or 4 would be the cleanest — the system prompt stays byte-stable across turns, and llama.cpp can reuse the KV cache for the entire prefix.

Environment

  • OpenClaw 2026.2.17
  • Provider: openai-completions pointing to llama.cpp server
  • Model: MiniMaxAI/MiniMax-M2.5 (196K context)

Relevant Code

  • src/auto-reply/reply/inbound-meta.tsbuildInboundMetaSystemPrompt()
  • src/auto-reply/reply/get-reply-run.ts → where extraSystemPrompt is assembled

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions