Problem

When running local models (e.g. Qwen3-Coder-Next on MLX with an M3 Ultra), prefix caching is critical for interactive performance. However, the current system prompt layout in buildAgentSystemPrompt() interleaves dynamic content with large static blocks, which invalidates the prefix cache on nearly every request.

At ~65 tok/s prompt processing on our M3 Ultra (256GB), a full OpenClaw context (~50k tokens) takes ~12-13 minutes without cache hits. With effective prefix caching, the static prefix would be cached and only the dynamic tail (~500-5k tokens) would need processing — bringing prompt time down to under 1-2 seconds for repeat interactions.

Current Section Order

The system prompt is assembled in this order (from buildAgentSystemPrompt() in reply-Cx57rl6c.js):

1. Tooling (~2k tokens) — STATIC
2. Tool Call Style — STATIC
3. Safety — STATIC
4. CLI Quick Reference — STATIC
5. Skills (~1.5k tokens) — STATIC
6. Memory Recall — STATIC
7. Self-Update — STATIC
8. Model Aliases — STATIC
9. Workspace dir — STATIC
10. Documentation — STATIC
11. Authorized Senders — STATIC
12. Current Date & Time — STATIC (timezone only)
13. Workspace Files header — STATIC
14. Reply Tags — STATIC
15. Messaging — STATIC
16. Group Chat Context / Inbound Metadata — ⚠️ DYNAMIC (changes every request: chat_id, channel, provider, etc.)
17. Reactions — STATIC
18. Project Context (~25k tokens of workspace .md files) — SEMI-STATIC (changes only on file edits)
19. Silent Replies — STATIC
20. Heartbeats — STATIC
21. Runtime line — ⚠️ DYNAMIC (model, channel, capabilities, thinking level)

The Problem

The Inbound Metadata JSON (#16) changes on nearly every request (different chat_id, timestamps, etc.) and sits before the massive ~25k-token Project Context block (#18). This means the entire Project Context is reprocessed every request even when no workspace files have changed.

Similarly, the Runtime line (#21) is dynamic but sits after static sections like Silent Replies and Heartbeats — those static sections between two dynamic sections get no cache benefit either way.

Proposed Fix

Reorder sections so all static content forms a contiguous prefix, with dynamic content pushed to the end:

[STATIC PREFIX — cacheable, ~30-45k tokens]
1. Tooling
2. Tool Call Style
3. Safety
4. CLI Quick Reference
5. Skills
6. Memory Recall
7. Self-Update
8. Model Aliases
9. Workspace dir
10. Documentation
11. Reply Tags
12. Messaging
13. Reactions
14. Silent Replies
15. Heartbeats
16. Project Context / workspace files (~25k)

[DYNAMIC TAIL — reprocessed each request, ~500 tokens]
17. Authorized Senders
18. Current Date & Time
19. Group Chat Context (inbound metadata)
20. Runtime line

Impact

• Without reorder: ~50k tokens reprocessed every request. At 65 tok/s = ~13 min prompt processing.
• With reorder: ~30-45k token prefix cached, ~500-5k tokens reprocessed. Prompt time drops to < 2 seconds for same-session interactions.
• No semantic change — the model sees the same information, just in a different order.
• Cloud models benefit too — Anthropic and OpenAI both offer prompt caching; a stable prefix means more cache hits and lower cost/latency.

Context

Tested on Mac Studio M3 Ultra (256GB) running Qwen3-Coder-Next-8bit via mlx-openai-server. The server implements LRU trie-based prefix caching, but cache misses occur because small dynamic injections early in the prompt invalidate everything downstream. vllm-mlx has similar prefix caching that would benefit from this reordering.

This is a zero-risk change that would significantly improve local model performance and reduce cloud API costs.