perf: 150k+ token system prompt investigation — upstream analysis, cost impact, and optimization plan

## Summary

Our SELD system sends **~166k input tokens per API call** to Gemini 3.1 Pro on the first turn of a conversation. This was observed in Google AI Studio showing 166,669 input tokens with only 29 output tokens. This issue documents the investigation, upstream comparison, cost impact, and optimization plan.

**TL;DR**: This is **intentional upstream (OpenClaw) architecture**, not a bug in our fork. The upstream project designed these limits for Claude's 200k context window. Multiple upstream issues confirm the community recognizes this as problematic but it remains unresolved. We have clear optimization opportunities.

---

## Investigation: Image Analysis

The Google AI Studio screenshot shows:
- **Model**: `models/gemini-3.1-pro-preview`
- **Input tokens**: 166,669
- **Output tokens**: 29
- **Total tokens**: 166,698
- **Tools**: Function calling enabled

The system instruction begins with: *"You are a personal assistant running inside Seld. ## Tooling. Tool availability (filtered by policy)..."* — this is the full output of `buildAgentSystemPrompt()` plus bootstrap files plus tool schemas.

---

## Root Cause Analysis: Prompt Assembly Architecture

The system prompt is assembled in `src/agents/system-prompt.ts` by `buildAgentSystemPrompt()` and includes these components:

### Token Budget Breakdown (estimated first-turn)

| Component | Max Chars | Est. Tokens | Source |
|-----------|-----------|-------------|--------|
| **Hardcoded system prompt sections** (Tooling, Safety, Shell, Context Persistence, Credentials, CLI, Skills header, Memory, Docs, Reply Tags, Messaging, Silent Replies, Heartbeats, Runtime) | ~12,000 | ~3,000 | `system-prompt.ts` |
| **Skills prompt** (56 bundled skills formatted as XML blocks) | 30,000 max | ~7,500 | `skills/workspace.ts` DEFAULT_MAX_SKILLS_PROMPT_CHARS |
| **Bootstrap files** (AGENTS.md, SOUL.md, TOOLS.md, IDENTITY.md, USER.md, HEARTBEAT.md, BOOTSTRAP.md, MEMORY.md) | 150,000 max total (20,000 per file) | ~37,500 | `bootstrap.ts` DEFAULT_BOOTSTRAP_TOTAL_MAX_CHARS |
| **Tool JSON schemas** (25+ tools sent via API `tools` parameter) | ~60,000-80,000 | ~15,000-20,000 | `pi-tools.ts`, `seld-tools.ts` |
| **Core memory XML** (if configured) | Variable | Variable | `coreMemoryXml` param |
| **Config schema** (injected for gateway tool) | ~100,000+ | ~25,000+ | Upstream issue #9828 |
| **Inbound metadata** (per-message context) | Variable | Variable | `extraSystemPrompt` param |
| **User message** | ~200-1,000 | ~100-250 | First user turn |
| **Total (theoretical max)** | **~432,000+** | **~108,000+** | |

### Key Default Constants

```typescript
// bootstrap.ts
DEFAULT_BOOTSTRAP_MAX_CHARS = 20_000;      // per file
DEFAULT_BOOTSTRAP_TOTAL_MAX_CHARS = 150_000; // total across all files

// skills/workspace.ts
DEFAULT_MAX_SKILLS_PROMPT_CHARS = 30_000;
DEFAULT_MAX_SKILLS_IN_PROMPT = 150;
DEFAULT_MAX_SKILL_FILE_BYTES = 256_000;

// defaults.ts
DEFAULT_CONTEXT_TOKENS = 200_000;
```

### Key Files

| Purpose | File |
|---------|------|
| System prompt builder | `src/agents/system-prompt.ts` |
| Bootstrap limits/truncation | `src/agents/pi-embedded-helpers/bootstrap.ts` |
| Skills loading/formatting | `src/agents/skills/workspace.ts` |
| Tool creation (SELD-specific) | `src/agents/seld-tools.ts` |
| Tool creation (coding tools) | `src/agents/pi-tools.ts` |
| Tool schema normalization | `src/agents/pi-tools.schema.ts` |
| Tool adapter (API serialization) | `src/agents/pi-tool-definition-adapter.ts` |
| Prompt diagnostics | `src/agents/system-prompt-report.ts` |
| Context command | `src/auto-reply/reply/commands-context-report.ts` |
| Workspace file definitions | `src/agents/workspace.ts` |
| Compaction | `src/agents/compaction.ts` |
| Context pruning | `src/agents/pi-extensions/context-pruning/pruner.ts` |

---

## Upstream (OpenClaw) Comparison

### Diff from upstream

The diff between `upstream/main` and our fork shows **only cosmetic changes** to the prompt assembly:
- `OpenClawConfig` → `SeldConfig` (type rename)
- `DEFAULT_PROVIDER: "anthropic"` → `"google"` (our model switch)
- `DEFAULT_MODEL: "claude-opus-4-6"` → `"gemini-3.1-pro-preview"` (our model switch)
- Import reordering

**All token budget defaults are identical to upstream.** The 150k char bootstrap limit, 30k char skills limit, and 200k token context fallback are upstream's design.

### Relevant Upstream Issues (all OPEN)

| Issue | Title | Key Finding |
|-------|-------|-------------|
| **#9828** | Config schema injected into every session consumes ~100k+ tokens unnecessarily | **The same problem.** Config schema eats 50-75% of 200k context. Tagged `bug, stale`. |
| **#20894** | Inbound metadata injection breaks Anthropic prompt caching (~100x cost) | Per-message `message_id` in system prompt invalidates prefix cache every turn. |
| **#19989** | Prompt cache constantly invalidated (10x cost increase) | Users report $0.50/call for zero output. cacheWrite dominates. |
| **#14619** | Option to skip tool list text in system prompt | Tool summaries duplicated between system prompt text and API tool definitions. |
| **#21698** | Configurable tool scopes for sub-agents (reduce prompt bloat) | Sub-agents receive full tool set even when they only need 2-3 tools. |
| **#20430** | Per-message metadata invalidates llama.cpp KV cache every turn | Same cache invalidation problem affects local models. |
| **#16357** | Support `prompt_cache_key` metadata for provider-side cache routing | Feature request for explicit cache management. |

### Relevant Upstream Commits

| Commit | Description |
|--------|-------------|
| `8a67016` | **Raised** bootstrap total cap + added /context truncation warning |
| `412990a` | Reduced prompt token overhead with leaner context injections |
| `4f2c57e` | Compact skill paths with `~` to reduce prompt tokens (~400-600 tokens saved) |
| `dec6859` | Reduce prompt token bloat from exec and context |
| `b8f66c2` | Reduce subagent token waste |

**Key insight**: Upstream deliberately **raised** the bootstrap cap (commit `8a67016`), indicating the large prompt is intentional for power users who load extensive workspace files. The architecture prioritizes capability over efficiency.

---

## Cost Impact Analysis

### Per-Model Pricing (Feb 2026)

| Model | Input Cost (per 1M tokens) | Cached Input (if available) |
|-------|---------------------------|---------------------------|
| Gemini 3.1 Pro | ~$1.25/1M (>128k: $2.50/1M) | $0.31/1M (prompt caching) |
| Claude Opus 4.6 | $15.00/1M | $1.50/1M (prompt caching) |
| Claude Sonnet 4.6 | $3.00/1M | $0.30/1M (prompt caching) |

### Cost Per Message at 166k Input Tokens

| Model | Without Caching | With Caching | Savings |
|-------|----------------|--------------|---------|
| **Gemini 3.1 Pro** | $0.42 | $0.05 | 88% |
| **Claude Opus 4.6** | $2.50 | $0.25 | 90% |
| **Claude Sonnet 4.6** | $0.50 | $0.05 | 90% |

### Monthly Cost Projections (100 messages/day)

| Model | No Cache | With Cache | Optimized (50k base + cache) |
|-------|----------|------------|------------------------------|
| **Gemini 3.1 Pro** | **$1,260/mo** | $150/mo | $47/mo |
| **Claude Opus 4.6** | **$7,500/mo** | $750/mo | $225/mo |
| **Claude Sonnet 4.6** | **$1,500/mo** | $150/mo | $45/mo |

### The Compounding Problem

Every tool call round-trip resends the full system prompt + accumulated history. A single user request that triggers 5 tool calls sends ~830k+ input tokens total. Multi-turn conversations with 10+ exchanges can hit millions of tokens per session.

---

## Optimization Plan

### Tier 1: Quick Wins (config changes only, no code)

| Change | Savings | Risk |
|--------|---------|------|
| **Reduce `bootstrapTotalMaxChars` to 50,000** via config (`agents.defaults.bootstrapTotalMaxChars: 50000`) | ~25,000 tokens | Bootstrap files may truncate earlier; monitor via `/context` |
| **Reduce `maxSkillsPromptChars` to 15,000** via config (`skills.limits.maxSkillsPromptChars: 15000`) | ~3,750 tokens | Some skills may be excluded from prompt; they can still be loaded on-demand |
| **Trim workspace bootstrap files** (AGENTS.md, TOOLS.md) to essential content | Variable | Requires manual editing |
| **Estimated total savings**: ~30,000 tokens | | |

### Tier 2: Code Changes (moderate effort)

| Change | Savings | Complexity | Upstream-able? |
|--------|---------|-----------|----------------|
| **Lazy config schema loading** — Only inject config schema when user asks about config or invokes gateway tool. Move from system prompt to on-demand tool result. | ~25,000 tokens | Medium | Yes (addresses upstream #9828) |
| **Move inbound metadata out of system prompt** — Inject as user message context instead of `extraSystemPrompt` to preserve cache prefix. | Prevents cache invalidation | Medium | Yes (addresses upstream #20894) |
| **Conditional skill loading** — Only include skills relevant to the current channel/context. Add `primaryEnv` filtering. | ~2,000-5,000 tokens | Low | Yes |
| **Deduplicate tool summaries** — Tool summaries exist both in system prompt text AND in API tool descriptions. Remove from system prompt (addresses upstream #14619). | ~1,500 tokens | Low | Yes |
| **Sub-agent tool scoping** — Only include tools actually needed by sub-agents instead of the full set. | ~5,000-10,000 tokens per sub-agent | Medium | Yes (addresses upstream #21698) |
| **Estimated total savings**: ~35,000-45,000 tokens | | | |

### Tier 3: Architectural (higher effort, maximum impact)

| Change | Savings | Complexity | Upstream-able? |
|--------|---------|-----------|----------------|
| **Implement provider prompt caching** — Ensure system prompt prefix stability for Anthropic/Google cache. Add `prompt_cache_key` support. | 80-90% cost reduction on cached portion | High | Yes (addresses upstream #16357) |
| **Dynamic tool loading** — Register tools as "available" but only send full schemas when invoked. Use a tool manifest with descriptions + a `get_tool_schema` meta-tool. | ~10,000-15,000 tokens | High | Maybe |
| **Context budget manager** — Implement a budget-aware prompt assembler that allocates tokens across components based on model context window and current usage. | Adaptive savings | High | Maybe |
| **Estimated additional savings**: 80-90% cost reduction via caching | | | |

### Recommended Implementation Order

1. **Immediate** (Tier 1): Reduce config limits via SELD config file. Zero code changes, instant savings.
2. **Sprint 1** (Tier 2): Lazy config schema + move metadata out of system prompt. Biggest bang for buck.
3. **Sprint 2** (Tier 2+3): Prompt caching implementation + conditional skill loading.
4. **Backlog** (Tier 3): Dynamic tool loading, context budget manager.

### Expected Impact

| Phase | Base Prompt Tokens | Cost Reduction |
|-------|-------------------|----------------|
| Current | ~166,000 | Baseline |
| After Tier 1 | ~136,000 | ~18% |
| After Tier 2 | ~90,000-100,000 | ~40% |
| After Tier 3 (caching) | ~90,000 (but 80% cached) | ~90% |

---

## Challenges

1. **Upstream compatibility**: Any code changes need to work within OpenClaw's architecture to allow future syncs.
2. **Cache stability**: Keeping system prompt prefix stable across turns is critical for cost savings but conflicts with per-message metadata injection.
3. **Config schema removal risk**: The gateway tool relies on having the schema available. Lazy loading needs to ensure the agent can still correctly modify config when asked.
4. **Skill truncation**: Reducing skill budget may drop useful skills. Need good fallback (on-demand loading).

## Opportunities

1. **Upstream contribution**: Several of our optimizations address open upstream issues. Contributing back builds goodwill and ensures compatibility.
2. **Provider prompt caching**: Both Anthropic and Google offer prompt caching that can reduce input costs by 80-90%. This is the single highest-impact optimization.
3. **Diagnostic infrastructure exists**: The `/context` command and `system-prompt-report.ts` already provide measurement tools. We can add monitoring.
4. **Config-driven**: Most limits are already configurable via `seld.yaml` — Tier 1 optimizations are zero-code.

---

## References

- Upstream repo: https://github.com/openclaw/openclaw
- Upstream #9828: Config schema 100k+ token injection
- Upstream #20894: Metadata breaks prompt caching
- Upstream #19989: Cache invalidation 10x cost
- Upstream #14619: Tool list deduplication
- Upstream #21698: Sub-agent tool scoping
- Upstream commit `8a67016`: Bootstrap cap raise
- Upstream commit `412990a`: Prompt token reduction

---

*Generated by /heavy multi-agent analysis (4 Opus agents: Upstream Archaeologist, Token Forensics, Cost Analyst, Optimization Architect)*

Commit	Description
`8a67016`	Raised bootstrap total cap + added /context truncation warning
`412990a`	Reduced prompt token overhead with leaner context injections
`4f2c57e`	Compact skill paths with `~` to reduce prompt tokens (~400-600 tokens saved)
`dec6859`	Reduce prompt token bloat from exec and context
`b8f66c2`	Reduce subagent token waste

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

perf: 150k+ token system prompt investigation — upstream analysis, cost impact, and optimization plan #21999

Summary

Investigation: Image Analysis

Root Cause Analysis: Prompt Assembly Architecture

Token Budget Breakdown (estimated first-turn)

Key Default Constants

Key Files

Upstream (OpenClaw) Comparison

Diff from upstream

Relevant Upstream Issues (all OPEN)

Relevant Upstream Commits

Cost Impact Analysis

Per-Model Pricing (Feb 2026)

Cost Per Message at 166k Input Tokens

Monthly Cost Projections (100 messages/day)

The Compounding Problem

Optimization Plan

Tier 1: Quick Wins (config changes only, no code)

Tier 2: Code Changes (moderate effort)

Tier 3: Architectural (higher effort, maximum impact)

Recommended Implementation Order

Expected Impact

Challenges

Opportunities

References

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Component	Max Chars	Est. Tokens	Source
Hardcoded system prompt sections (Tooling, Safety, Shell, Context Persistence, Credentials, CLI, Skills header, Memory, Docs, Reply Tags, Messaging, Silent Replies, Heartbeats, Runtime)	~12,000	~3,000	`system-prompt.ts`
Skills prompt (56 bundled skills formatted as XML blocks)	30,000 max	~7,500	`skills/workspace.ts` DEFAULT_MAX_SKILLS_PROMPT_CHARS
Bootstrap files (AGENTS.md, SOUL.md, TOOLS.md, IDENTITY.md, USER.md, HEARTBEAT.md, BOOTSTRAP.md, MEMORY.md)	150,000 max total (20,000 per file)	~37,500	`bootstrap.ts` DEFAULT_BOOTSTRAP_TOTAL_MAX_CHARS
Tool JSON schemas (25+ tools sent via API `tools` parameter)	~60,000-80,000	~15,000-20,000	`pi-tools.ts`, `seld-tools.ts`
Core memory XML (if configured)	Variable	Variable	`coreMemoryXml` param
Config schema (injected for gateway tool)	~100,000+	~25,000+	Upstream issue #9828
Inbound metadata (per-message context)	Variable	Variable	`extraSystemPrompt` param
User message	~200-1,000	~100-250	First user turn
Total (theoretical max)	~432,000+	~108,000+

Purpose	File
System prompt builder	`src/agents/system-prompt.ts`
Bootstrap limits/truncation	`src/agents/pi-embedded-helpers/bootstrap.ts`
Skills loading/formatting	`src/agents/skills/workspace.ts`
Tool creation (SELD-specific)	`src/agents/seld-tools.ts`
Tool creation (coding tools)	`src/agents/pi-tools.ts`
Tool schema normalization	`src/agents/pi-tools.schema.ts`
Tool adapter (API serialization)	`src/agents/pi-tool-definition-adapter.ts`
Prompt diagnostics	`src/agents/system-prompt-report.ts`
Context command	`src/auto-reply/reply/commands-context-report.ts`
Workspace file definitions	`src/agents/workspace.ts`
Compaction	`src/agents/compaction.ts`
Context pruning	`src/agents/pi-extensions/context-pruning/pruner.ts`

Issue	Title	Key Finding
#9828	Config schema injected into every session consumes ~100k+ tokens unnecessarily	The same problem. Config schema eats 50-75% of 200k context. Tagged `bug, stale`.
#20894	Inbound metadata injection breaks Anthropic prompt caching (~100x cost)	Per-message `message_id` in system prompt invalidates prefix cache every turn.
#19989	Prompt cache constantly invalidated (10x cost increase)	Users report $0.50/call for zero output. cacheWrite dominates.
#14619	Option to skip tool list text in system prompt	Tool summaries duplicated between system prompt text and API tool definitions.
#21698	Configurable tool scopes for sub-agents (reduce prompt bloat)	Sub-agents receive full tool set even when they only need 2-3 tools.
#20430	Per-message metadata invalidates llama.cpp KV cache every turn	Same cache invalidation problem affects local models.
#16357	Support `prompt_cache_key` metadata for provider-side cache routing	Feature request for explicit cache management.

Model	Input Cost (per 1M tokens)	Cached Input (if available)
Gemini 3.1 Pro	~$1.25/1M (>128k: $2.50/1M)	$0.31/1M (prompt caching)
Claude Opus 4.6	$15.00/1M	$1.50/1M (prompt caching)
Claude Sonnet 4.6	$3.00/1M	$0.30/1M (prompt caching)

Model	Without Caching	With Caching	Savings
Gemini 3.1 Pro	$0.42	$0.05	88%
Claude Opus 4.6	$2.50	$0.25	90%
Claude Sonnet 4.6	$0.50	$0.05	90%

Model	No Cache	With Cache	Optimized (50k base + cache)
Gemini 3.1 Pro	$1,260/mo	$150/mo	$47/mo
Claude Opus 4.6	$7,500/mo	$750/mo	$225/mo
Claude Sonnet 4.6	$1,500/mo	$150/mo	$45/mo

Change	Savings	Risk
Reduce `bootstrapTotalMaxChars` to 50,000 via config (`agents.defaults.bootstrapTotalMaxChars: 50000`)	~25,000 tokens	Bootstrap files may truncate earlier; monitor via `/context`
Reduce `maxSkillsPromptChars` to 15,000 via config (`skills.limits.maxSkillsPromptChars: 15000`)	~3,750 tokens	Some skills may be excluded from prompt; they can still be loaded on-demand
Trim workspace bootstrap files (AGENTS.md, TOOLS.md) to essential content	Variable	Requires manual editing
Estimated total savings: ~30,000 tokens

Change	Savings	Complexity	Upstream-able?
Lazy config schema loading — Only inject config schema when user asks about config or invokes gateway tool. Move from system prompt to on-demand tool result.	~25,000 tokens	Medium	Yes (addresses upstream #9828)
Move inbound metadata out of system prompt — Inject as user message context instead of `extraSystemPrompt` to preserve cache prefix.	Prevents cache invalidation	Medium	Yes (addresses upstream #20894)
Conditional skill loading — Only include skills relevant to the current channel/context. Add `primaryEnv` filtering.	~2,000-5,000 tokens	Low	Yes
Deduplicate tool summaries — Tool summaries exist both in system prompt text AND in API tool descriptions. Remove from system prompt (addresses upstream #14619).	~1,500 tokens	Low	Yes
Sub-agent tool scoping — Only include tools actually needed by sub-agents instead of the full set.	~5,000-10,000 tokens per sub-agent	Medium	Yes (addresses upstream #21698)
Estimated total savings: ~35,000-45,000 tokens

Change	Savings	Complexity	Upstream-able?
Implement provider prompt caching — Ensure system prompt prefix stability for Anthropic/Google cache. Add `prompt_cache_key` support.	80-90% cost reduction on cached portion	High	Yes (addresses upstream #16357)
Dynamic tool loading — Register tools as "available" but only send full schemas when invoked. Use a tool manifest with descriptions + a `get_tool_schema` meta-tool.	~10,000-15,000 tokens	High	Maybe
Context budget manager — Implement a budget-aware prompt assembler that allocates tokens across components based on model context window and current usage.	Adaptive savings	High	Maybe
Estimated additional savings: 80-90% cost reduction via caching

Phase	Base Prompt Tokens	Cost Reduction
Current	~166,000	Baseline
After Tier 1	~136,000	~18%
After Tier 2	~90,000-100,000	~40%
After Tier 3 (caching)	~90,000 (but 80% cached)	~90%

Uh oh!

Uh oh!

perf: 150k+ token system prompt investigation — upstream analysis, cost impact, and optimization plan #21999

Description

Summary

Investigation: Image Analysis

Root Cause Analysis: Prompt Assembly Architecture

Token Budget Breakdown (estimated first-turn)

Key Default Constants

Key Files

Upstream (OpenClaw) Comparison

Diff from upstream

Relevant Upstream Issues (all OPEN)

Relevant Upstream Commits

Cost Impact Analysis

Per-Model Pricing (Feb 2026)

Cost Per Message at 166k Input Tokens

Monthly Cost Projections (100 messages/day)

The Compounding Problem

Optimization Plan

Tier 1: Quick Wins (config changes only, no code)

Tier 2: Code Changes (moderate effort)

Tier 3: Architectural (higher effort, maximum impact)

Recommended Implementation Order

Expected Impact

Challenges

Opportunities

References

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions