fix(system-prompt): improve prompt cache locality with unique agent ID by mrx-arafat · Pull Request #23736 · openclaw/openclaw

mrx-arafat · 2026-02-22T16:50:46Z

Summary

5x API cost reduction by making system prompts unique per agent, improving prompt cache locality.

Problem

All OpenClaw users shared the identical system prompt prefix (~4k tokens):

"You are a personal assistant running inside OpenClaw."

With many users, requests are distributed across multiple machines. When user A's request hits machine 1, the prompt gets cached. When user A's next request hits machine 2, the cache misses because machine 2 has a different cache key (different user's prompt).

Result: ~5x higher API costs because most tokens could be cache reads but instead become input tokens.

Solution

Include the agent ID in the first line:

- "You are a personal assistant running inside OpenClaw."
+ "You are {agentId}, a personal assistant running inside OpenClaw."

Example:

User 1: "You are agent:user1:main, a personal assistant..."
User 2: "You are agent:user2:main, a personal assistant..."

Each user now has a unique prompt hash, so their requests consistently hit the same cache entries regardless of which machine serves them.

Changes

buildAgentSystemPrompt(): Uses runtimeInfo.agentId in first line
"none" prompt mode: Also uses unique first line
Added 3 tests verifying the fix

Impact

Before: 0-4k cache read tokens per request
After: High cache hit rate after first request
Cost reduction: ~80% (5x → 1x)

Test Coverage

✓ includes agent ID in first line for cache locality
✓ uses 'unknown' agent ID when runtimeInfo is not provided  
✓ uses unique first line in 'none' prompt mode

All 38 system-prompt tests pass.

Greptile Summary

This PR contains two unrelated fixes bundled together:

System Prompt Changes (for issue #23715):

Modified buildAgentSystemPrompt() to include agentId in the first line of system prompts
Updated both full and "none" prompt modes to use the unique first line
Added 3 test cases covering the agent ID inclusion

Critical Issue: The cache locality explanation appears logically contradictory - making prompts unique per user (by including agentId) would reduce cache sharing across users, not improve it. Before the change, all users shared identical prompt prefixes which would maximize cache hits. The PR description needs clarification on how unique prompts improve cache locality.

Update Check Changes (for issue #23647):

Added parseVersionWithBuild() to handle version strings with -N build suffixes
Modified compareSemverStrings() to treat -N as build increments (where higher N = newer)
Added comprehensive test coverage for build suffix comparisons

Note: The implementation assumes -N suffixes are always build numbers, not semver prereleases. Traditional prerelease tags like -beta.1 would not be correctly compared relative to release versions.

Process Issue: This PR bundles two unrelated fixes. Best practice would be separate PRs for distinct issues.

Confidence Score: 2/5

This PR has a critical logical issue in the system-prompt changes that needs clarification before merging
The score reflects a significant concern with the cache locality explanation which appears contradictory to basic caching principles. Making prompts unique per user reduces cache sharing rather than improving it. The update-check changes appear sound but lack edge case handling for semver prereleases. Additionally, bundling two unrelated fixes in one PR is non-standard practice.
src/agents/system-prompt.ts requires explanation of the cache locality claim; src/infra/update-check.ts should document prerelease version handling

_{Last reviewed commit: d881716}

_{(3/5) Reply to the agent's comments like "Can you suggest a fix for this @greptileai?" or ask follow-up questions!}

The update checker was incorrectly reporting versions like 2026.2.21-2 as older than 2026.2.21. This happened because the semver comparison ignored the -N build suffix entirely. Changes: - Add parseVersionWithBuild() to extract build numbers from version strings - Update compareSemverStrings() to treat -N suffix as build increment (higher build number = newer version) - Add comprehensive tests for build suffix comparison Fixes #23647

…cache locality Issue #23715: 5x API costs due to ineffective prompt caching Problem: All OpenClaw users shared the same system prompt prefix (~4k tokens), causing cache misses when requests were routed to different machines. This resulted in ~5x higher API costs than necessary. Solution: Include the agent ID in the first line of the system prompt: - Before: "You are a personal assistant running inside OpenClaw." - After: "You are {agentId}, a personal assistant running inside OpenClaw." This makes each user's system prompt hash unique, improving cache locality when requests are distributed across multiple machines. Changes: - Modified buildAgentSystemPrompt() to include runtimeInfo.agentId in first line - Updated "none" prompt mode to also use the unique first line - Added 3 new tests to verify the fix Fixes #23715

greptile-apps

_{4 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2026-02-22T16:54:06Z

+  // Build a unique first line per agent to improve prompt cache locality.
+  // Without this, all OpenClaw users share the same system prompt prefix,
+  // causing cache misses when requests are routed to different machines.
+  // Including the agent ID makes the hash unique per user/agent.
+  const agentId = runtimeInfo?.agentId ?? "unknown";
+  const firstLine = `You are ${agentId}, a personal assistant running inside OpenClaw.`;


The cache locality explanation appears contradictory. Making prompts unique per user (including agentId) means users can no longer share cache entries. The PR description states "all users shared the identical system prompt prefix" was causing cache misses, but sharing the same prefix would actually improve cache hits across users, not reduce them. With unique prefixes per user, each user needs their own cache entry. Could you clarify how making prompts unique improves cache locality?

Prompt To Fix With AI

This is a comment left during a code review. Path: src/agents/system-prompt.ts Line: 425-430 Comment: The cache locality explanation appears contradictory. Making prompts unique per user (including `agentId`) means users can no longer share cache entries. The PR description states "all users shared the identical system prompt prefix" was causing cache misses, but sharing the same prefix would actually improve cache hits across users, not reduce them. With unique prefixes per user, each user needs their own cache entry. Could you clarify how making prompts unique improves cache locality? How can I resolve this? If you propose a fix, please make it concise.

Read the issue for details. Sharing prefix across users indeed improves cache hits across users, but this only extends to the prefix that is common across users, i.e. 4k tokens. The rest of the prompt is user specific, constitutes a great majority of the context, and becomes a cache miss. We therefore do not want to share cache key across users.

greptile-apps · 2026-02-22T16:54:07Z

+  // Extract build suffix: "2026.2.21-2" -> build=2, "2026.2.21" -> build=0
+  const buildMatch = version.match(/-(\d+)(?:[+.]|$)/);
+  const build = buildMatch ? Number.parseInt(buildMatch[1], 10) : 0;


regex won't match semver prerelease tags like -beta.1, so 1.0.0-beta.1 and 1.0.0 would compare as equal (both build=0), violating semver conventions. consider adding test case or documentation clarifying -N are build numbers only, not prereleases

Prompt To Fix With AI

This is a comment left during a code review. Path: src/infra/update-check.ts Line: 358-360 Comment: regex won't match semver prerelease tags like `-beta.1`, so `1.0.0-beta.1` and `1.0.0` would compare as equal (both `build=0`), violating semver conventions. consider adding test case or documentation clarifying `-N` are build numbers only, not prereleases How can I resolve this? If you propose a fix, please make it concise.

svenssonaxel · 2026-02-23T03:42:09Z

+  // causing cache misses when requests are routed to different machines.
+  // Including the agent ID makes the hash unique per user/agent.
+  const agentId = runtimeInfo?.agentId ?? "unknown";
+  const firstLine = `You are ${agentId}, a personal assistant running inside OpenClaw.`;


Almost everyone has an agent with ID main or agent:main:main. So, this will often fail to create variation across users.

svenssonaxel

Read issue again. agentId won't necessarily vary across users.

steipete · 2026-02-23T18:50:18Z

Thanks for the investigation and the write-up.

Closing this PR as superseded / needs split:

The prompt-caching regression path discussed in [Bug]: Prompt cache constantly invalidated — cacheWrite dominates over cacheRead, causing 10x cost increase #19989/[Bug]: Inbound metadata injection into system prompt breaks Anthropic prompt caching (~100x cost increase) #20894 was fixed by merged #20597 (moving volatile per-turn IDs out of the system-prompt block). So this PR’s cache-locality change is no longer the right fix path.
The agentId-in-first-line approach is not a reliable per-install cache key (many installs share agent:main:main), so it does not robustly solve cross-install routing/cache locality.
This branch also bundles unrelated update-version comparison changes (#23647).

If you still want to land the update-check fix, please open a focused PR with only that change and tests.

Axis added 2 commits February 22, 2026 16:40

openclaw-barnacle Bot added agents Agent runtime and tooling size: S labels Feb 22, 2026

greptile-apps Bot reviewed Feb 22, 2026

View reviewed changes

svenssonaxel reviewed Feb 23, 2026

View reviewed changes

svenssonaxel suggested changes Feb 23, 2026

View reviewed changes

steipete closed this Feb 23, 2026

svenssonaxel mentioned this pull request Mar 2, 2026

[Bug]: 5x API costs due to ineffective prompt caching (Re-file of #23715) #31708

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(system-prompt): improve prompt cache locality with unique agent ID#23736

fix(system-prompt): improve prompt cache locality with unique agent ID#23736
mrx-arafat wants to merge 2 commits into
openclaw:mainfrom
mrx-arafat:fix-prompt-caching-cache-locality

mrx-arafat commented Feb 22, 2026 •

edited by greptile-apps Bot

Loading

Uh oh!

greptile-apps Bot left a comment

Uh oh!

greptile-apps Bot Feb 22, 2026

Uh oh!

svenssonaxel Feb 23, 2026

Uh oh!

greptile-apps Bot Feb 22, 2026

Uh oh!

svenssonaxel Feb 23, 2026

Uh oh!

svenssonaxel left a comment

Uh oh!

steipete commented Feb 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

mrx-arafat commented Feb 22, 2026 • edited by greptile-apps Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Problem

Solution

Changes

Impact

Test Coverage

Greptile Summary

Confidence Score: 2/5

Uh oh!

greptile-apps Bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Feb 22, 2026

Choose a reason for hiding this comment

Uh oh!

svenssonaxel Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Feb 22, 2026

Choose a reason for hiding this comment

Uh oh!

svenssonaxel Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

svenssonaxel left a comment

Choose a reason for hiding this comment

Uh oh!

steipete commented Feb 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mrx-arafat commented Feb 22, 2026 •

edited by greptile-apps Bot

Loading