fix(agents): add per-install prompt cache partition key to prevent cross-install cache-slot contention#25977
fix(agents): add per-install prompt cache partition key to prevent cross-install cache-slot contention#25977Water-Enjoyer wants to merge 3 commits into
Conversation
|
The scope is good and I expect the solution to partly solve the problem. However, it's not obvious that the prompt cache partition key should be unique per install. I can see a few scenarios where more granular partitions would be more suitable:
It is tempting to conclude that a per-session partition key makes the most sense, but there will be scenarios where it causes a cache miss for the system prompt when a per-install key would not. In order to decide on the best way forward, I believe we first need to know how common scenario (C) is:
|
|
Discussion continues here: #31708 (comment) |
|
This pull request has been automatically marked as stale due to inactivity. |
|
This pull request has been automatically marked as stale due to inactivity. |
|
Closing due to inactivity. |
Summary
Code changes in this PR were made with the help of AI.
Describe the problem and fix in 2–5 bullets:
agents.defaults.promptCachePartition, injected as the first system-prompt line, and threaded into OpenRouterprompt_cache_keyas an additive routing hint.Change Type (select all)
Scope (select all touched areas)
Linked Issue/PR
User-visible / Behavior Changes
agents.defaults.promptCachePartition.<!-- openclaw-cache-partition:<key> -->(includingpromptMode: "none").prompt_cache_keyfrom this partition unless already set or overridden.Security Impact (required)
No)No)No)No)No)Yes, explain risk + mitigation:Repro + Verification
Environment
openai-completions), Anthropic path unchangedagents.defaults.promptCachePartition = <auto-generated 32-hex>Steps
agents.defaults.promptCachePartition.Expected
prompt_cache_key(unless already present/overridden).Actual
Evidence
Attach at least one:
Human Verification (required)
What you personally verified (not just CI), and how:
Compatibility / Migration
Yes)Yes)No)Failure Recovery (if this breaks)
agents.defaults.promptCachePartitionand revert this PR.src/agents/prompt-cache-partition.tssrc/config/io.tssrc/agents/system-prompt.tssrc/agents/pi-embedded-runner/extra-params.tsprompt_cache_keydespite a configured partition.Risks and Mitigations
List only real risks for this PR. Add/remove entries as needed. If none, write
None.Greptile Summary
This PR adds per-install prompt cache partitioning to improve cache-read token efficiency and reduce cross-install cache slot contention. The implementation introduces an auto-generated, persisted
agents.defaults.promptCachePartitionconfig field that is injected as the first line of system prompts and threaded into OpenRouter'sprompt_cache_keyparameter.Key changes:
prompt-cache-partition.tsmodule withensurePromptCachePartitiongenerates a 32-character hex key if missingownerDisplaySecretpattern)<!-- openclaw-cache-partition:<key> -->as line 1 for all prompt modes including "none"prompt_cache_keyfrom partition unless already set or overridden via model paramsThe implementation follows existing patterns for auto-generated config fields, includes comprehensive test coverage, and maintains backward compatibility.
Confidence Score: 5/5
ownerDisplaySecretauto-generation), includes comprehensive unit tests for all new functionality, maintains backward compatibility, and has a clear scope boundary that explicitly excludes Anthropic-specific behavior. The changes are additive and non-breaking, with proper test coverage for edge cases like missing configs, existing values, and provider-specific integration.Last reviewed commit: 52b6e53
(2/5) Greptile learns from your feedback when you react with thumbs up/down!