Problem

When running multiple sessions (e.g., 18+ sessions), identical or similar prompts may be routed to different GPU clusters by OpenRouter, causing cache misses and higher latency/cost.

Proposed Solution

Expose prompt_cache_key (or similar mechanism) in configuration to allow users to:

1. Route similar prompts to the same cache bucket
2. Improve cache hit rates for high-frequency patterns (e.g., heartbeat checks)
3. Optimize for multi-session setups

Use Case

• 18 active sessions with 1h heartbeat intervals
• Each heartbeat sends similar system context
• Currently cache misses due to random routing
• Would benefit from stable routing keys

Context

This request is inspired by Kimi K2.5's implementation guidance (via @Hxtu0 on X/Twitter), which highlights:

• Explicit prompt_cache_key helps route requests to appropriate clusters
• Setting session_id in metadata.user_id achieves similar routing
• Avoiding round hours (e.g., 10:00) prevents thundering herd issues

References

• OpenRouter prompt caching best practices: https://openrouter.ai/docs/guides/best-practices/prompt-caching
• Kimi K2.5 technical thread (cache key routing discussion)

Priority

Nice-to-have for cost optimization in high-session-count deployments.