Skip to content

[FEATURE]: Set prompt_cache_ttl for OpenRouter provider #16848

Closed as not planned

Description

@rndmcnlly

[FEATURE]: Set prompt_cache_ttl for OpenRouter provider

Feature hasn't been suggested before.

  • I have verified this feature I'm about to request hasn't been suggested before.

(Note: #5416 and #5422 cover broader caching config. This is about one missing parameter on an existing code path.)

Describe the enhancement you want to request

We already set prompt_cache_key for OpenRouter in ProviderTransform.options() (packages/opencode/src/provider/transform.ts, line 790), but we don't set a TTL. OpenRouter defaults to 5 minutes.

Without a longer TTL, any idle gap over 5 minutes between prompts triggers a full cache miss and rewrite on the next turn. For Anthropic models this is common in real usage — reviewing output, editing files, thinking, etc. The 5-minute default ends up costing more overall because the cache is constantly being rewritten.

Caution

The fix originally sketched below is broken. The top-level prompt_cache_ttl parameter is silently ignored by OpenRouter — no error, no cache write, nothing. See the conclusive empirical research in #16850, including a 3-way verification script that tested with a 6-minute delay past the default TTL.

The fix is one line next to the existing prompt_cache_key assignment:

// BROKEN — silently ignored by OpenRouter
if (input.model.providerID === "openrouter") {
  result["prompt_cache_key"] = input.sessionID
  result["prompt_cache_ttl"] = 3600 // does nothing
}

The correct documented mechanism is cache_control: { type: "ephemeral", ttl: "1h" } on message content (OpenRouter docs). This is a one-line change in the existing applyCaching function. The fix is implemented and tested in #16850.

Activity

  1. added
    coreAnything pertaining to core functionality of the application (opencode server stuff)
    on Mar 10, 2026
  2. rndmcnlly commented on Mar 10, 2026

    @rndmcnlly
    Author

    Adding some empirical backing: analyzing my last 30 days of claude-4.6-opus usage via OpenRouter (~6,000 calls), modeling a 5-minute vs. 1-hour TTL shows my Opus bill dropping from $443 to $319 — a 41% reduction. The savings come almost entirely from 710 calls that were cold cache misses under the 5-min TTL but would have been hits under 1-hour. The higher write cost of the 1-hour tier only costs back ~$17 against ~$137 in miss savings — roughly an 8:1 payoff ratio. I also swept a hybrid strategy (stay on 5-min early in a session, upgrade to 1-hour after some threshold) and the savings curve is monotonically decreasing: there's no crossover point where staying on 5-min longer pays off. Always upgrading immediately is optimal.

  3. added a commit that references this issue on Mar 13, 2026
    40e83a5
  4. rndmcnlly commented on Mar 13, 2026

    @rndmcnlly
    Author

    @thdxr — the fix for this is ready in #16850 (one-line diff + test, just squashed clean).

    The approach originally described in this issue was wrong (prompt_cache_ttl is silently ignored), but the PR has conclusive empirical testing showing the correct mechanism: cache_control: { type: "ephemeral", ttl: "1h" } on message content, which is already the documented OpenRouter approach.

    This has real cost impact — every opencode user on OpenRouter/Anthropic is paying for full cache rewrites whenever they pause for more than 5 minutes between prompts. Extending to 1h eliminates most of those redundant writes.

    Would you mind taking a look when you get a chance?

  5. assigned and unassigned on Apr 26, 2026
  6. removed
    coreAnything pertaining to core functionality of the application (opencode server stuff)
    on May 3, 2026
  7. github-actions commented on Aug 1, 2026

    @github-actions
    Contributor

    To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions