fix(openai): enable prompt cache keys for Azure#98259
Conversation
|
Codex review: needs changes before merge. Reviewed July 25, 2026, 6:49 AM ET / 10:49 UTC. ClawSweeper reviewWhat this changesThe PR moves prompt-cache-key enablement into the shared OpenAI Chat Completions compatibility rules so direct OpenAI and Azure OpenAI endpoints automatically send stable session cache keys. Merge readinessKeep this PR open for one narrow correction: its shared Azure-host rule still omits regional Azure AI Foundry Priority: P2 Review scores
Verification
How this fits togetherOpenClaw resolves a model’s provider and endpoint compatibility before building OpenAI Chat Completions requests. Those resolved capabilities determine whether session cache-routing fields and long cache retention reach the upstream Azure or OpenAI endpoint. flowchart LR
A[Model and endpoint config] --> B[Completions compatibility rules]
B --> C[Resolved cache capabilities]
C --> D[Legacy and boundary-aware request builders]
D --> E[Azure OpenAI Chat Completions]
E --> F[Prompt-cache usage metrics]
Before merge
Findings
Agent review detailsSecurityNone. PR surfaceSource +58, Tests +43, Docs 0. Total +101 across 5 files. View PR surface stats
Review metrics
Stored data modelPersistent data-model change detected: Merge-risk optionsMaintainer options:
Copy recommended automerge instructionTechnical reviewBest possible solution: Add the regional Do we have a high-confidence way to reproduce the issue? Yes, from source: resolve compatibility for a custom OpenAI-compatible model with base URL Is this the best way to solve the issue? No. Centralizing the behavior in shared completions compatibility is the right shape, but the current implementation is not the best complete fix until it includes the regional Azure AI Foundry hostname family it claims to support. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against b8bb08a1ad37. LabelsLabel changes:
Label justifications:
EvidenceAcceptance criteria:
What I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (1 earlier review cycle)
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4570f5c205
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| provider === "azure-openai" || | ||
| baseUrl.includes(".openai.azure.com") || | ||
| baseUrl.includes(".cognitiveservices.azure.com") || | ||
| baseUrl.includes(".services.ai.azure.com"); |
There was a problem hiding this comment.
Include the regional Azure Foundry host in detection
When a custom OpenAI-compatible Chat Completions provider is pointed at an Azure AI Foundry regional endpoint such as https://westus.api.cognitive.microsoft.com/openai/v1, this URL-only Azure detection still returns false unless the provider id is exactly azure-openai. That host is documented by Microsoft for the same Foundry OpenAI API and is already treated as Azure-compatible elsewhere in the repo (src/shared/azure-openai-responses-client-compat.ts), so those Azure chat-completions users will continue to have supportsPromptCacheKey false and buildParams will drop their session prompt_cache_key.
Useful? React with 👍 / 👎.
| supportsLongCacheRetention: | ||
| !(isTogether || isCloudflareWorkersAI || isCloudflareAiGateway) && !isAzureOpenAI, |
There was a problem hiding this comment.
Honor Azure long prompt-cache retention
For Azure OpenAI models where the caller requests cacheRetention: "long", this new !isAzureOpenAI forces supportsLongCacheRetention false, so buildParams never emits prompt_cache_retention: "24h". Microsoft’s Azure OpenAI prompt-caching docs say prompt_cache_retention is configurable on Chat Completions and list GPT-5/GPT-4.1 families for extended retention, so this silently downgrades supported Azure deployments to short/in-memory retention and also regresses existing Azure configs that had only opted into supportsPromptCacheKey.
Useful? React with 👍 / 👎.
| sendSessionAffinityHeaders: false, | ||
| supportsPromptCacheKey: false, | ||
| supportsLongCacheRetention: !(isTogether || isCloudflareWorkersAI || isCloudflareAiGateway), | ||
| supportsPromptCacheKey: isAzureOpenAI, |
There was a problem hiding this comment.
Update the OpenClaw completions transport path
This auto-detection only changes the legacy src/llm/providers/openai-completions.ts builder, but embedded agent runs also route openai-completions through the boundary-aware transport (src/agents/provider-transport-stream.ts → src/agents/openai-transport-stream.ts), whose getCompat still enables prompt_cache_key only when model.compat.supportsPromptCacheKey === true. Azure Chat Completions sessions using that default transport will still omit the cache key, so the fix does not cover the main runtime path unless the same Azure detection is applied to the transport/extra-params gates.
Useful? React with 👍 / 👎.
4570f5c to
eb7f01f
Compare
|
@clawsweeper re-review Addressed the review findings: moved Azure prompt-cache-key capability into the shared completions compat defaults, wired the boundary-aware transport path to those defaults, added the regional .api.cognitive.microsoft.com Foundry host, preserved Azure long retention, added regression tests for both request paths, and updated the PR body with redacted live Azure proof showing 200 responses and cachedTokens=3840 on the repeat-prefix request. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: eb7f01f035
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| requiresReasoningContentOnAssistantMessages: | ||
| compatDefaults.requiresReasoningContentOnAssistantMessages, | ||
| requiresNonEmptyUserOrAssistantMessage: compatDefaults.requiresNonEmptyUserOrAssistantMessage, | ||
| supportsPromptCacheKey: compatDefaults.supportsPromptCacheKey, |
There was a problem hiding this comment.
Honor retention overrides for detected Azure completions
When an Azure openai-completions model relies on this new auto-detected support instead of explicit model.compat.supportsPromptCacheKey, embedded-agent model params such as cacheRetention: "none" or "long" are still dropped because src/agents/embedded-agent-runner/extra-params.ts:528-542 only treats explicit compat as cache-key eligible. In a normal session with sessionId, this line then makes the transport default to short retention: cacheRetention: "none" still emits prompt_cache_key, and cacheRetention: "long" omits the requested prompt_cache_retention: "24h". Please use the same Azure detection in the extra-params eligibility path before auto-enabling cache keys here.
Useful? React with 👍 / 👎.
|
🦞🧹 I asked ClawSweeper to review this item again. |
eb7f01f to
c851085
Compare
|
@clawsweeper re-review Addressed latest extra-params finding: embedded-agent cacheRetention eligibility now uses the same shared Azure OpenAI completions compat detection, while explicit compat.supportsPromptCacheKey overrides still win. Added regression tests for Azure cacheRetention long and none propagation through embedded-agent extra params. |
|
🦞🧹 I asked ClawSweeper to review this item again. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c851085959
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| | "supportsNativeStreamingUsageCompat" | ||
| | "supportsOpenAICompletionsStreamingUsageCompat" | ||
| | "usesExplicitProxyLikeEndpoint" | ||
| | "supportsPromptCacheKeyCompat" |
There was a problem hiding this comment.
This Pick<ProviderRequestCapabilities, ...> now includes supportsPromptCacheKeyCompat, but ProviderRequestCapabilities does not define that property in src/agents/provider-attribution.ts, so TypeScript rejects the file before the tests can run. Since this value is a local override passed into resolveOpenAICompletionsCompatDefaults, keep it outside the Pick (or add it to the source type) to restore typecheck/build.
Useful? React with 👍 / 👎.
c851085 to
8f71708
Compare
|
@clawsweeper re-review Addressed the remaining Codex/ClawSweeper findings on the latest head:
|
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
Codex Review: Didn't find any major issues. More of your lovely PRs please. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Azure OpenAI chat completions support prompt_cache_key, but OpenClaw only auto-enabled the field for api.openai.com. Move supportsPromptCacheKey into the OpenAI-completions compat matrix so first-party OpenAI and Azure OpenAI endpoints both opt in, and drop the hardcoded api.openai.com check from the transport. Other OpenAI-compatible proxies still stay off by default. Co-authored-by: Copilot <[email protected]>
8f71708 to
9718bf6
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 07775908d0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| baseUrlIncludes: [ | ||
| ".openai.azure.com", | ||
| ".services.ai.azure.com", | ||
| ".cognitiveservices.azure.com", |
There was a problem hiding this comment.
Parse Azure URLs before matching host suffixes
When a custom provider uses a valid uppercase Azure hostname such as https://TENANT.OPENAI.AZURE.COM/openai/v1, the raw, case-sensitive baseUrl.includes check misses it and omits the cache key. It can also misclassify proxies whose path, query, or longer hostname merely contains one of these fragments, causing unsupported cache fields to be sent despite the documented proxy opt-in requirement. Parse model.baseUrl and compare its normalized hostname using exact/suffix matching, as the sibling transport path does.
Useful? React with 👍 / 👎.
Codex P2: the OpenAI-completions compat matrix matched provider endpoints with a raw, case-sensitive baseUrl.includes() check. That missed valid uppercase Azure hostnames (e.g. https://TENANT.OPENAI.AZURE.COM/...) and could misclassify proxies whose path or query merely contained a known fragment. Parse model.baseUrl and compare its normalized (lowercased) hostname using exact/suffix/prefix matching, mirroring the sibling transport path (isAzureOpenAICompatibleHost). Co-authored-by: Copilot <[email protected]>
Summary
.api.cognitive.microsoft.comFoundry hostsprompt_cache_retention: "24h") when callers requestcacheRetention: "long", and preserve explicitcacheRetention: "none"What Problem This Solves
OpenClaw already gates OpenAI-compatible prompt cache keys behind
compat.supportsPromptCacheKey, but Azure OpenAI chat-completions endpoints were not auto-detected. Azure users therefore needed explicit per-model compat config before stable session prompt-cache keys were forwarded, which can silently reduce prompt-cache hit opportunities for long stable prefixes.The change treats Azure OpenAI chat-completions endpoints (
azure-openai,.openai.azure.com,.cognitiveservices.azure.com,.services.ai.azure.com, and.api.cognitive.microsoft.com) as prompt-cache-key capable by default. The default now lives in the shared completions compat path used by both request builders and by embedded-agent extra-param cache-retention eligibility; explicitcompat.supportsPromptCacheKey/compat.supportsLongCacheRetentionoverrides continue to win.Evidence
toolsbeforeinputin OpenAI Responses request body for prefix-cache stability #78747, [Bug]: Prompt-cache reads drop with no tracked cache input change #82939, DeepSeek cache hit rate <10% after 6.x upgrade - boundary-aware caching breaks prefix matching #94518, [Feature]: AddsupportsPromptCacheKeyto Mistral transport compat patch #83709, feat(agents): forward prompt_cache_key on openai-completions transport when compat.supportsPromptCacheKey is set #69272, fix(agents): add compat.supportsPromptCacheKey to opt out of prompt_cache_key strip #67427), but no open PR specifically enabling Azure OpenAI chat-completions prompt cache keys by default.mainonly enabled chat-completionsprompt_cache_keyforapi.openai.comor explicitcompat.supportsPromptCacheKey; Azure endpoint detection was absent in both relevant request paths and extra-params retention eligibility.prompt_cache_key, preserveprompt_cache_retention: "24h"for long retention, preservecacheRetention: "none", cover the regional Azure Foundry host, and propagate embedded-agentcacheRetentionfor detected Azure models.thomais2026v2-resource.openai.azure.com/openai/v1/chat/completionsusinggpt-5.5,prompt_cache_key, andprompt_cache_retention: "24h":{"label":"warmup","status":200,"ok":true,"model":"gpt-5.5-2026-04-24","elapsedMs":1328,"promptTokens":4090,"completionTokens":8,"cachedTokens":0,"hasPromptCacheKey":true,"requestedRetention":"24h","errorCode":null,"errorMessage":null} {"label":"repeat-prefix","status":200,"ok":true,"model":"gpt-5.5-2026-04-24","elapsedMs":1204,"promptTokens":4090,"completionTokens":5,"cachedTokens":3840,"hasPromptCacheKey":true,"requestedRetention":"24h","errorCode":null,"errorMessage":null}Test plan
pnpm exec vitest run src/llm/providers/openai-completions.test.ts --config test/vitest/vitest.unit.config.tspnpm exec vitest run src/agents/openai-transport-stream.test.ts --config test/vitest/vitest.agents.config.tsnode scripts/test-projects.mjs src/agents/openai-completions-compat.test.tsnode scripts/test-projects.mjs src/agents/embedded-agent-runner-extraparams.test.tspnpm exec oxfmt --check --threads=1 src/agents/embedded-agent-runner/extra-params.ts src/agents/embedded-agent-runner-extraparams.test.ts src/agents/openai-completions-compat.ts src/agents/openai-completions-compat.test.ts src/agents/openai-transport-stream.ts src/agents/openai-transport-stream.test.ts src/llm/providers/openai-completions.ts src/llm/providers/openai-completions.test.ts