feat(agents): add prompt cache break diagnostics#60707
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1521eb2ac7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Greptile SummaryThis PR adds embedded-runner prompt-cache diagnostics: a All findings are P2 (quality/robustness): the module-level Confidence Score: 5/5Safe to merge — all findings are P2 style/robustness suggestions with no present runtime defects. No P0 or P1 issues found. The three P2 comments (unbounded tracker map, diamond-ref digest, order-sensitive tool digest) are quality improvements rather than present bugs; the feature is opt-in (disabled unless OPENCLAW_CACHE_TRACE or debug logging is active), so the impact of the identified issues is further contained. src/agents/pi-embedded-runner/prompt-cache-observability.ts (tracker map growth + tool digest ordering) and src/agents/cache-trace.ts (stableStringify cycle detection).
|
| pendingChanges: PromptCacheChange[] | null; | ||
| }; | ||
|
|
||
| const trackers = new Map<string, PromptCacheTracker>(); |
There was a problem hiding this comment.
Unbounded module-level tracker map
trackers is never pruned. In long-running gateway processes that serve many distinct sessions, an entry is added (and never removed) for every unique sessionKey/sessionId string the runner sees. The CLAUDE.md prompt-cache guidelines call out correctness/perf-critical treatment for state that can grow without bound.
Consider a WeakMap keyed by a shared session object, or a bounded LRU eviction (e.g., remove the oldest entry when the map exceeds a soft cap), or at minimum a comment explaining why growth is expected to remain bounded for this deployment model.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/prompt-cache-observability.ts
Line: 48
Comment:
**Unbounded module-level tracker map**
`trackers` is never pruned. In long-running gateway processes that serve many distinct sessions, an entry is added (and never removed) for every unique `sessionKey`/`sessionId` string the runner sees. The CLAUDE.md prompt-cache guidelines call out correctness/perf-critical treatment for state that can grow without bound.
Consider a `WeakMap` keyed by a shared session object, or a bounded LRU eviction (e.g., remove the oldest entry when the map exceeds a soft cap), or at minimum a comment explaining why growth is expected to remain bounded for this deployment model.
How can I resolve this? If you propose a fix, please make it concise.| streamStrategy: params.streamStrategy, | ||
| transport: params.transport, | ||
| systemPromptDigest: digestText(params.systemPrompt), | ||
| toolDigest: digestText(JSON.stringify(params.toolNames)), |
There was a problem hiding this comment.
toolDigest is sensitive to tool registration order
digestText(JSON.stringify(params.toolNames)) is order-dependent. promptCacheToolNames is built from [...builtInTools, ...allCustomTools], so if either list's iteration order varies between turns (e.g., after a plugin reload or if custom tools arrive from a non-deterministic source), the digest will change and trigger a false "tools changed" cache-break entry, even when the actual tool set is identical.
Sorting the names before hashing would make the digest set-stable:
| toolDigest: digestText(JSON.stringify(params.toolNames)), | |
| toolDigest: digestText(JSON.stringify([...params.toolNames].sort())), |
Only do this if tool order is genuinely not cache-relevant for any supported provider; if providers like Anthropic or OpenAI cache the tool list positionally, keep the current order-sensitive behaviour and add a comment explaining it.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/prompt-cache-observability.ts
Line: 140
Comment:
**`toolDigest` is sensitive to tool registration order**
`digestText(JSON.stringify(params.toolNames))` is order-dependent. `promptCacheToolNames` is built from `[...builtInTools, ...allCustomTools]`, so if either list's iteration order varies between turns (e.g., after a plugin reload or if custom tools arrive from a non-deterministic source), the digest will change and trigger a false "tools changed" cache-break entry, even when the actual tool set is identical.
Sorting the names before hashing would make the digest set-stable:
```suggestion
toolDigest: digestText(JSON.stringify([...params.toolNames].sort())),
```
Only do this if tool order is genuinely not cache-relevant for any supported provider; if providers like Anthropic or OpenAI cache the tool list positionally, keep the current order-sensitive behaviour and add a comment explaining it.
How can I resolve this? If you propose a fix, please make it concise.1521eb2 to
054e072
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 054e072f44
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| function resolveProviderBaseUrl(model: LiveResolvedModel["model"]): string | undefined { | ||
| const candidate = (model as { baseUrl?: unknown }).baseUrl; | ||
| return typeof candidate === "string" && candidate.trim().length > 0 ? candidate : undefined; | ||
|
|
There was a problem hiding this comment.
Close resolveProviderBaseUrl before cache-trace helpers
This change drops the original return ... and closing brace in resolveProviderBaseUrl(model), so subsequent helper declarations are parsed inside that function and the file terminates with a '}' expected parse error. In this state src/agents/pi-embedded-runner.cache.live.test.ts cannot be parsed/executed, which breaks the live cache test path before any of the new assertions can run.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 25f6c89a14
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| function resolveProviderBaseUrl(fixture: LiveResolvedModel): string | undefined { | ||
| return resolveProviderBaseUrl(fixture.model); |
There was a problem hiding this comment.
Avoid shadowing resolveProviderBaseUrl with self-calling overload
The second resolveProviderBaseUrl declaration shadows the earlier implementation and then calls resolveProviderBaseUrl(fixture.model), so calls that pass a model (for example in buildEmbeddedRunnerConfig) no longer hit the model-based resolver and instead recurse into the shadowed function until it dereferences fixture.model on undefined and throws. This breaks live cache probe config assembly as soon as this helper is invoked.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 42791bee7d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (!liveCacheTraceFile) { | ||
| throw new Error("live cache trace file not initialized"); | ||
| } | ||
| const raw = await fs.readFile(liveCacheTraceFile, "utf8").catch(() => ""); |
There was a problem hiding this comment.
Wait for queued trace writes before asserting cache stages
readCacheTraceEvents reads the trace file once immediately, but cache-trace events are appended through the queued writer asynchronously, so the last cache:state/cache:result lines may still be in flight right after a run completes. In live CI (slow disk or busy workers), this creates nondeterministic failures where the stage assertion races the writer; add a short retry/poll (or flush hook) before asserting stage presence.
Useful? React with 👍 / 👎.
| cacheTrace.recordStage("cache:result", { | ||
| note: "state changed without a cache-read break", | ||
| options: { | ||
| cacheRead: attemptUsage?.cacheRead ?? 0, |
There was a problem hiding this comment.
Preserve missing usage in cache-result diagnostics
When a turn ends without usage metadata (getUsageTotals() can be undefined on abort/error), this code records cacheRead as 0, which makes trace output look like a hard cache miss rather than “no measurement.” That can mislead cache-break investigations, especially because this feature is intended for root-cause diagnostics; omit cacheRead when usage is absent or emit an explicit no-usage marker instead of coercing to zero.
Useful? React with 👍 / 👎.
* feat(agents): add prompt cache break diagnostics * test(agents): wire cache trace into live cache suite * fix(agents): always record cache trace result stage * feat(status): show cache reuse in verbose output * fix(agents): ignore missing prompt cache usage * chore(changelog): note prompt cache diagnostics * fix(agents): harden prompt cache diagnostics
* feat(agents): add prompt cache break diagnostics * test(agents): wire cache trace into live cache suite * fix(agents): always record cache trace result stage * feat(status): show cache reuse in verbose output * fix(agents): ignore missing prompt cache usage * chore(changelog): note prompt cache diagnostics * fix(agents): harden prompt cache diagnostics
* feat(agents): add prompt cache break diagnostics * test(agents): wire cache trace into live cache suite * fix(agents): always record cache trace result stage * feat(status): show cache reuse in verbose output * fix(agents): ignore missing prompt cache usage * chore(changelog): note prompt cache diagnostics * fix(agents): harden prompt cache diagnostics
* feat(agents): add prompt cache break diagnostics * test(agents): wire cache trace into live cache suite * fix(agents): always record cache trace result stage * feat(status): show cache reuse in verbose output * fix(agents): ignore missing prompt cache usage * chore(changelog): note prompt cache diagnostics * fix(agents): harden prompt cache diagnostics
* feat(agents): add prompt cache break diagnostics * test(agents): wire cache trace into live cache suite * fix(agents): always record cache trace result stage * feat(status): show cache reuse in verbose output * fix(agents): ignore missing prompt cache usage * chore(changelog): note prompt cache diagnostics * fix(agents): harden prompt cache diagnostics
* feat(agents): add prompt cache break diagnostics * test(agents): wire cache trace into live cache suite * fix(agents): always record cache trace result stage * feat(status): show cache reuse in verbose output * fix(agents): ignore missing prompt cache usage * chore(changelog): note prompt cache diagnostics * fix(agents): harden prompt cache diagnostics
Summary
status --verbose.Change Type (select all)
Scope (select all touched areas)
Linked Issue/PR
Root Cause (if applicable)
Regression Test Plan (if applicable)
src/agents/pi-embedded-runner/prompt-cache-observability.test.tssrc/agents/cache-trace.test.tssrc/agents/pi-embedded-runner.cache.live.test.tssrc/commands/status.format.test.tssrc/commands/status.test.tscache:statepluscache:resulttrace entries, andstatus --verboserenders an explicit cache-reuse summary for recent sessions.User-visible / Behavior Changes
status --verbosenow shows a per-session cache line in the Sessions table, for example40% hit · read 2.0k · write 1.0k.Diagram (if applicable)
Security Impact (required)
Yes/No) NoYes/No) NoYes/No) NoYes/No) NoYes/No) NoYes, explain risk + mitigation:Repro + Verification
Environment
OPENCLAW_CACHE_TRACE/ debug logging only; live suite now self-configures cache trace env vars into a temp fileSteps
openclaw status --verboseto inspect session-level cache reuse.OPENCLAW_LIVE_TEST=1 OPENCLAW_LIVE_CACHE_TEST=1to verify runner-path cache behavior and emitted trace stages together.Expected
cache:resultstage.status --verboserenders explicit cache reuse for recent sessions.Actual
Evidence
Human Verification (required)
pnpm exec vitest run --config vitest.config.ts --maxWorkers=1 src/agents/pi-embedded-runner/prompt-cache-observability.test.tspnpm exec vitest run --config vitest.config.ts --maxWorkers=1 src/agents/cache-trace.test.tspnpm exec vitest run --config vitest.config.ts --maxWorkers=1 src/commands/status.format.test.tsgit diff --checkcache:resulteventpnpm buildgreen state, because latestorigin/maincurrently fails in unrelatedsrc/agents/openai-ws-stream.tsattempt.test.ts, which remains non-reporting in this environmentsrc/commands/status.test.ts, which remains blocked here by the existingtelegram command config contract surface is unavailablefailure in the branch test stackReview Conversations
Compatibility / Migration
Yes/No) YesYes/No) NoYes/No) NoRisks and Mitigations
cache:stateandcache:result.status --verbosecould drift from the token suffix math over time.src/commands/status.format.test.tslocks that in.AI-assisted: yes.
Testing degree: lightly tested plus targeted unit coverage; live wiring added but not executed.