feat: expose prompt-cache runtime context to context engines#62179
Conversation
openclaw-barnacle
Bot
added
agents
a0d1f1c to
8f3d3ab
Compare
Greptile SummaryThis PR adds a typed Confidence Score: 5/5Safe to merge; all findings are P2 style suggestions. No P0 or P1 issues found. The implementation is type-safe, additive, and backwards-compatible. The single P2 note is a test-coverage gap on the overflow path, which mirrors working code that is already covered on the timeout path. No files require special attention before merging.
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a0d1f1c301
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const lastCallUsage = normalizeUsage( | ||
| (lastAssistant as { usage?: UsageLike } | undefined)?.usage, | ||
| ); |
There was a problem hiding this comment.
lastCallUsage is computed from the most recent assistant message in the entire snapshot, not from messages produced by the current attempt. When a turn exits before producing a new assistant message (for example preflight overflow/skip-prompt paths or early prompt failures), this picks a prior-turn assistant and forwards stale usage as runtimeContext.promptCache.lastCallUsage, which can mislead context engines during post-turn maintenance and recovery compaction decisions. Restrict the lookup to messages added in this attempt (e.g. after prePromptMessageCount) or use current-attempt usage data when no new assistant exists.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 09b03925ec
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| observation: promptCacheObservation, | ||
| lastCacheTouchAt: readLastCacheTtlTimestamp(sessionManager), | ||
| }); |
There was a problem hiding this comment.
lastCacheTouchAt is sourced from readLastCacheTtlTimestamp(sessionManager) without checking whether that TTL entry matches the current provider/model, so a prior Anthropic/Gemini turn can leak a stale timestamp into runtimeContext.promptCache for a later turn on a different model/provider. This makes cache-aware engines think the current cache was touched recently when it was not, which can skew compaction/maintenance decisions after provider switches.
Useful? React with 👍 / 👎.
| const lastCallUsage = | ||
| normalizeUsage((currentAttemptAssistant as { usage?: UsageLike } | undefined)?.usage) ?? | ||
| normalizeUsage(attemptUsage); |
There was a problem hiding this comment.
When no current-attempt assistant message is found, lastCallUsage falls back to attemptUsage, but attemptUsage is a turn aggregate (getUsageTotals) rather than the most recent API call. In compaction/retry flows this can combine multiple model calls, so runtimeContext.promptCache.lastCallUsage over-reports the latest call and violates the field’s own contract, which can mislead context engines that use per-call cache-read deltas.
Useful? React with 👍 / 👎.
09b0392 to
5147a47
Compare
|
Nice work @jalehman this is clean and well-scoped. The A few observations from review: What I like:
Potential edge cases to consider:
Why this matters to us: We're working on a cache keep-warm feature (#62475) that fires minimal Happy to help test or address any reviewer feedback if useful. |
Add a typed prompt-cache payload to the context-engine runtime context and populate it from the embedded runner's resolved retention, last-call usage, cache-break observation, and cache-touch metadata. Also pass the same payload through the retry compaction runtime context when a run attempt already has it. Regeneration-Prompt: | Expose OpenClaw prompt-cache telemetry to context engines in a narrow, additive way without changing compaction policy. Keep the public change on the OpenClaw side only: add a typed promptCache payload to the context-engine runtime context, thread it into afterTurn, and also into compact where the existing run loop already has the data cheaply available. Use OpenClaw's resolved cache retention, not raw config. Use last-call usage for the new payload, not accumulated retry or tool-loop totals. Reuse the existing prompt-cache observability result and tracked change causes instead of inventing a new heuristic. If cache-touch metadata is already available from the cache-TTL bookkeeping, include it; do not invent expiry timestamps for providers where OpenClaw cannot know them confidently. Keep the interface backward-compatible for engines that ignore the new field. Add focused tests around the existing attempt/context-engine helpers and the compaction runtime-context propagation path rather than broad new integration coverage.
Regeneration-Prompt: | Fix PR openclaw#62179 so context-engine prompt-cache metadata uses only the current attempt's usage. The review comment pointed out that early exits could reuse a prior turn's assistant usage when no new assistant message was produced. Restrict the prompt-cache lastCallUsage lookup to assistant messages added after prePromptMessageCount, and fall back to current-attempt usage totals instead of stale snapshot history. Also repair the PR's new context-engine test typings and add a regression test for the stale prior-turn case. Two import-only fixes in doctor-state-integrity and config/talk were already broken on origin/main, but they blocked build/check and the gateway-watch regression harness, so include the minimum unblocking imports as well.
624d5f5 to
fa96c07
Compare
🔒 Aisle Security AnalysisWe found 1 potential security issue(s) in this PR:
1. 🟡 Untrusted session custom entry can spoof cache-ttl timestamp to bypass context pruning (resource exhaustion)
DescriptionThe cache-ttl based pruning logic trusts the latest
Vulnerable code: const ts = typeof data?.timestamp === "number" ? data.timestamp : null;
if (ts && Number.isFinite(ts)) {
last = ts;
break;
}This value is later used in TTL logic: if (ttlMs > 0 && Date.now() - lastTouch < ttlMs) {
return undefined;
}RecommendationTreat session-provided timestamps as untrusted input.
Example hardening in const now = Date.now();
const ts = typeof data?.timestamp === "number" ? data.timestamp : null;
if (typeof ts === "number" && Number.isFinite(ts)) {
// allow up to 5 minutes of skew
const maxAllowed = now + 5 * 60 * 1000;
if (ts > 0 && ts <= maxAllowed) {
last = ts;
break;
}
}And/or defensively handle in the TTL check: const safeLastTouch = Math.min(lastTouch, Date.now());
if (Date.now() - safeLastTouch < ttlMs) return;Analyzed PR: #62179 at commit Last updated on: 2026-04-07T16:15:05Z |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fa96c07a8e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| .slice(Math.max(0, params.prePromptMessageCount)) | ||
| .toReversed() | ||
| .find((message) => message.role === "assistant"); |
There was a problem hiding this comment.
findCurrentAttemptAssistantMessage slices from prePromptMessageCount, but the same attempt can run SDK compaction that rewrites history and shrinks messagesSnapshot (the code later notes compaction can restructure messages). In that case the start index is past the end of the compacted snapshot, so the slice is empty and lastCallUsage is dropped even though this attempt produced an assistant response. That under-reports runtimeContext.promptCache.lastCallUsage specifically in compaction/retry flows where cache-aware engines need accurate per-call telemetry.
Useful? React with 👍 / 👎.
Persist prompt-cache telemetry after turns and use it to gate incremental leaf compaction. Hot cache sessions now defer best-effort incremental passes unless raw history pressure is clearly above target, while cold cache sessions can run bounded catch-up passes in a single maintenance cycle. Full threshold sweeps keep their existing behavior. Also add cacheAwareCompaction config/schema support, docs, a changeset, and regression coverage for hot/cold/unknown prompt-cache behavior. Regeneration-Prompt: | Implement the cache-aware incremental compaction spec for lossless-claw using the new prompt-cache telemetry exposed by the dependent OpenClaw branch tied to openclaw/openclaw#62179. Persist lightweight per- conversation cache telemetry after each turn, classify sessions as hot, cold, or unknown, and use that state to decide whether afterTurn() should run incremental leaf compaction. Preserve the existing full-sweep compaction behavior, but let cold-cache sessions do a bounded number of extra leaf passes to catch up while hot-cache sessions defer passes unless raw history pressure is clearly above target. Add the minimal config surface for enabling the feature and setting the cold-cache pass cap, keep the plugin manifest and docs in sync, and cover the behavior with focused engine and config tests.
…izing (#318) * feat: add cache-aware incremental compaction Persist prompt-cache telemetry after turns and use it to gate incremental leaf compaction. Hot cache sessions now defer best-effort incremental passes unless raw history pressure is clearly above target, while cold cache sessions can run bounded catch-up passes in a single maintenance cycle. Full threshold sweeps keep their existing behavior. Also add cacheAwareCompaction config/schema support, docs, a changeset, and regression coverage for hot/cold/unknown prompt-cache behavior. Regeneration-Prompt: | Implement the cache-aware incremental compaction spec for lossless-claw using the new prompt-cache telemetry exposed by the dependent OpenClaw branch tied to openclaw/openclaw#62179. Persist lightweight per- conversation cache telemetry after each turn, classify sessions as hot, cold, or unknown, and use that state to decide whether afterTurn() should run incremental leaf compaction. Preserve the existing full-sweep compaction behavior, but let cold-cache sessions do a bounded number of extra leaf passes to catch up while hot-cache sessions defer passes unless raw history pressure is clearly above target. Add the minimal config surface for enabling the feature and setting the cold-cache pass cap, keep the plugin manifest and docs in sync, and cover the behavior with focused engine and config tests. * feat: add dynamic leaf chunk sizing Add the next compaction spec on top of cache-aware incremental compaction. Incremental maintenance can now grow its working leaf chunk target in busy sessions using internal low/medium/high activity bands, keep the configured static leafChunkTokens value as the floor, and cap growth at a bounded max. When cache-aware compaction is enabled and the prompt cache is cold, incremental compaction now jumps to the max working chunk. This also persists minimal refill telemetry alongside the existing compaction telemetry, threads optional leaf chunk overrides through the incremental compaction path, and retries with smaller chunk targets when a provider rejects an oversized compaction request on token/context-window limits. Full sweeps remain unchanged. Regeneration-Prompt: | Implement the dynamic leafChunkTokens spec from the 2026-04-07 Pagedrop page on top of the existing cache-aware incremental compaction branch. Keep the feature default-off in v1. Reuse the static leafChunkTokens as the floor, add only a minimal dynamicLeafChunkTokens config object with enabled and max, and store lightweight per-conversation refill telemetry needed to derive a simple low/medium/high activity band with internal hysteresis. Use that band to choose a working incremental leaf chunk target, but keep full sweeps unchanged. If cache-aware compaction is enabled and the cache is cold, force incremental compaction to use the max working chunk. Clamp the working chunk against budget-derived limits, and if a provider still rejects an oversized chunk due to token/context window limits, retry with the next smaller chunk target instead of failing immediately. Update the plugin manifest, docs, migration/store schema, and regression tests for config parsing, trigger overrides, cold-cache max bumping, and retry fallback behavior. * chore: add debug logs for compaction decisions Add debug-level tracing around the new cache-aware incremental compaction and dynamic leaf chunk sizing paths so live runs can be diagnosed without querying SQLite directly. This logs telemetry updates, incremental decision inputs and reasons, and leaf compaction start/result state, with focused engine coverage for the new messages. Regeneration-Prompt: | User asked for better observability for the two new incremental compaction features added on this branch: cache-aware prompt-cache handling and dynamic leaf chunk sizing. The requirement was to add debug logs, not info logs, and to explain how to enable those logs in a live OpenClaw instance. Inspect the new policy code in the LCM engine and add low-noise debug traces at the decision points that matter operationally: telemetry persistence after afterTurn, the incremental compaction decision with cache state / activity band / chosen chunk / reason / max passes, reset after a successful leaf compaction pass, and compactLeafAsync start/result. Preserve existing behavior and keep the logs structured enough to grep in production. Add focused tests that prove the debug logger is called for the hot-cache telemetry path, the hot-cache defer path, and the dynamic high-band chunk selection path.
…w#62179) * Context engine: plumb prompt cache runtime context Add a typed prompt-cache payload to the context-engine runtime context and populate it from the embedded runner's resolved retention, last-call usage, cache-break observation, and cache-touch metadata. Also pass the same payload through the retry compaction runtime context when a run attempt already has it. Regeneration-Prompt: | Expose OpenClaw prompt-cache telemetry to context engines in a narrow, additive way without changing compaction policy. Keep the public change on the OpenClaw side only: add a typed promptCache payload to the context-engine runtime context, thread it into afterTurn, and also into compact where the existing run loop already has the data cheaply available. Use OpenClaw's resolved cache retention, not raw config. Use last-call usage for the new payload, not accumulated retry or tool-loop totals. Reuse the existing prompt-cache observability result and tracked change causes instead of inventing a new heuristic. If cache-touch metadata is already available from the cache-TTL bookkeeping, include it; do not invent expiry timestamps for providers where OpenClaw cannot know them confidently. Keep the interface backward-compatible for engines that ignore the new field. Add focused tests around the existing attempt/context-engine helpers and the compaction runtime-context propagation path rather than broad new integration coverage. * Agents: fix prompt-cache afterTurn usage Regeneration-Prompt: | Fix PR openclaw#62179 so context-engine prompt-cache metadata uses only the current attempt's usage. The review comment pointed out that early exits could reuse a prior turn's assistant usage when no new assistant message was produced. Restrict the prompt-cache lastCallUsage lookup to assistant messages added after prePromptMessageCount, and fall back to current-attempt usage totals instead of stale snapshot history. Also repair the PR's new context-engine test typings and add a regression test for the stale prior-turn case. Two import-only fixes in doctor-state-integrity and config/talk were already broken on origin/main, but they blocked build/check and the gateway-watch regression harness, so include the minimum unblocking imports as well. * Agents: document prompt-cache context * Agents: address prompt-cache review feedback * Doctor: drop unused isRecord import
…w#62179) * Context engine: plumb prompt cache runtime context Add a typed prompt-cache payload to the context-engine runtime context and populate it from the embedded runner's resolved retention, last-call usage, cache-break observation, and cache-touch metadata. Also pass the same payload through the retry compaction runtime context when a run attempt already has it. Regeneration-Prompt: | Expose OpenClaw prompt-cache telemetry to context engines in a narrow, additive way without changing compaction policy. Keep the public change on the OpenClaw side only: add a typed promptCache payload to the context-engine runtime context, thread it into afterTurn, and also into compact where the existing run loop already has the data cheaply available. Use OpenClaw's resolved cache retention, not raw config. Use last-call usage for the new payload, not accumulated retry or tool-loop totals. Reuse the existing prompt-cache observability result and tracked change causes instead of inventing a new heuristic. If cache-touch metadata is already available from the cache-TTL bookkeeping, include it; do not invent expiry timestamps for providers where OpenClaw cannot know them confidently. Keep the interface backward-compatible for engines that ignore the new field. Add focused tests around the existing attempt/context-engine helpers and the compaction runtime-context propagation path rather than broad new integration coverage. * Agents: fix prompt-cache afterTurn usage Regeneration-Prompt: | Fix PR openclaw#62179 so context-engine prompt-cache metadata uses only the current attempt's usage. The review comment pointed out that early exits could reuse a prior turn's assistant usage when no new assistant message was produced. Restrict the prompt-cache lastCallUsage lookup to assistant messages added after prePromptMessageCount, and fall back to current-attempt usage totals instead of stale snapshot history. Also repair the PR's new context-engine test typings and add a regression test for the stale prior-turn case. Two import-only fixes in doctor-state-integrity and config/talk were already broken on origin/main, but they blocked build/check and the gateway-watch regression harness, so include the minimum unblocking imports as well. * Agents: document prompt-cache context * Agents: address prompt-cache review feedback * Doctor: drop unused isRecord import
…w#62179) * Context engine: plumb prompt cache runtime context Add a typed prompt-cache payload to the context-engine runtime context and populate it from the embedded runner's resolved retention, last-call usage, cache-break observation, and cache-touch metadata. Also pass the same payload through the retry compaction runtime context when a run attempt already has it. Regeneration-Prompt: | Expose OpenClaw prompt-cache telemetry to context engines in a narrow, additive way without changing compaction policy. Keep the public change on the OpenClaw side only: add a typed promptCache payload to the context-engine runtime context, thread it into afterTurn, and also into compact where the existing run loop already has the data cheaply available. Use OpenClaw's resolved cache retention, not raw config. Use last-call usage for the new payload, not accumulated retry or tool-loop totals. Reuse the existing prompt-cache observability result and tracked change causes instead of inventing a new heuristic. If cache-touch metadata is already available from the cache-TTL bookkeeping, include it; do not invent expiry timestamps for providers where OpenClaw cannot know them confidently. Keep the interface backward-compatible for engines that ignore the new field. Add focused tests around the existing attempt/context-engine helpers and the compaction runtime-context propagation path rather than broad new integration coverage. * Agents: fix prompt-cache afterTurn usage Regeneration-Prompt: | Fix PR openclaw#62179 so context-engine prompt-cache metadata uses only the current attempt's usage. The review comment pointed out that early exits could reuse a prior turn's assistant usage when no new assistant message was produced. Restrict the prompt-cache lastCallUsage lookup to assistant messages added after prePromptMessageCount, and fall back to current-attempt usage totals instead of stale snapshot history. Also repair the PR's new context-engine test typings and add a regression test for the stale prior-turn case. Two import-only fixes in doctor-state-integrity and config/talk were already broken on origin/main, but they blocked build/check and the gateway-watch regression harness, so include the minimum unblocking imports as well. * Agents: document prompt-cache context * Agents: address prompt-cache review feedback * Doctor: drop unused isRecord import
…w#62179) * Context engine: plumb prompt cache runtime context Add a typed prompt-cache payload to the context-engine runtime context and populate it from the embedded runner's resolved retention, last-call usage, cache-break observation, and cache-touch metadata. Also pass the same payload through the retry compaction runtime context when a run attempt already has it. Regeneration-Prompt: | Expose OpenClaw prompt-cache telemetry to context engines in a narrow, additive way without changing compaction policy. Keep the public change on the OpenClaw side only: add a typed promptCache payload to the context-engine runtime context, thread it into afterTurn, and also into compact where the existing run loop already has the data cheaply available. Use OpenClaw's resolved cache retention, not raw config. Use last-call usage for the new payload, not accumulated retry or tool-loop totals. Reuse the existing prompt-cache observability result and tracked change causes instead of inventing a new heuristic. If cache-touch metadata is already available from the cache-TTL bookkeeping, include it; do not invent expiry timestamps for providers where OpenClaw cannot know them confidently. Keep the interface backward-compatible for engines that ignore the new field. Add focused tests around the existing attempt/context-engine helpers and the compaction runtime-context propagation path rather than broad new integration coverage. * Agents: fix prompt-cache afterTurn usage Regeneration-Prompt: | Fix PR openclaw#62179 so context-engine prompt-cache metadata uses only the current attempt's usage. The review comment pointed out that early exits could reuse a prior turn's assistant usage when no new assistant message was produced. Restrict the prompt-cache lastCallUsage lookup to assistant messages added after prePromptMessageCount, and fall back to current-attempt usage totals instead of stale snapshot history. Also repair the PR's new context-engine test typings and add a regression test for the stale prior-turn case. Two import-only fixes in doctor-state-integrity and config/talk were already broken on origin/main, but they blocked build/check and the gateway-watch regression harness, so include the minimum unblocking imports as well. * Agents: document prompt-cache context * Agents: address prompt-cache review feedback * Doctor: drop unused isRecord import
…w#62179) * Context engine: plumb prompt cache runtime context Add a typed prompt-cache payload to the context-engine runtime context and populate it from the embedded runner's resolved retention, last-call usage, cache-break observation, and cache-touch metadata. Also pass the same payload through the retry compaction runtime context when a run attempt already has it. Regeneration-Prompt: | Expose OpenClaw prompt-cache telemetry to context engines in a narrow, additive way without changing compaction policy. Keep the public change on the OpenClaw side only: add a typed promptCache payload to the context-engine runtime context, thread it into afterTurn, and also into compact where the existing run loop already has the data cheaply available. Use OpenClaw's resolved cache retention, not raw config. Use last-call usage for the new payload, not accumulated retry or tool-loop totals. Reuse the existing prompt-cache observability result and tracked change causes instead of inventing a new heuristic. If cache-touch metadata is already available from the cache-TTL bookkeeping, include it; do not invent expiry timestamps for providers where OpenClaw cannot know them confidently. Keep the interface backward-compatible for engines that ignore the new field. Add focused tests around the existing attempt/context-engine helpers and the compaction runtime-context propagation path rather than broad new integration coverage. * Agents: fix prompt-cache afterTurn usage Regeneration-Prompt: | Fix PR openclaw#62179 so context-engine prompt-cache metadata uses only the current attempt's usage. The review comment pointed out that early exits could reuse a prior turn's assistant usage when no new assistant message was produced. Restrict the prompt-cache lastCallUsage lookup to assistant messages added after prePromptMessageCount, and fall back to current-attempt usage totals instead of stale snapshot history. Also repair the PR's new context-engine test typings and add a regression test for the stale prior-turn case. Two import-only fixes in doctor-state-integrity and config/talk were already broken on origin/main, but they blocked build/check and the gateway-watch regression harness, so include the minimum unblocking imports as well. * Agents: document prompt-cache context * Agents: address prompt-cache review feedback * Doctor: drop unused isRecord import
2 participants
Summary
runtimeContext.promptCachepayload, populated it in the embedded runner forafterTurn(), and forwarded it intocompact()on the existing recovery paths that already have the attempt result.Change Type (select all)
Scope (select all touched areas)
Linked Issue/PR
Root Cause (if applicable)
Regression Test Plan (if applicable)
src/agents/pi-embedded-runner/run/attempt.spawn-workspace.context-engine.test.ts,src/agents/pi-embedded-runner/run.timeout-triggered-compaction.test.ts, and the existingbuildAfterTurnRuntimeContexttests insrc/agents/pi-embedded-runner/run/attempt.test.ts.afterTurn()receives prompt-cache info when available, the field stays absent/partial when unavailable, retention and cache-break observations are threaded through correctly, andcompact()receives the same payload on the current recovery paths.buildAfterTurnRuntimeContextcoverage remains in place and stays compatible with callers that ignore the new field.User-visible / Behavior Changes
None.
Diagram (if applicable)
Security Impact (required)
Yes, explain risk + mitigation:Repro + Verification
Environment
Steps
pnpm test src/agents/pi-embedded-runner/run/attempt.spawn-workspace.context-engine.test.ts src/agents/pi-embedded-runner/run.timeout-triggered-compaction.test.ts.pnpm test src/agents/pi-embedded-runner/run/attempt.test.ts -t "buildAfterTurnRuntimeContext".Expected
runtimeContext.promptCacheis available to context engines when the embedded runner has cache telemetry.Actual
Evidence
Human Verification (required)
buildAfterTurnRuntimeContextcoverage, against the final committed tree.promptCacheabsent, effective retention is the resolved value, and cache-break observations carry through when the heuristic reports a meaningful drop.Review Conversations
Compatibility / Migration
Risks and Mitigations
compact()only receives prompt-cache info on recovery paths that already have an attempt result.expiresAtis intentionally absent for Anthropic/OpenAI because OpenClaw cannot know it confidently.