Skip to content

fix: stabilize isolated cron prompt cache affinity#87309

Merged
steipete merged 8 commits into
openclaw:mainfrom
ferminquant:codex/62809-cron-cache-affinity
May 27, 2026
Merged

fix: stabilize isolated cron prompt cache affinity#87309
steipete merged 8 commits into
openclaw:mainfrom
ferminquant:codex/62809-cron-cache-affinity

Conversation

@ferminquant

@ferminquant ferminquant commented May 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Fixes #62809

Summary

  • Add a stable hashed prompt-cache affinity key for embedded isolated cron runs.
  • Thread that key through the embedded runner into OpenClaw-managed OpenAI transports.
  • Preserve fresh isolated run session ids and existing session-bound and CLI cron behavior.

AI assistance disclosure

AI assistance was used. I reviewed and understand the affected cron executor, embedded runner, stream-resolution, and OpenAI transport paths.

Real behavior proof

Behavior addressed: recurring isolated cron runs now keep stable OpenAI prompt-cache affinity while preserving fresh run sessions.

Real environment tested: local source checkout, actual OpenAI transport parameter builders, no live credentials.

Exact steps or command run after this patch:

node --import tsx --input-type=module

with a small script invoking buildOpenAIResponsesParams and buildOpenAICompletionsParams using a sanitized cron-style promptCacheKey plus a distinct run sessionId.

Evidence after fix:

responses.prompt_cache_key=openclaw:cron:0123456789abcdef0123456789abcdef
responses.cache_disabled.prompt_cache_key=absent
completions.prompt_cache_key=openclaw:cron:0123456789abcdef0123456789abcdef
session_identity_still_separate=true

Observed result after fix: OpenAI Responses and opted-in OpenAI-compatible Completions payloads use the stable cron prompt-cache key, while cacheRetention: "none" omits prompt_cache_key and the run session identity remains separate.

What was not tested: live OpenAI cache-hit telemetry was not exercised because it would require real provider credentials and billing-side cache observations. Testbox pnpm check:changed was also blocked because the local Crabbox wrapper could not find a usable crabbox binary.

Root Cause

Isolated cron deliberately rotates the transcript/session identity for each run so recurring jobs do not accumulate an ever-growing transcript. The OpenAI payload builders used sessionId as the prompt_cache_key fallback, so the same recurring isolated cron job also rotated its prompt-cache affinity across runs.

Dependency contract

OpenAI's SDK exposes prompt_cache_key on Responses and Chat Completions request params as the cache-affinity field for similar requests. The local @earendil-works/[email protected] stream options expose sessionId and cacheRetention, but no first-class promptCacheKey; its OpenAI providers use sessionId for prompt_cache_key when prompt caching is enabled. OpenClaw therefore needs a separate internal option so cache affinity can be stable without changing session semantics.

Regression Test Plan

Focused tests cover:

  • isolated cron embedded runs get a stable hashed prompt-cache key across fresh run sessions
  • session-bound cron and CLI cron do not receive the embedded prompt-cache key
  • runEmbeddedPiAgent forwards the key into the embedded attempt
  • stream resolution propagates the key without overwriting caller-supplied values
  • OpenAI Responses and opted-in Completions prefer promptCacheKey over sessionId, and omit it when caching is disabled

Security Impact

No credentials, lockfiles, config files, or secret-handling paths changed. The upstream cache key is derived from stable cron/provider/model identity but sent as a truncated SHA-256 digest, so raw cron names, local session labels, and per-run session ids are not exposed in prompt_cache_key.

Human Verification

I reviewed the affected runtime path from cron execution through embedded runner stream resolution and the OpenAI transport builders, then rechecked the diff against issue #62809 after focused validation.

CODEOWNERS / owner review

.github/CODEOWNERS only names src/cron/stagger.ts and src/cron/service/jobs.ts under cron. Neither file is touched, so no direct CODEOWNERS owner gate was found for this diff.

Changelog

No CHANGELOG.md edit. This is a normal contributor bugfix PR.

Review conversations

codex review --base origin/main reported no actionable correctness issues after the final patch.

Risks

Live OpenAI cache-hit telemetry was not tested. The local proof validates the outgoing request payload contract, not OpenAI's server-side cache accounting.

Behavior tradeoffs

The patch keeps transcript/session isolation and transport session identity separate from prompt-cache affinity. That means recurring isolated cron jobs can share cache buckets across runs without sharing a transcript.

Scope boundaries

This targets OpenClaw-managed embedded isolated cron OpenAI transport paths. It does not change session-bound cron jobs, CLI cron execution, prompt-prefix construction, source-dependent tool ordering, or arbitrary custom stream implementations that do not use OpenClaw's managed OpenAI transports.

Validation

node scripts/run-vitest.mjs src/cron/isolated-agent/run.session-key-isolation.test.ts src/agents/pi-embedded-runner/stream-resolution.test.ts src/agents/pi-embedded-runner/run.before-agent-reply-cron.test.ts src/agents/openai-transport-stream.test.ts src/agents/pi-embedded-runner/prompt-cache-retention.test.ts
Test Files  8 passed (8)
Tests       469 passed (469)

pnpm exec oxfmt --check --threads=1 src/agents/openai-transport-stream.ts src/agents/openai-transport-stream.test.ts src/agents/pi-embedded-runner/prompt-cache-retention.test.ts src/cron/isolated-agent/run-executor.ts src/cron/isolated-agent/run.session-key-isolation.test.ts src/agents/pi-embedded-runner/run/params.ts src/agents/pi-embedded-runner/run.ts src/agents/pi-embedded-runner/run/attempt.ts src/agents/pi-embedded-runner/stream-resolution.ts src/agents/pi-embedded-runner/stream-resolution.test.ts src/agents/pi-embedded-runner/run.before-agent-reply-cron.test.ts
All matched files use the correct format.

git diff --check
passed

codex review --base origin/main
No actionable correctness issues found

Blocked broad validation:

node scripts/crabbox-wrapper.mjs run --provider blacksmith-testbox --blacksmith-org openclaw --blacksmith-workflow .github/workflows/ci-check-testbox.yml --blacksmith-job check --blacksmith-ref main --idle-timeout 90m --ttl 240m --timing-json -- corepack pnpm check:changed
[crabbox] selected binary failed basic --version/--help sanity checks

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: M proof: supplied External PR includes structured after-fix real behavior proof. labels May 27, 2026
@clawsweeper

clawsweeper Bot commented May 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed May 27, 2026, 7:30 PM ET / 23:30 UTC.

Summary
This PR adds a stable hashed promptCacheKey for embedded isolated cron runs and threads it through embedded stream resolution, OpenAI-family transports, proxy serialization, and regression tests.

PR surface: Source +101, Tests +328. Total +429 across 23 files.

Reproducibility: yes. for the source-level payload identity: current main derives OpenAI prompt-cache affinity from rotating run session ids, and isolated cron intentionally creates fresh run sessions. Live OpenAI cache-hit telemetry was not reproduced in this review.

Review metrics: 1 noteworthy metric.

  • Plugin-facing stream option: 1 added. StreamOptions.promptCacheKey is re-exported through src/plugin-sdk/llm.ts, so this is an API-surface change maintainers should notice before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Confirm whether promptCacheKey should be a plugin-facing stream option or an internal embedded-run field.
  • Get green required validation for the final head, including the changed-check lane if available.

Risk before merge

  • The diff adds promptCacheKey to plugin-facing StreamOptions, so maintainers should confirm that this is the intended public provider/plugin contract before merge.
  • Live OpenAI cache-hit telemetry and broad Testbox check:changed proof were not supplied; the current proof validates outgoing payloads and focused tests, not server-side cache accounting.

Maintainer options:

  1. Accept the stream option contract
    Treat promptCacheKey as the public provider/plugin stream-affinity option and merge once the final head has green required validation.
  2. Keep cache affinity internal
    Move the key through an embedded-run-only internal path if maintainers do not want to publish this SDK option yet.

Next step before merge
The remaining action is maintainer confirmation of the plugin-facing stream option plus final validation, not a narrow automated code repair.

Security
Cleared: No concrete security or supply-chain regression was found; the cron-derived key is hashed and the diff does not touch secrets, lockfiles, or dependency sources.

Review details

Best possible solution:

Land after maintainers confirm the new promptCacheKey stream-option contract is intended and final CI/changed validation is green; otherwise keep the cache-affinity key internal to embedded cron plumbing.

Do we have a high-confidence way to reproduce the issue?

Yes for the source-level payload identity: current main derives OpenAI prompt-cache affinity from rotating run session ids, and isolated cron intentionally creates fresh run sessions. Live OpenAI cache-hit telemetry was not reproduced in this review.

Is this the best way to solve the issue?

Mostly yes: a stable hashed cron key preserves transcript isolation while fixing transport affinity. The remaining solution-fit question is whether exposing promptCacheKey through plugin-facing stream options is the intended API boundary.

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 92043f754796.

Label changes

Label changes:

  • add merge-risk: 🚨 compatibility: The PR adds a new plugin-facing stream option, which creates an SDK contract decision even though the field is optional.
  • add proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes terminal output from a local source checkout invoking actual OpenAI payload builders after the patch; live cache telemetry was not exercised.

Label justifications:

  • P2: This is a normal-priority bug fix for isolated cron prompt-cache efficiency with a bounded provider/runtime surface.
  • merge-risk: 🚨 compatibility: The PR adds a new plugin-facing stream option, which creates an SDK contract decision even though the field is optional.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes terminal output from a local source checkout invoking actual OpenAI payload builders after the patch; live cache telemetry was not exercised.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes terminal output from a local source checkout invoking actual OpenAI payload builders after the patch; live cache telemetry was not exercised.
Evidence reviewed

PR surface:

Source +101, Tests +328. Total +429 across 23 files.

View PR surface stats
Area Files Added Removed Net
Source 15 113 12 +101
Tests 8 334 6 +328
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 23 447 18 +429

What I checked:

Likely related people:

  • steipete: Current-main blame for the existing OpenAI prompt-cache and isolated cron paths points to Peter Steinberger, and the final PR commits also come from this account. (role: recent area contributor; confidence: high; commits: 171675b54be9, 5ca161da153d; files: src/cron/isolated-agent/run-executor.ts, src/llm/providers/openai-responses.ts, src/llm/providers/openai-completions.ts)
  • vincentkoc: Recent git history on the provider transport surface includes fix(provider): honor Codex response timeouts, making this a plausible adjacent routing/review owner. (role: recent provider-runtime contributor; confidence: medium; commits: c20a055341ac; files: src/llm/providers/openai-codex-responses.ts, src/agents/openai-transport-stream.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels May 27, 2026
@clawsweeper

clawsweeper Bot commented May 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

✨ Hatched: 🥚 common Brave Test Hopper

Hatch command

Comment @clawsweeper hatch when this PR is hatchable.

Hatchability rules:

  • Merged PRs are hatchable.
  • Open PRs are hatchable when they are status: 👀 ready for maintainer look, status: 🚀 automerge armed, or labeled clawsweeper:automerge.
  • Closed unmerged PRs are hatchable only when one of those hatchable labels is still present in the durable record.

Rarity: 🥚 common.
Trait: sniffs out flaky tests.
Image traits: location diff observatory; accessory CI status badge; palette pearl, teal, and neon green; mood bright-eyed; pose pointing at a small proof artifact; shell woven fiber shell; lighting moonlit rim light; background miniature CI buoys.
Share on X: post this hatch
Copy: My PR egg hatched a 🥚 common Brave Test Hopper in ClawSweeper.

What is this egg doing here?
  • Eggs appear after the PR passes real-behavior proof. It is here for vibes, not verdicts: it does not change labels, ratings, merge decisions, or automation.
  • The shell reacts to review momentum: open follow-up work warms it up, re-review makes it wobble, and a clean final review lets it hatch.
  • Hatchability usually comes from sufficient real-behavior proof, no blocking P0/P1/P2 findings, no security attention needed, and clean correctness. A merged PR is already final, so merge makes the egg hatchable independently.
  • The hatch is seeded from this repository and PR number, so the same PR keeps the same creature; the reviewed head SHA can only change safe visual details.
  • Rarity is just collectible sparkle: 🥚 common, 🌱 uncommon, 💎 rare, ✨ glimmer, and 🌈 legendary.

@ferminquant
ferminquant force-pushed the codex/62809-cron-cache-affinity branch from 108b9e2 to cba0e70 Compare May 27, 2026 15:10
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 27, 2026
@ferminquant

ferminquant commented May 27, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented May 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. P2 Normal backlog priority with limited blast radius. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels May 27, 2026
@ferminquant
ferminquant marked this pull request as ready for review May 27, 2026 15:32
@ferminquant ferminquant mentioned this pull request May 27, 2026
Closed
@ferminquant
ferminquant force-pushed the codex/62809-cron-cache-affinity branch from cba0e70 to 34c55eb Compare May 27, 2026 18:02
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 27, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 27, 2026
@ferminquant
ferminquant force-pushed the codex/62809-cron-cache-affinity branch from 34c55eb to 9077c83 Compare May 27, 2026 21:17
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 27, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 27, 2026
@steipete steipete self-assigned this May 27, 2026
@steipete
steipete force-pushed the codex/62809-cron-cache-affinity branch from 9077c83 to 6a1acda Compare May 27, 2026 22:02
@clawsweeper clawsweeper Bot added rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. and removed proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels May 27, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. labels May 27, 2026
@steipete
steipete force-pushed the codex/62809-cron-cache-affinity branch from ac11225 to 5ca161d Compare May 27, 2026 23:24
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 27, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. labels May 27, 2026
@steipete

steipete commented May 27, 2026

Copy link
Copy Markdown
Contributor

Verification for rewritten branch 5ca161d:

Behavior addressed: isolated cron runs now use a stable prompt cache affinity key that is separate from the per-run session id and is forwarded through embedded runner, provider, proxy, custom stream, and observability paths.

Real environment tested: local macOS checkout plus GitHub Actions PR CI.

Exact steps or command run after this patch:

  • pnpm test src/agents/runtime/proxy.test.ts src/llm/providers/openai-codex-responses.test.ts src/cron/isolated-agent/run.session-key-isolation.test.ts src/agents/embedded-agent-runner/prompt-cache-observability.test.ts src/agents/embedded-agent-runner/stream-resolution.test.ts src/agents/openai-transport-stream.test.ts
  • pnpm check:test-types
  • autoreview --mode branch --base origin/main
  • gh pr checks 87309 --watch --fail-fast --interval 15

Evidence after fix: local focused Vitest passed 3 shards / 246 tests, local test typecheck passed, autoreview reported no accepted/actionable findings, and GitHub CI run 26544652281 passed required checks including checks-node-core-runtime-cron-isolated-agent and checks-node-agentic-agents.

Observed result after fix: promptCacheKey is stable for isolated cron, valid for OpenAI prompt_cache_key, clamped before OpenAI-compatible payloads, suppressed when cache retention is disabled, and preserved across provider/proxy/custom stream paths.

What was not tested: no live provider API call; this was verified through unit coverage and PR CI.

@steipete
steipete merged commit 3f9d241 into openclaw:main May 27, 2026
118 of 120 checks passed
@ferminquant ferminquant mentioned this pull request May 28, 2026
Open
@ferminquant
ferminquant deleted the codex/62809-cron-cache-affinity branch May 28, 2026 11:05
This was referenced Jun 25, 2026
Merged
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@steipete steipete

Labels

agents Agent runtime and tooling extensions: codex merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: M status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Recurring isolated cron jobs have unstable first-turn OpenAI prompt-cache hits across runs

2 participants

@ferminquant @steipete