Skip to content

fix(agents): restore runtime context fallback for heartbeat prompts#91817

Closed
solodmd wants to merge 4 commits into
openclaw:mainfrom
solodmd:fix/runtime-context-prompt-fallback
Closed

fix(agents): restore runtime context fallback for heartbeat prompts#91817
solodmd wants to merge 4 commits into
openclaw:mainfrom
solodmd:fix/runtime-context-prompt-fallback

Conversation

@solodmd

@solodmd solodmd commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Problem: During the pi-embedded-runnerembedded-agent-runner refactor, resolveRuntimeContextPromptParts lost a legacy fallback. When transcriptPrompt is defined but not a substring of effectivePrompt (e.g. heartbeat placeholder [OpenClaw heartbeat poll] vs exec event output), the effective text should be used as implicit runtimeContext. Without it, buildRuntimeContextCustomMessage skips creating the custom_message, and the model doesn't receive the runtime context on heartbeat turns.
  • Why now: Discovered while investigating exec background completion notification behavior — exec events trigger a heartbeat wake, but the runtime context containing the exec result was silently dropped.
  • Outcome: Restored the fallback so exec event data flows correctly through runtimeContextbuildRuntimeContextCustomMessagecustom_message.
  • Intentionally out of scope: Persisting the custom_message across turns (current ephemeral design is intentional per 0503853 — only visible on the current turn).
  • Success: Unit test covers the fallback path; temporary console.log in attempt.ts confirmed runtimeContextForHook is populated (303 chars) and custom_message is created.
  • Reviewer focus: The condition transcriptPrompt !== undefined && hiddenRuntimeContext === undefined && extracted.runtimeContext === undefined — verify it correctly distinguishes "not found in text" from "found but no surrounding context."

Linked context

No related issue directly — discovered while investigating exec background completion notification behavior.

This is a refactoring omission: the fallback was present in the old pi-embedded-runner and lost during the rename/restructure to embedded-agent-runner.

  • Related: #85341 — the refactor that moved pi-embedded-runner to embedded-agent-runner, where the fallback was dropped.
  • Related: #86875 — subsequent fix to resolveRuntimeContextPromptParts for hook context handling.

Real behavior proof

  • Behavior or issue addressed: Exec-event heartbeat wake does not inject runtime context custom_message into model messages.
  • Real environment tested: Local gateway with agent model: zte/Qwen3-235B-A22B, ran exec command, triggered heartbeat.
  • Exact steps or command run after this patch: Added temporary console.log at the buildRuntimeContextCustomMessage call site, rebuilt, ran gateway, executed exec sleep 5 background=true, waited for heartbeat wake.
  • Evidence after fix (terminal capture):
    [runtime-context-v] hook present, chars: 303; 
    [runtime-context-v] custom msg created: true
    
    Full evidence (temp logging code + log output): fix(agents): restore runtime context fallback for heartbeat prompts #91817 (comment)
  • Observed result after fix: runtimeContextForHook contains the exec event text (303 chars), buildRuntimeContextCustomMessage successfully creates the custom_message.
  • What was not tested: The actual model response to the custom_message (depends on provider/model behavior). Verification limited to confirming the custom_message object is built and injected into messagesForCurrentPrompt.
  • Proof limitations: The custom_message is ephemeral (not persisted to transcript/jsonl), so proof relies on temporary logging at the construction site. The unit test in runtime-context-prompt.test.ts provides structural coverage of the extraction logic.
  • Before evidence: Without the fix, runtimeContextForHook is undefinedbuildRuntimeContextCustomMessage(undefined) returns undefined → no custom_message created. This is the current behavior on main.

Tests and validation

  • pnpm test src/agents/embedded-agent-runner/run/runtime-context-prompt.test.ts: 22/22 pass.
  • Added test "uses effective text as implicit runtime context when transcriptPrompt is not found" — covers the exact heartbeat/exec-event fallback scenario.
  • All existing tests for marker-based runtime context extraction, modelPrompt splitting, and runtime-only events remain unchanged.
  • Also ran live gateway test (described above) with temporary logging to verify the fix triggers on actual exec events.

Risk checklist

  • Did user-visible behavior change? No — internal runtime context; only affects model-visible messages on exec-event heartbeat turns (restoring behavior that existed in pi-embedded-runner).
  • Did config, environment, or migration behavior change? No
  • Did security, auth, secrets, network, or tool execution behavior change? No
  • What is the highest-risk area? The fallback condition transcriptPrompt !== undefined && hiddenRuntimeContext === undefined && extracted.runtimeContext === undefined — if a future caller passes a transcriptPrompt that unintentionally doesn't match effectivePrompt, the entire effective text becomes runtimeContext. This is narrowly scoped and was the previous behavior.
  • How is that risk mitigated? Unit test covering the exact fallback path; gated by three conditions that ensure it only fires when no other context mechanism provided data.

Current review state

  • Next action: Author ready for review.
  • Waiting on: None.
  • Comments addressed: N/A (first review).

resolveRuntimeContextPromptParts lost the fallback during the
pi-embedded-runner to embedded-agent-runner refactor. When transcriptPrompt
is not found in effectivePrompt (e.g. heartbeat placeholder vs exec event),
use extracted text as implicit runtimeContext.

Verification: pnpm test runtime-context-prompt.test.ts (22/22 pass)
@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: XS triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels Jun 10, 2026
@clawsweeper

clawsweeper Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 10, 2026, 3:49 AM ET / 07:49 UTC.

Summary
The PR adds a heartbeat-only unmatched transcript fallback in resolveRuntimeContextPromptParts, rewrites hook-expanded model prompts to avoid duplicate event text, and adds focused runtime-context prompt tests.

PR surface: Source +27, Tests +75. Total +102 across 3 files.

Reproducibility: yes. at source level: current main returns the heartbeat transcript placeholder without runtimeContext when the actual effective prompt does not contain that placeholder. The PR's focused unit case exercises that failing shape directly.

Review metrics: none identified.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Next step before merge

  • [P2] No repair lane is needed; this open PR already contains the focused code and regression tests, with no blocking finding for automation to fix.

Security
Cleared: The diff changes only agent prompt-splitting logic and tests; it does not touch secrets, dependencies, CI, install scripts, or other supply-chain surfaces.

Review details

Best possible solution:

Land the focused heartbeat-only fallback after normal maintainer review and CI, keeping the runtime context ephemeral and avoiding new config or persistence surfaces.

Do we have a high-confidence way to reproduce the issue?

Yes, at source level: current main returns the heartbeat transcript placeholder without runtimeContext when the actual effective prompt does not contain that placeholder. The PR's focused unit case exercises that failing shape directly.

Is this the best way to solve the issue?

Yes; this is the best observed fix shape because the fallback is restored in the prompt-splitting helper but is enabled only by the heartbeat call site. Non-heartbeat mismatches remain unchanged, and hook-expanded model prompts are handled without duplicating the event text.

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against db5b883a9cb7.

Label changes

Label changes:

  • add proof: sufficient: Contributor real behavior proof is sufficient. The PR supplies after-fix local gateway logs showing runtime context length and custom-message creation for the heartbeat exec-event path, plus focused unit coverage.
  • add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • remove rating: 🐚 platinum hermit: Current PR rating is rating: 🦞 diamond lobster, so this older rating label is no longer current.

Label justifications:

  • P1: The PR fixes a broken heartbeat agent workflow where exec completion event context can be omitted from the model turn.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (logs): The PR supplies after-fix local gateway logs showing runtime context length and custom-message creation for the heartbeat exec-event path, plus focused unit coverage.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR supplies after-fix local gateway logs showing runtime context length and custom-message creation for the heartbeat exec-event path, plus focused unit coverage.
Evidence reviewed

PR surface:

Source +27, Tests +75. Total +102 across 3 files.

View PR surface stats
Area Files Added Removed Net
Source 2 29 2 +27
Tests 1 75 0 +75
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 3 104 2 +102

What I checked:

Likely related people:

  • Shakker: git blame attributes the current main version of the embedded runner runtime-context helper and attempt call site to commit 3b7631e. (role: recent area contributor; confidence: medium; commits: 3b7631e50db9; files: src/agents/embedded-agent-runner/run/runtime-context-prompt.ts, src/agents/embedded-agent-runner/run/attempt.ts)
  • steipete: The broad internalized-agent-runtime refactor in merged PR refactor: internalize OpenClaw agent runtime #85341 created the current OpenClaw-owned embedded-runner surface that this PR repairs. (role: refactor author; confidence: medium; commits: bb46b79d3c14; files: src/agents/embedded-agent-runner/run/runtime-context-prompt.ts)
  • Alix-007: Merged PR fix(agents): keep hook context in embedded prompts #86875 substantially changed resolveRuntimeContextPromptParts for hook/runtime-context handling, adjacent to this fallback. (role: adjacent runtime-context fix author; confidence: high; commits: 8b7a4826a1ff; files: src/agents/embedded-agent-runner/run/runtime-context-prompt.ts, src/agents/embedded-agent-runner/run/runtime-context-prompt.test.ts)
  • vincentkoc: The live PR timeline shows vincentkoc assigned, and the current PR head includes follow-up commits scoping and de-duplicating the fallback behavior. (role: recent branch maintainer; confidence: medium; commits: cbac10427f67, 61f893495dee, 36cee42c1ef6; files: src/agents/embedded-agent-runner/run/runtime-context-prompt.ts, src/agents/embedded-agent-runner/run/attempt.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P1 High-priority user-facing bug, regression, or broken workflow. labels Jun 10, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 10, 2026
@solodmd

solodmd commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

Verification evidence

1. Temporary logging code added to attempt.ts

const runtimeContextForHook = promptSubmission.runtimeOnly
  ? undefined
  : promptSubmission.runtimeContext?.trim();
// TEMP: verify exec-event runtime context fix
if (params.trigger === "heartbeat" && runtimeContextForHook) {
  console.log("[runtime-context-v] hook present, chars:", runtimeContextForHook.length);
}
const runtimeContextMessageForCurrentTurn =
  buildRuntimeContextCustomMessage(runtimeContextForHook);
// TEMP: verify custom message was created
if (params.trigger === "heartbeat") {
  console.log("[runtime-context-v] custom msg created:", !!runtimeContextMessageForCurrentTurn);
}

2. Gateway log output (after fix)

[runtime-context-v] hook present, chars: 303
[runtime-context-v] custom msg created: true

3. How to reproduce

  1. Apply the temporary logging to src/agents/embedded-agent-runner/run/attempt.ts
  2. Rebuild and start gateway
  3. Send exec sleep 5 background=true via WebChat
  4. Wait for the heartbeat wake to fire (within heartbeat interval)
  5. Check gateway log for the [runtime-context-v] lines

The 303 characters captured contain the structured exec completion event (id, exit code, stdout/stderr output).

@openclaw-barnacle openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels Jun 10, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label Jun 10, 2026
@vincentkoc vincentkoc self-assigned this Jun 10, 2026
@openclaw-barnacle openclaw-barnacle Bot added size: S and removed size: XS proof: sufficient ClawSweeper judged the real behavior proof convincing. labels Jun 10, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jun 10, 2026
@vincentkoc

Copy link
Copy Markdown
Member

Closing after maintainer autoreview because the proposed fallback is not safe at this boundary.

The patch makes the model consume the unmatched heartbeat/exec-event payload through a hidden runtime-context message while the current before_agent_run input gate still receives only [OpenClaw heartbeat poll]. Prompt-based content-policy and security plugins can therefore miss model-visible command output or runtime text.

During review we also found and fixed narrower issues around media prompt trust, explicit runtime-context composition, and hook-expanded prompt duplication, but the remaining problem requires an explicit contract for exposing all current-turn model-visible context to pre-model gates. That is broader than this PR and should be designed at the hook/input boundary rather than restored as a generic prompt fallback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling P1 High-priority user-facing bug, regression, or broken workflow. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants