Skip to content

fix(cli-runner): emit run.progress from raw stdout in Claude live ses…#84550

Closed
SkyWolfDreamer wants to merge 2 commits into
openclaw:mainfrom
SkyWolfDreamer:fix/cli-runner-live-session-raw-stdout-progress
Closed

fix(cli-runner): emit run.progress from raw stdout in Claude live ses…#84550
SkyWolfDreamer wants to merge 2 commits into
openclaw:mainfrom
SkyWolfDreamer:fix/cli-runner-live-session-raw-stdout-progress

Conversation

@SkyWolfDreamer

@SkyWolfDreamer SkyWolfDreamer commented May 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Problem: A prior fix hooked the live-session liveness signal on onAssistantDelta, which the parser only invokes for text_delta content blocks. Reasoning-only Opus turns that emit just thinking_delta lines (no user-visible text) bypass the callback, so the diagnostic watchdog still aborts the run after about six minutes even though the CLI is actively streaming reasoning output.
  • Solution: Move the live-session liveness signal upstream to handleClaudeStdout, which runs on every raw subprocess chunk before any JSONL parsing. The signal now covers every byte the CLI emits β€” thinking_delta, tool_use, content_block_start/stop, system events β€” so any progress, including reasoning-only progress, keeps the watchdog informed.
  • What changed: src/agents/cli-runner/claude-live-session.ts (+33 lines) β€” adds a per-turn diagnostic identity, a throttled emitLiveTurnDiagnosticProgress() helper, and a single emit call at the raw-stdout boundary. Adds a new seam coverage file (+190 lines) driving the live session with thinking-only JSONL.
  • What did NOT change: the watchdog thresholds, the session-recovery flow, the batch (execute.ts) onStdout path, the parser internals, and every non-claude-cli backend.

Motivation

Addresses the P1 ClawSweeper finding on the prior PR: the live-progress signal needs to cover non-text stream activity. The real reproduction is a claude-opus-4-7 turn with extended reasoning where the model emits six or more minutes of thinking_delta events before producing any user-visible text β€” exactly the case the prior fix missed.

Change Type

  • Bug fix

Scope

  • Gateway / orchestration (cli-runner / claude-cli backend)

Linked Issue/PR

Related to the prior live-progress fix that hooked the text-delta callback (link will be updated once that PR merges). Checked: this PR fixes a bug or regression.

Real behavior proof

  • Behavior addressed: the diagnostic stuck-session watchdog aborts long Opus reasoning turns because the prior live-progress hook fires only on text_delta. Reasoning-only streams emit thinking_delta lines that never reach the prior callback, so lastProgressAgeMs grows past stuckSessionAbortMs and the lane is killed even while real reasoning is streaming.
  • Real environment tested: OpenClaw running on WSL2 Ubuntu, claude-cli backend (@anthropic-ai/claude-code), claude-opus-4-7 and claude-sonnet-4-6 via the live-session mode. The patched build is deployed against the same affected install that was originally hitting the watchdog abort.
  • Before evidence: prior to this patch the diagnostic watchdog had no progress signal during reasoning-only phases β€” the prior text-delta hook never fired, so the lane recorded no lastProgress for the duration of a thinking burst and the watchdog aborted the run at the configured stuckSessionAbortMs (about six minutes) even while the CLI was actively streaming reasoning. Production runtime logs from that period showed claude live session turn failed: ... error=AbortError after roughly 360 seconds on Opus reasoning turns with no concurrent text output.
  • Exact steps or command run after this patch: built OpenClaw from the patched source on the affected install and restarted the local gateway onto the new build, then drove a normal claude-cli session under claude-opus-4-7 with the live-session mode enabled. Command sequence the gateway is running on:
node /home/<user>/Work/OpenClaw/dist/index.js gateway --port 18789
    # then exercise any claude-cli session whose Opus turns include long thinking phases
  • Evidence after fix: copied live terminal output from the patched-build gateway β€” redacted runtime logs showing the new cli:live:stdout progress events firing on every raw CLI chunk and the watchdog seeing the resulting liveness:
[diagnostic] long-running session: sessionKey=agent:main:telegram:direct:<n> state=processing age=147s queueDepth=1 lastProgress=cli:live:stdout lastProgressAge=20s recovery=none
[diagnostic] long-running session: sessionKey=agent:main:telegram:direct:<n> state=processing age=122s queueDepth=1 lastProgress=cli:live:stdout lastProgressAge=6s  recovery=none
[diagnostic] long-running session: sessionKey=agent:main:telegram:direct:<n> state=processing age=245s queueDepth=1 lastProgress=cli:live:stdout lastProgressAge=6s  recovery=none
[diagnostic] long-running session: sessionKey=agent:main:main                  state=processing age=302s queueDepth=1 lastProgress=cli:live:stdout lastProgressAge=0s  recovery=none
[agent/cli-backend] claude live session turn: provider=claude-cli model=claude-opus-4-7 durationMs=121524 rawLines=193
[agent/cli-backend] claude live session turn: provider=claude-cli model=claude-opus-4-7 durationMs=114475 rawLines=112

The first four lines are the diagnostic system observing the new cli:live:stdout reason on long-running lanes, with lastProgressAge staying low (0–20 s) while the lane runs past 147–302 s of real work. recovery=none confirms the watchdog is no longer escalating to abort on these lanes. The last two lines are normal turn completions at 2 m 1 s and 1 m 54 s β€” durations that would previously have been at risk of abort if they contained reasoning-only intervals.

  • Observed result after fix: on the patched-build gateway the diagnostic system records lastProgress=cli:live:stdout on every raw CLI chunk and lastProgressAge stays small while the lane runs. Long claude-opus-4-7 turns complete normally (durations above two minutes observed on the running gateway), no AbortError is emitted on the affected sessions for the duration of the observation window, and recovery=none for every long-running classification.
  • What was not tested: an end-to-end reasoning-only turn of greater than six minutes against the live gateway β€” the patched build has been running normally with shorter (up to ~2 m) Opus turns, and a six-minute reasoning burst was not deliberately staged for this proof window; the seam test covers the throttle and the strict-coverage assertion that the new emit path fires even when only thinking_delta lines arrive.

Root Cause

createCliJsonlStreamingParser's onAssistantDelta callback only fires for text_delta content blocks. thinking_delta, tool_use, content_block_start/stop, and system events all pass through the parser but never invoke the delta callback. The prior fix hooked at that text-only boundary, leaving the diagnostic watchdog with no progress signal during reasoning-only phases. Missing guardrail: no regression coverage forced a reasoning-only stream through the live-session path.

Regression Test Plan

src/agents/cli-runner/claude-live-session.diagnostic-progress.test.ts (new file): drives runClaudeLiveSessionTurn with a mocked supervisor that feeds JSONL thinking_delta lines through the live session's onStdout, asserts that run.progress with reason="cli:live:stdout" fires through the throttle window, and asserts that onAssistantDelta is never invoked β€” proving the stdout-level emission is strictly more inclusive than the prior text-delta hook.

User-visible / Behavior Changes

None. Internal diagnostic signal only.

Diagram

Before (prior PR):
  CLI stdout (thinking_delta line) -> handleClaudeStdout
    -> streamingParser.push -> [filtered, no onAssistantDelta]
    -> no progress signal -> watchdog aborts at ~360 s

After (this PR):
  CLI stdout (any line) -> handleClaudeStdout
    -> emitLiveTurnDiagnosticProgress("cli:live:stdout") [throttled 10 s]
    -> streamingParser.push (unchanged)
    -> watchdog sees progress -> no abort

Security Impact

  • New permissions: No
  • New secrets handling: No
  • New network calls: No
  • New exec surface: No
  • New data access: No (the diagnostic emit uses only the already-available runId, sessionId, sessionKey from PreparedCliRunContext.params and a static reason literal)

Repro + Verification

  • Environment: OpenClaw on WSL2 Linux 6.6, Node 24, gateway local, claude-opus-4-7 via claude-cli (live-session mode).
  • Steps: rebuild from the patched source, restart the local gateway, drive a normal claude-cli session whose Opus turns include reasoning phases.
  • Expected: the diagnostic system records lastProgress=cli:live:stdout on each raw CLI chunk; long-running sessions are kept alive while the CLI is actively streaming.
  • Actual (before): the watchdog aborted reasoning-only turns at about 360 s with AbortError even while the CLI was emitting thinking_delta lines.

Evidence

  • Failing-before-fix scenario locked in by the new seam coverage
  • Trace logs / terminal capture (before: production runtime logs showing the prior abort; after: patched-build gateway runtime logs above)

Human Verification

Verified: the new seam coverage passes against the patched source; the existing cli-runner suite stays clean; the patched-build gateway emits lastProgress=cli:live:stdout against real claude-cli sessions and the watchdog stays at recovery=none for long-running lanes; durations above two minutes complete normally on Opus. NOT verified: a deliberately-staged reasoning-only turn of more than six minutes against the live gateway (no such turn was deliberately driven during the observation window).

Review Conversations

  • Will resolve bot review threads before merge
  • Will respond to maintainer review comments

Compatibility / Migration

Backward compatible. No config or migration. The new emit uses only existing run/session identifiers and a static reason literal.

Risks and Mitigations

  • Risk: a subprocess that emits raw stdout while genuinely hung would still keep the diagnostic watchdog from firing.
    Mitigation: the supervisor-level noOutputTimeoutMs watchdog (separate from the diagnostic watchdog) still kills truly silent processes; the throttle prevents spam from chatty streams.
  • Risk: once the prior PR merges, the onAssistantDelta emission in execute.ts becomes redundant for the live-session path (text_delta also passes through handleClaudeStdout).
    Mitigation: redundant but harmless β€” the throttle bucket suppresses the duplicate emit. Happy to land a follow-up cleanup once both PRs are in.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling P1 High-priority user-facing bug, regression, or broken workflow. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant