-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
Bug: genericRepeat critical/circuit-breaker never fires when exec results vary slightly #93917
Copy link
Copy link
Open
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.Crash, hang, restart loop, or process-level availability failure.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Description
Metadata
Metadata
Assignees
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:linked-pr-openClawSweeper found an open linked pull request for this issue.ClawSweeper found an open linked pull request for this issue.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.Crash, hang, restart loop, or process-level availability failure.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Type
Fields
Priority
None yet
Bug:
genericRepeatcritical/circuit-breaker never fires when exec results vary slightlySummary
The
genericRepeatdetector intool-loop-detection.tshas two detection paths:recentCount— counts all calls with identical args, regardless of result. ✅ Works correctly.noProgressStreak— requires identical result hash across consecutive calls. ❌ Fails for exec commands.noProgressStreak. ❌ Same failure mode.Root cause
getNoProgressStreak()(the function used by critical and circuit-breaker) requires the result hash to be identical across consecutive calls. Forexectool calls,hashExecToolOutcome()includes the fulloutputtext in the hash. When a command fails repeatedly (e.g., SSH timeout, docker connection refused), the error output often varies slightly — different timestamps, different error messages, different connection states — causing the result hash to differ each time.This means
noProgressStreakis always 1, never reaching the critical threshold (default 20) or circuit breaker (default 30), even after hundreds of identical failed calls.Real-world evidence
From a production OpenClaw session (
aa7aef13), 1252 totalexeccalls were made in one session. The top repeated commands:sleep 3 && docker exec alist /opt/alist/alist admin set admin123sleep 8 && ssh -q ... "curl -s http://127.0.0.1:9233/json/version"The loop detection config was set aggressively (
warningThreshold: 5,criticalThreshold: 10,globalCircuitBreakerThreshold: 15), yet none of the critical/breaker thresholds fired. The warning path did fire but the model (DeepSeek v4 Pro) ignored the text warning.This eventually led to thread pool exhaustion and a Gateway restart — the exact failure mode loop detection was designed to prevent.
Proposed fix
For
exec/bashtools where output is inherently variable, the critical path should not require identical result hashes. Options:Option A: For
exectools, userecentCount(same as warning path) for critical/breaker thresholds too, since "same command called N times" is itself the danger signal regardless of output variation.Option B: Make
hashExecToolOutcome()more lenient — hash only{status, exitCode, timedOut}without the full output text, so that "failed with exit code 1" always produces the same hash regardless of error message variation.Option C: Add a separate
execRepeatThresholdthat usesrecentCountand bypasses the result-hash requirement entirely for shell commands.Environment
2026.6.6tools.loopDetection.enabled: true,warningThreshold: 5,criticalThreshold: 10,globalCircuitBreakerThreshold: 15xopdeepseekv4provia headroom provider