Skip to content

Cron scheduler enters infinite retry loop on model rejection #11438

@jzOcb

Description

@jzOcb

Summary

When a cron job fails due to "model not allowed" error, the scheduler enters a rapid retry loop with no backoff or attempt limit, generating 150+ failed attempts in ~10 seconds.

Environment

  • OpenClaw version: 2026.2.6-3 (commit f072249)
  • OS: Ubuntu 22.04 (Linux 6.8.0-94-generic)

Steps to Reproduce

  1. Create an isolated cron job with a model that will be rejected:
{
  "name": "retry-storm-test",
  "schedule": {"kind": "at", "at": "<soon>"},
  "payload": {"kind": "agentTurn", "message": "Test", "model": "sonnet"},
  "sessionTarget": "isolated"
}
  1. Wait for scheduled time

  2. Check job runs via cron.runs

Expected Behavior

  • Job fails once with clear error
  • Either: retry with exponential backoff, or mark job as failed and stop

Actual Behavior

Massive retry storm:

{"ts": 1770491574481, "status": "error", "error": "model not allowed: anthropic/claude-sonnet-4", "durationMs": 4},
{"ts": 1770491574520, "status": "error", "error": "model not allowed: anthropic/claude-sonnet-4", "durationMs": 4},
{"ts": 1770491574558, "status": "error", "error": "model not allowed: anthropic/claude-sonnet-4", "durationMs": 4},
// ... 150+ more entries in ~10 seconds

Each retry is ~30-40ms apart, no backoff applied.

Evidence

Single job generated 150+ run entries in the cron.runs response, all within a 10-second window.

Impact

  • Pollutes run history with hundreds of duplicate errors
  • Could cause performance issues with rapid retries
  • Makes debugging harder (signal lost in noise)
  • nextRunAtMs never advances (stays at original scheduled time)

Suggested Fix

  1. Implement exponential backoff for retries
  2. Cap retry attempts (e.g., max 3-5 retries)
  3. Distinguish between retryable errors (network timeout) and permanent errors (model not allowed)
  4. For permanent errors like "model not allowed", fail immediately without retry

Related

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions