fix #97115: classify isolated cron fallback results by jincheng-xydt · Pull Request #99913 · openclaw/openclaw

jincheng-xydt · 2026-07-04T09:45:56Z

What Problem This Solves

Isolated cron runs already use runWithModelFallback, but they did not pass the embedded runner's result fallback classifier or exhausted-result merge helper. That meant zero-token/empty terminal "success" results could be treated as successful cron turns instead of moving to the configured fallback chain, and exhausted fallback summaries could lose the richer embedded result.

Why This Change Was Made

The embedded agent path already has the canonical fallback classifier and exhausted-result merge behavior. This change wires those same helpers through the isolated cron runtime boundary into createCronPromptExecutor so cron fallback behavior matches normal embedded agent fallback behavior without adding a second classifier.

User Impact

Cron agent runs that receive empty/reasoning-only/planning-only terminal results can now continue to configured fallback models instead of silently completing with no useful message. When every fallback is exhausted, cron keeps the embedded runner's richer terminal summary instead of replacing it with a weaker generic result.

Evidence

node scripts/run-vitest.mjs src/cron/isolated-agent/run.payload-fallbacks.test.ts --reporter=verbose — 7 passed
node scripts/run-vitest.mjs src/cron/isolated-agent/run*.test.ts --reporter=verbose — 21 files / 198 tests passed
node scripts/run-vitest.mjs src/agents/outcome-fallback-runtime-contract.test.ts --reporter=verbose — 10 passed
node scripts/run-vitest.mjs src/agents/model-fallback.test.ts --reporter=verbose — 90 passed
node scripts/run-oxlint.mjs src/cron/isolated-agent/run-execution.runtime.ts src/cron/isolated-agent/run-executor.ts src/cron/isolated-agent/run.test-harness.ts src/cron/isolated-agent/run.payload-fallbacks.test.ts
git diff --check

Real Behavior Proof

Loopback cron fallback proof was run locally with the PR applied. It uses real runWithModelFallback, classifyEmbeddedAgentRunResultForModelFallback, and mergeEmbeddedAgentRunResultForModelFallbackExhaustion, with a loopback cron-shaped run function. No external provider, API key, account id, phone number, endpoint, or private path is used. Synthetic run/session identifiers are redacted.

Command:

node_modules/.bin/tsx /tmp/openclaw-97115-loopback-proof.mjs

Observed output:

OpenClaw #97115 loopback cron fallback proof
No external provider, API key, account id, phone number, endpoint, or private path is used.
Private runtime identifiers are redacted in the synthetic run/session ids.
This exercises the same classifier and exhaustion merge helpers wired into isolated cron in PR #99913.

[baseline without cron result hooks]
outcome=completed
winner=openai/gpt-5-loopback-primary
visibleText=""
runCalls=[{"provider":"openai","model":"gpt-5-loopback-primary","isFinalFallbackAttempt":false}]
classifications=[]
fallbackAttempts=[]

[model-fallback/decision] model fallback decision: decision=candidate_failed requested=openai/gpt-5-loopback-primary candidate=openai/gpt-5-loopback-primary reason=format next=anthropic/claude-sonnet-loopback-fallback detail=openai/gpt-5-loopback-primary ended without a visible assistant reply
[model-fallback/decision] model fallback decision: decision=candidate_succeeded requested=openai/gpt-5-loopback-primary candidate=anthropic/claude-sonnet-loopback-fallback reason=unknown next=none

[after PR hook wiring]
outcome=completed
winner=anthropic/claude-sonnet-loopback-fallback
visibleText="fallback winner delivered visible cron response"
runCalls=[{"provider":"openai","model":"gpt-5-loopback-primary","isFinalFallbackAttempt":false},{"provider":"anthropic","model":"claude-sonnet-loopback-fallback","isFinalFallbackAttempt":true}]
classifications=[{"provider":"openai","model":"gpt-5-loopback-primary","code":"empty_result","reason":"format"},{"provider":"anthropic","model":"claude-sonnet-loopback-fallback","code":null,"reason":null}]
fallbackAttempts=[{"provider":"openai","model":"gpt-5-loopback-primary","reason":"format","code":"empty_result"}]

clawsweeper · 2026-07-04T09:48:29Z

Codex review: needs maintainer review before merge. Reviewed July 4, 2026, 6:13 AM ET / 10:13 UTC.

Summary
The PR re-exports the embedded result fallback classifier and exhaustion merge helper through the isolated cron runtime, passes them into the cron runWithModelFallback call, and adds focused cron fallback wiring coverage.

PR surface: Source +25, Tests +64. Total +89 across 4 files.

Reproducibility: yes. Source inspection shows current main omits classifyResult and mergeExhaustedResult in the isolated cron fallback call, while the fallback helper only advances returned result-level failures when a classifier is supplied; the PR body adds before/after terminal proof.

Review metrics: 1 noteworthy metric.

Cron fallback decision surface: 1 call changed. One isolated cron runWithModelFallback call now receives result classification and exhaustion merging, which changes scheduled provider/model attempts.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #96525
Summary: This PR is a candidate fix for the canonical isolated cron embedded result-level fallback classifier gap; the broader linked issue also includes a separate timeout-abort claim.

Members:

canonical: [Bug]: isolated cron model-fallback chain never engages on result-level failures (reasoning-only / empty / incomplete_turn), silently dropping the answer #96525 - It describes the same isolated cron embedded result-level fallback gap and remains the narrow canonical bug report.
partial_overlap: Bug: Cron fallback chain broken — models return zero-token success or killed by shared AbortController #97115 - It includes the returned-result and zero-token failure mode this PR targets, but also reports distinct shared AbortController timeout and cache-poisoning paths.
same_root_cause: fix(cron): retry fallback on empty embedded cron results #99911 - It is another open clean candidate PR for the same cron classifier wiring gap.
same_root_cause: fix(cron): engage model fallback on embedded result-level failures (reasoning-only / empty / incomplete_turn) #96529 - It implements the same narrow classifier wiring idea and has positive proof, but it is currently conflicting rather than a clean superseding landing path.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

none.

Risk before merge

[P2] Merging changes scheduled embedded cron provider/model routing: fallback-safe empty, reasoning-only, or incomplete primary results can now invoke configured fallback providers that current main would not call, with auth-profile and cost implications.
[P2] The broader linked issue also reports shared AbortController timeout and cache-poisoning behavior; this PR targets the returned-result classifier gap and does not prove or fix that separate timeout path.
[P1] There are overlapping open candidate PRs for the same root cause, so maintainers should pick one canonical landing branch before retiring duplicates.

Maintainer options:

Accept Embedded Cron Fallback Routing (recommended)
Maintainers can accept that scheduled embedded cron now honors configured fallback providers for result-level failures, including the resulting auth-profile and cost behavior.
Land One Canonical Branch
Pick this PR or another equivalent clean candidate as the single landing path, then close or retire duplicate branches after merge.
Pause For Timeout Scope
If maintainers want the shared AbortController timeout claim handled in the same change, pause this narrow PR and require broader design and proof first.

Next step before merge

No automated repair is indicated; maintainer action is to choose the canonical landing PR and accept the provider-routing behavior before merge.

Security
Cleared: The diff only changes TypeScript cron fallback wiring and focused tests, with no dependency, workflow, secret, install, package, or permission changes.

Review details

Best possible solution:

Land one narrow classifier-hook PR with current real behavior proof, then close the returned-result fallback reports while tracking timeout-abort behavior separately.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main omits classifyResult and mergeExhaustedResult in the isolated cron fallback call, while the fallback helper only advances returned result-level failures when a classifier is supplied; the PR body adds before/after terminal proof.

Is this the best way to solve the issue?

Yes for the narrow returned-result gap. Reusing the existing embedded classifier and exhaustion merge helper matches sibling agent paths, while the separate timeout-abort claim should stay out of this patch unless maintainers explicitly broaden scope.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 79d2e568a0aa.

Label changes

Label justifications:

P1: The PR targets a recurring scheduled-agent workflow that can drop the useful answer instead of recovering through configured fallback models.
merge-risk: 🚨 auth-provider: The diff changes provider/model fallback routing for embedded cron result-level failures and can invoke configured fallback providers current main would not call.
rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): Sufficient terminal proof: the PR body includes redacted loopback before/after output showing the empty primary path and after-fix fallback winner through the real fallback helpers.
proof: sufficient: Contributor real behavior proof is sufficient. Sufficient terminal proof: the PR body includes redacted loopback before/after output showing the empty primary path and after-fix fallback winner through the real fallback helpers.

Evidence reviewed

PR surface:

Source +25, Tests +64. Total +89 across 4 files.

View PR surface stats

Area	Files	Added	Net
Source	3	25	+25
Tests	1	64	+64
Docs	0	0	0
Config	0	0	0
Generated	0	0	0
Other	0	0	0
Total	4	89	+89

What I checked:

Repository policy read: Root AGENTS.md and scoped agent guidance were read and applied; fallback/provider routing is compatibility-sensitive review context, and agent test guidance supports narrow dependency-injected coverage for expensive runtime paths. (AGENTS.md:34, 79d2e568a0aa)
Current main cron gap: Current main calls runWithModelFallback from isolated cron with provider/model, abort signal, preparation, fallback overrides, and a run callback, but no classifyResult or mergeExhaustedResult. (src/cron/isolated-agent/run-executor.ts:319, 79d2e568a0aa)
Fallback helper contract: runWithModelFallback only converts an ok returned result into a failed fallback candidate when a supplied classifier returns a classification; exhausted-result preservation is gated on mergeExhaustedResult. (src/agents/model-fallback.ts:403, 79d2e568a0aa)
Existing classifier covers the target result class: The embedded classifier handles fallback-safe incomplete turns, harness empty/reasoning-only/planning-only classifications, provider error payloads, and protects delivered output and policy blocks from retry. (src/agents/embedded-agent-runner/result-fallback-classifier.ts:175, 79d2e568a0aa)
Sibling fallback surfaces: Interactive agent command and auto-reply paths already pass result classification and exhausted-result merge helpers into the same fallback helper, supporting reuse rather than a cron-specific classifier. (src/agents/agent-command.ts:1899, 79d2e568a0aa)
Documented cron fallback contract: Cron docs state that configured fallback chains still apply when a cron job primary model fails, and payload fallbacks replace the configured chain for that job. Public docs: docs/automation/cron-jobs.md. (docs/automation/cron-jobs.md:182, 79d2e568a0aa)

Likely related people:

steipete: GitHub commit history shows foundational model fallback/classifier work and recent cron execution structure changes on the central files. (role: fallback and classifier feature owner by history; confidence: high; commits: 0314819f918a, be5906e19dd7; files: src/agents/model-fallback.ts, src/agents/embedded-agent-runner/result-fallback-classifier.ts, src/cron/isolated-agent/run-executor.ts)
vincentkoc: Recent history shows work on model fallback/error normalization and cron/runtime-adjacent fallback state around this execution boundary. (role: recent area contributor; confidence: high; commits: 80805ad7a583, aa3797c8d0d7, 14e448e0e13d; files: src/agents/model-fallback.ts, src/cron/isolated-agent/run-executor.ts, src/agents/agent-command.ts)
shengting: A recent merged change adjusted provider-side AbortError fallback behavior and forwarded cron abort signals through the same fallback call this PR modifies. (role: adjacent fallback contributor; confidence: medium; commits: 98ed83f848de; files: src/agents/model-fallback.ts, src/cron/isolated-agent/run-executor.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Review history (2 earlier review cycles)

reviewed 2026-07-04T09:51:47.356Z sha d6c1808 :: needs real behavior proof before merge. :: none
reviewed 2026-07-04T10:04:10.618Z sha d6c1808 :: needs maintainer review before merge. :: none

jincheng-xydt · 2026-07-04T09:59:53Z

@clawsweeper re-review

clawsweeper · 2026-07-04T09:59:56Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

jincheng-xydt · 2026-07-04T12:11:31Z

Real behavior proof for #99913:

I ran a local loopback cron fallback proof with this PR applied. This uses the real runWithModelFallback, classifyEmbeddedAgentRunResultForModelFallback, and mergeEmbeddedAgentRunResultForModelFallbackExhaustion runtime helpers, with a loopback cron-shaped run function.

No external provider, API key, account id, phone number, endpoint, or private path is used. Synthetic run/session identifiers are redacted.

OpenClaw #97115 loopback cron fallback proof
No external provider, API key, account id, phone number, endpoint, or private path is used.
Private runtime identifiers are redacted in the synthetic run/session ids.
This exercises the same classifier and exhaustion merge helpers wired into isolated cron in PR #99913.

[baseline without cron result hooks]
outcome=completed
winner=openai/gpt-5-loopback-primary
visibleText=""
runCalls=[{"provider":"openai","model":"gpt-5-loopback-primary","isFinalFallbackAttempt":false}]
classifications=[]
fallbackAttempts=[]

[model-fallback/decision] model fallback decision: decision=candidate_failed requested=openai/gpt-5-loopback-primary candidate=openai/gpt-5-loopback-primary reason=format next=anthropic/claude-sonnet-loopback-fallback detail=openai/gpt-5-loopback-primary ended without a visible assistant reply
[model-fallback/decision] model fallback decision: decision=candidate_succeeded requested=openai/gpt-5-loopback-primary candidate=anthropic/claude-sonnet-loopback-fallback reason=unknown next=none

[after PR hook wiring]
outcome=completed
winner=anthropic/claude-sonnet-loopback-fallback
visibleText="fallback winner delivered visible cron response"
runCalls=[{"provider":"openai","model":"gpt-5-loopback-primary","isFinalFallbackAttempt":false},{"provider":"anthropic","model":"claude-sonnet-loopback-fallback","isFinalFallbackAttempt":true}]
classifications=[{"provider":"openai","model":"gpt-5-loopback-primary","code":"empty_result","reason":"format"},{"provider":"anthropic","model":"claude-sonnet-loopback-fallback","code":null,"reason":null}]
fallbackAttempts=[{"provider":"openai","model":"gpt-5-loopback-primary","reason":"format","code":"empty_result"}]

Conclusion: before the cron result hooks, an empty primary result completed successfully with no visible text. With this PR's hook wiring, the same empty primary result is classified as empty_result, the fallback chain advances, and the fallback winner returns visible output.

Source: #99913 Co-authored-by: jincheng-xydt <[email protected]>

* fix(agents): harden LSP process failures Source: #100450 Co-authored-by: morluto <[email protected]> * fix(sandbox): report effective workspace layout Sources: #100435, #100439 Co-authored-by: Aniruddha Adak <[email protected]> Co-authored-by: ZengWen-DT <[email protected]> * fix(security): fail install checks on stream errors Source: #100413 Co-authored-by: 陈宪彪0668000387 <[email protected]> * fix(android): normalize all-day calendar events Source: #100032 Co-authored-by: NianJiuZst <[email protected]> * fix(ios): serialize push-to-talk lifecycle Source: #99942 Co-authored-by: NianJiuZst <[email protected]> * fix(talk): reject inherited provider names Source: #99849 Co-authored-by: zenglingbiao <[email protected]> * fix(android): stop voice capture in background Source: #99840 Co-authored-by: xialonglee <[email protected]> * fix(cron): preserve fallback result classification Source: #99913 Co-authored-by: jincheng-xydt <[email protected]> * fix(google): bound Vertex response decompression Source: #99812 Co-authored-by: 黄剑雄0668001315 <[email protected]> * fix(plugins): report malformed discovery JSON Source: #99892 Co-authored-by: 陈宪彪0668000387 <[email protected]> * test(sandbox): configure non-default workspace fixture * test: fix small-fix batch validation --------- Co-authored-by: morluto <[email protected]> Co-authored-by: ZengWen-DT <[email protected]> Co-authored-by: 陈宪彪0668000387 <[email protected]> Co-authored-by: NianJiuZst <[email protected]> Co-authored-by: zenglingbiao <[email protected]> Co-authored-by: xialonglee <[email protected]> Co-authored-by: jincheng-xydt <[email protected]> Co-authored-by: 黄剑雄0668001315 <[email protected]>

steipete · 2026-07-05T23:10:40Z

Thanks @jincheng-xydt — your isolated cron fallback diagnosis was incorporated into #100483, landed as aaf5ab9. The landed version injects the canonical result classifier and exhaustion merge into the isolated fallback runtime, with wiring and empty-success regression coverage. Contributor credit is preserved. Closing this superseded PR.

* fix(agents): harden LSP process failures Source: openclaw#100450 Co-authored-by: morluto <[email protected]> * fix(sandbox): report effective workspace layout Sources: openclaw#100435, openclaw#100439 Co-authored-by: Aniruddha Adak <[email protected]> Co-authored-by: ZengWen-DT <[email protected]> * fix(security): fail install checks on stream errors Source: openclaw#100413 Co-authored-by: 陈宪彪0668000387 <[email protected]> * fix(android): normalize all-day calendar events Source: openclaw#100032 Co-authored-by: NianJiuZst <[email protected]> * fix(ios): serialize push-to-talk lifecycle Source: openclaw#99942 Co-authored-by: NianJiuZst <[email protected]> * fix(talk): reject inherited provider names Source: openclaw#99849 Co-authored-by: zenglingbiao <[email protected]> * fix(android): stop voice capture in background Source: openclaw#99840 Co-authored-by: xialonglee <[email protected]> * fix(cron): preserve fallback result classification Source: openclaw#99913 Co-authored-by: jincheng-xydt <[email protected]> * fix(google): bound Vertex response decompression Source: openclaw#99812 Co-authored-by: 黄剑雄0668001315 <[email protected]> * fix(plugins): report malformed discovery JSON Source: openclaw#99892 Co-authored-by: 陈宪彪0668000387 <[email protected]> * test(sandbox): configure non-default workspace fixture * test: fix small-fix batch validation --------- Co-authored-by: morluto <[email protected]> Co-authored-by: ZengWen-DT <[email protected]> Co-authored-by: 陈宪彪0668000387 <[email protected]> Co-authored-by: NianJiuZst <[email protected]> Co-authored-by: zenglingbiao <[email protected]> Co-authored-by: xialonglee <[email protected]> Co-authored-by: jincheng-xydt <[email protected]> Co-authored-by: 黄剑雄0668001315 <[email protected]>

fix(cron): classify isolated fallback results

d6c1808

openclaw-barnacle Bot added the size: S label Jul 4, 2026

clawsweeper Bot mentioned this pull request Jul 4, 2026

fix(cron): retry fallback on empty embedded cron results #99911

Open

clawsweeper Bot mentioned this pull request Jul 5, 2026

model.fallbacks chain does not engage on reasoning-only / empty-visible-reply failures #85422

Closed

steipete self-assigned this Jul 5, 2026

steipete mentioned this pull request Jul 5, 2026

fix: land ten small reliability fixes #100483

Merged

steipete added a commit that referenced this pull request Jul 5, 2026

fix(cron): preserve fallback result classification

66e4bf3

Source: #99913 Co-authored-by: jincheng-xydt <[email protected]>

steipete closed this Jul 5, 2026

clawsweeper Bot mentioned this pull request Jul 6, 2026

fix(cron): engage model fallback on embedded result-level failures (reasoning-only / empty / incomplete_turn) #96529

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix #97115: classify isolated cron fallback results#99913

fix #97115: classify isolated cron fallback results#99913
jincheng-xydt wants to merge 1 commit into
openclaw:mainfrom
jincheng-xydt:fix/97115-cron-fallback-classifier

jincheng-xydt commented Jul 4, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 4, 2026 •

edited

Loading

Uh oh!

jincheng-xydt commented Jul 4, 2026

Uh oh!

clawsweeper Bot commented Jul 4, 2026 •

edited

Loading

Uh oh!

jincheng-xydt commented Jul 4, 2026

Uh oh!

steipete commented Jul 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

jincheng-xydt commented Jul 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Real Behavior Proof

Uh oh!

clawsweeper Bot commented Jul 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jincheng-xydt commented Jul 4, 2026

Uh oh!

clawsweeper Bot commented Jul 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jincheng-xydt commented Jul 4, 2026

Uh oh!

steipete commented Jul 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jincheng-xydt commented Jul 4, 2026 •

edited

Loading

clawsweeper Bot commented Jul 4, 2026 •

edited

Loading

clawsweeper Bot commented Jul 4, 2026 •

edited

Loading