Skip to content

fix: abort generic no-progress tool loops#80668

Merged
frankekn merged 2 commits into
mainfrom
codex/generic-tool-loop-critical
May 11, 2026
Merged

fix: abort generic no-progress tool loops#80668
frankekn merged 2 commits into
mainfrom
codex/generic-tool-loop-critical

Conversation

@frankekn

Copy link
Copy Markdown
Contributor

Summary

  • Problem: generic repeated tool calls only warned even when the same calls kept returning identical no-progress outcomes.
  • Why it matters: agents could keep spending model/tool cycles after a loop warning instead of stopping at the configured critical threshold.
  • What changed: generic repeat detection now escalates identical tool+args+outcome loops to a critical block at criticalThreshold; docs and tests describe the no-progress requirement.
  • What did NOT change (scope boundary): repeated calls with changing outcomes still warn instead of hard-blocking; the global circuit breaker remains a fallback.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

  • Closes #
  • Related #
  • This PR fixes a bug or regression

Real behavior proof (required for external PRs)

  • Behavior or issue addressed: generic same-tool/same-args loops now stop when repeated calls also return identical outcomes.
  • Real environment tested: local OpenClaw source checkout on macOS with Node/pnpm.
  • Exact steps or command run after this patch: pnpm test src/agents/pi-tools.before-tool-call.e2e.test.ts
  • Evidence after fix (screenshot, recording, terminal capture, console output, redacted runtime log, linked artifact, or copied live output): test output reported 42 passing tests, including the generic repeat escalation coverage.
  • Observed result after fix: generic repeat diagnostics escalate from warning at 10 calls to critical block at 20 identical no-progress outcomes.
  • What was not tested: live provider behavior against a real model.
  • Before evidence (optional but encouraged): prior test coverage expected generic repeats to remain warn-only until the global circuit breaker.

Root Cause (if applicable)

  • Root cause: the generic repeat detector only emitted warnings for repeated tool+args patterns and did not apply criticalThreshold to repeated no-progress outcomes.
  • Missing detection / guardrail: generic loops had no critical no-progress stop before the global circuit breaker.
  • Contributing context (if known): specialized poll and ping-pong detectors already had critical paths, but generic repeats did not.

Regression Test Plan (if applicable)

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file: src/agents/tool-loop-detection.test.ts, src/agents/pi-tools.before-tool-call.e2e.test.ts
  • Scenario the test should lock in: same tool, same args, and identical outcomes escalate from warning to critical at criticalThreshold; changing outcomes remain warnings.
  • Why this is the smallest reliable guardrail: it exercises the detector directly and through the before-tool-call wrapper without invoking a live model.
  • Existing test that already covers this (if any): N/A

User-visible / Behavior Changes

Agents with tools.loopDetection.enabled now abort generic repeated no-progress tool loops at criticalThreshold instead of waiting for the global circuit breaker.

Diagram (if applicable)

Before:
same tool + same args + same outcome -> warning -> warning until global circuit breaker

After:
same tool + same args -> warning
same tool + same args + same outcome -> critical block at threshold

Security Impact (required)

  • New permissions/capabilities? (Yes/No) No
  • Secrets/tokens handling changed? (Yes/No) No
  • New/changed network calls? (Yes/No) No
  • Command/tool execution surface changed? (Yes/No) Yes
  • Data access scope changed? (Yes/No) No
  • If any Yes, explain risk + mitigation: the tool loop guard can now block repeated generic no-progress tool calls sooner. It only blocks after identical outcomes, while changing outcomes remain allowed.

Repro + Verification

Environment

  • OS: macOS
  • Runtime/container: local Node/pnpm source checkout
  • Model/provider: N/A
  • Integration/channel (if any): N/A
  • Relevant config (redacted): tools.loopDetection.enabled: true

Steps

  1. Enable loop detection.
  2. Repeat a generic tool call with identical arguments and identical outcomes until criticalThreshold.
  3. Repeat a generic tool call with identical arguments but changing outcomes.

Expected

  • Identical no-progress outcomes are blocked at criticalThreshold.
  • Changing outcomes are not hard-blocked by the generic detector.

Actual

  • Matches expected.

Evidence

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios: generic repeat warning, generic repeat critical block, run-id isolation, exec outcome normalization, changed-scope gate.
  • Edge cases checked: changing exec output remains warning-only; global circuit breaker still works when generic repeat is disabled.
  • What you did not verify: live model/provider run.

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

  • Backward compatible? (Yes/No) Yes
  • Config/env changes? (Yes/No) No
  • Migration needed? (Yes/No) No
  • If yes, exact upgrade steps: N/A

Risks and Mitigations

  • Risk: legitimate repeated generic calls with identical outcomes may stop earlier than before.
    • Mitigation: the guard requires identical outcomes, preserves warning-only behavior for changing results, and remains controlled by tools.loopDetection.enabled and criticalThreshold.

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation agents Agent runtime and tooling size: S maintainer Maintainer-authored PR labels May 11, 2026
@clawsweeper

clawsweeper Bot commented May 11, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs changes before merge.

Summary
The PR makes generic repeated same-tool/same-args calls block at the loop detector critical threshold only when prior outcomes are identical, and updates focused docs, tests, and changelog text.

Reproducibility: yes. from source inspection: current main records identical outcomes but generic repeats return warning-level until the global no-progress breaker, and current tests assert that behavior. I did not execute the tests because this review is read-only.

Real behavior proof
Not applicable: The external-contributor proof gate does not apply because this is a MEMBER/maintainer-labeled PR; the PR body’s local test output is supplemental review evidence.

Next step before merge
A narrow automated follow-up could align the remaining config docs/help text; no runtime repair is indicated from this review.

Security
Cleared: The diff changes an existing loop guard, docs, tests, and changelog only; it does not add dependencies, secrets handling, permissions, network calls, or new code execution paths.

Review findings

  • [P3] Align the remaining loop config help — docs/tools/loop-detection.md:83
Review details

Best possible solution:

Land the focused detector change once the remaining config/help text is aligned and the normal maintainer PR gates finish green.

Do we have a high-confidence way to reproduce the issue?

Yes, from source inspection: current main records identical outcomes but generic repeats return warning-level until the global no-progress breaker, and current tests assert that behavior. I did not execute the tests because this review is read-only.

Is this the best way to solve the issue?

Mostly yes: putting the critical branch in detectToolCallLoop is the narrow owner-boundary fix for repeated generic no-progress calls. The safer finishing step is to align the remaining config reference/help text with the changed detector semantics.

Full review comments:

  • [P3] Align the remaining loop config help — docs/tools/loop-detection.md:83
    This PR updates docs/tools/loop-detection.md for genericRepeat blocking, but the config reference still says detectors.genericRepeat only warns on same-tool/same-args calls. After this behavior change, users would get conflicting config semantics between the tool-loop page and the gateway config reference, so please update the remaining config contract/help text with the new no-progress block behavior.
    Confidence: 0.86

Overall correctness: patch is correct
Overall confidence: 0.87

Acceptance criteria:

  • pnpm exec oxfmt --check --threads=1 docs/tools/loop-detection.md docs/gateway/config-tools.md src/config/schema.help.ts src/config/types.tools.ts
  • pnpm test src/agents/tool-loop-detection.test.ts src/agents/pi-tools.before-tool-call.e2e.test.ts

What I checked:

  • PR patch behavior: The PR adds a generic_repeat critical branch when non-poll calls have identical arguments and identical outcomes at criticalThreshold, while leaving the warning path for same-tool/same-args repeats below that point. (src/agents/tool-loop-detection.ts:617, 49af9d7e4a27)
  • Current main source behavior: Current main only has the global no-progress breaker before the generic repeat warning, and the generic repeat detector is explicitly warn-only in source. (src/agents/tool-loop-detection.ts:528, 166f2bcee476)
  • Current main test contract: Current main test coverage expects generic no-progress repeats at CRITICAL_THRESHOLD to remain warning-level, showing the behavioral gap the PR is changing. (src/agents/tool-loop-detection.test.ts:391, 166f2bcee476)
  • Config documentation gap: The gateway config reference still describes detectors.genericRepeat as warning-only, which will conflict with the PR's new block-at-critical behavior unless updated. Public docs: docs/gateway/config-tools.md. (docs/gateway/config-tools.md:158, 166f2bcee476)
  • PR status and discussion: The PR is open, authored by a MEMBER, carries the protected maintainer label, has no review comments, and includes a prior Codex review comment reporting no major issues. (49af9d7e4a27)
  • History provenance: GitHub commit history for the central files shows @steipete introduced configurable tool loop detection and made several recent loop/outcome fixes; @vincentkoc recently rewrote loop-detection docs; @efpiva contributed adjacent post-compaction loop guard work. (src/agents/tool-loop-detection.ts:498, 076df941a326)

Likely related people:

  • steipete: Introduced configurable tool loop detection and made multiple recent fixes around loop outcome hashing, run scoping, post-compaction observation, and before-tool-call behavior. (role: feature owner and recent area contributor; confidence: high; commits: 076df941a326, e9bce3f81c3a, 6a55a00da461; files: src/agents/tool-loop-detection.ts, src/agents/pi-tools.before-tool-call.ts, src/config/schema.help.ts)
  • vincentkoc: Recently rewrote the loop-detection docs and owns recent config-tools documentation edits that should stay aligned with the changed semantics. (role: recent docs and adjacent area contributor; confidence: medium; commits: 180e295dc604, ff7f0df8712d, 8bff73cfb07f; files: docs/tools/loop-detection.md, docs/gateway/config-tools.md, src/agents/tool-loop-detection.ts)
  • efpiva: Contributed nearby post-compaction loop guard wiring and config/observation fixes that use the same tool outcome history and loop-detection contract. (role: adjacent loop-guard contributor; confidence: medium; commits: 2a702f927fb5, 4c4825679b69; files: src/agents/tool-loop-detection.ts, src/agents/pi-embedded-runner/run/attempt.ts, src/config/zod-schema.agent-runtime.ts)

Remaining risk / open question:

  • I did not run local tests because this review was required to keep the checkout read-only, and several CI checks for the PR head were still queued at inspection time.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 80d1df6e6830.

@frankekn
frankekn marked this pull request as ready for review May 11, 2026 13:57
@frankekn

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown
Contributor

Codex Review: Didn't find any major issues. Can't wait for the next one!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@frankekn
frankekn force-pushed the codex/generic-tool-loop-critical branch from 49af9d7 to 7fa287c Compare May 11, 2026 16:28
@frankekn
frankekn merged commit 678b251 into main May 11, 2026
22 checks passed
@frankekn
frankekn deleted the codex/generic-tool-loop-critical branch May 11, 2026 16:29
@frankekn frankekn self-assigned this May 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling docs Improvements or additions to documentation maintainer Maintainer-authored PR size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant