Skip to content

Add a DetectLoops capability to interrupt repeated-action loops - #11471

Merged
mpfaffenberger merged 6 commits into
mainfrom
agent/clai2-9008
Oct 27, 2026
Merged

mpfaffenberger merged 6 commits into
mainfrom
agent/clai2-9008

Conversation

@mpfaffenberger

@mpfaffenberger mpfaffenberger commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Closes #9008

Why we make these changes

An autonomous agent can keep calling the same tool with the same arguments, get the same error back, alternate between two calls, or repeat the same text while output validation retries, until its request or token limit runs out. Nothing in core or harness notices. This adds DetectLoops to the harness (Control & safety), as proposed in #9008. It notices these patterns and interrupts the run before the rest of the budget is spent.

New public surface

  • pydantic_ai_harness.DetectLoops, a capability with window, repeated_call_threshold, repeated_result_threshold, alternating_calls_threshold, repeated_text_threshold (each None to disable), and action ('nudge', 'error', or a sync or async (ctx, loop) handler)
  • From pydantic_ai_harness.detect_loops: DetectedLoop, LoopDetectedError (an AgentRunError), LoopHandler, and LoopPattern

User-visible behavior

agent = Agent(model, capabilities=[DetectLoops(action='error')])
agent.run_sync('Summarize missing.txt')  # model keeps calling read_file('missing.txt')
# before: runs until UsageLimitExceeded (request_limit)
# after:  LoopDetectedError after the 3rd identical call and result; loop.pattern == 'repeated_result'

With the default action='nudge', the request after the loop carries a conversation-scoped SystemPromptPart marked metadata['origin'] == 'pydantic-ai-harness.detect_loops.nudge.v1'. Calls are recorded in after_model_request, and their results are read from the request that follows. The capability runs outermost, so compaction can't make different calls look alike. Each detection emits a loop detected span.

Verification

  • test_detect_loops.py: the four patterns, the actions, canonical arguments and results, tool call ids reused across responses, ClearToolResults/SlidingWindowCompaction composition, SkipModelRequest, request prefix stability over two runs, spans, and config validation
  • test_temporal.py, test_dbos.py, test_prefect.py: nudge and error under Temporal including history replay, DBOS crash recovery, and Prefect flows
  • 100% line and branch coverage of detect_loops/_capability.py. The examples in docs/harness/detect-loops.md now run in tests/test_examples.py
  • Not done: a live provider probe of the nudge (no credentials here). The nudge goes through core's existing SystemPromptPart rendering and adds no new framing tag

What changes for existing users

Nothing; the capability is opt-in.

PR decisions

2026-10-26 · Detect from run history in before_model_request, no store · iter -

  • Decision: Detection rebuilds call/result/text steps from this runs messages each request; the only in-memory state is how many responses a reported loop consumed
  • Why: Replay-safe under durable engines and per-run by construction, so the issues optional pluggable store is not needed (YAGNI)
  • Source: Add a DetectLoops capability to interrupt repeated-action loops #9008

2026-10-26 · Nudge is a conversation-scoped SystemPromptPart with origin metadata · iter -

2026-10-26 · Defaults: repeated_call 5/10, repeated_result 3, alternating 3 cycles, repeated_text 3; None disables · iter -

2026-10-26 · Results compare as outcome + sorted-key structured content + file identifiers · iter -

2026-10-26 · Context-window error loops, truncation, restart left out · iter -

2026-10-26 · Telemetry: zero-duration loop detected span on ctx.tracer · iter -

2026-10-26 · Live provider probe of the nudge not run · iter -

2026-10-26 · pre-push review 1 of 3 at aa68cc5 · iter -

2026-10-26 · pre-push review 1 of 3 at aa68cc5 · iter -

2026-10-26 · Nudge keeps conversation scope, not request scope · iter -

  • Decision: Review finding 1 dismissed: a loop nudge is a course correction that should keep applying, like TrajectoryJudge steering; request scope is for nudges about the current request only
  • Why: System Reminders docs on main already described loop detection as a durable nudge; conversation scope appends without moving the cached prefix (tested)
  • Source: Add a DetectLoops capability to interrupt repeated-action loops #9008

2026-10-26 · pre-push review 2 of 3 at 3812d53 · iter -

2026-10-26 · pre-push review 2 of 3 at 3812d53 · iter -

2026-10-26 · pre-push review 3 of 3 at 82b6a68 · iter -

2026-10-26 · pre-push review 3 of 3 at 82b6a68 · iter -

2026-10-27 · Rebased onto main to resolve System Reminders conflict · iter -

Checklist

  • No breaking changes in accordance with the version policy.
  • Any permitted compatibility impact has a label, warning, migration and release note, and an exact API-check waiver if the gate reports it.
  • PR title is fit for the release changelog.

@mpfaffenberger mpfaffenberger added feature New feature request, or PR implementing a feature area:durable-exec Durable execution: `durable_exec`, `@durable_operation`, replay, every engine in core or Harness. pkg:harness pydantic-ai-harness (src/pydantic_ai_harness) labels Oct 26, 2026
@github-actions github-actions Bot added the size: L Large PR (501-1500 weighted lines) label Oct 26, 2026
@macroscopeapp

macroscopeapp Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at e6693d1

Macroscope's review found this PR approvable — Adds opt-in loop detection that can steer or stop repetitive agent runs. Existing agents remain unchanged, with per-run isolation, configurable thresholds, and content-aware telemetry safeguards limiting exposure.

You can add or adjust custom eligibility rules. Learn more.

@mpfaffenberger mpfaffenberger added final-review Run the CI Review agent on this PR's head new-capability Requests and PRs adding a first-party capability or a new provider-backed implementation. labels Oct 27, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at 117abba9af6ce0d294239cab4c8ff24ddfac2775.

All 22 changed files were reviewed. The two inline comments (both LOW) are the only findings that survived verification: the action docstring misdescribes the nudge's deliberate conversation scope, and result recording can mis-pair tool returns when another capability short-circuits a model request with a canned tool-call response.

Generated by 🔎 CI Review for #11471

…n loops

An autonomous agent can keep making the same tool call, getting the same
result, alternating between two calls, or repeating the same text until it
exhausts its budget. DetectLoops checks the current run's history before each
model request and nudges the model, raises LoopDetectedError, or calls a user
handler. Deriving detection from history keeps it per-run and replay-safe
under durable execution without a separate state store.

Closes #9008
Re-deriving calls and results from history each request let compaction that
drops or rewrites messages shift the reported boundary or make different
calls look alike. Record each response's calls in after_model_request and its
results from the request that follows, gate fingerprints behind
trace_include_content, and cover DBOS, Prefect, Temporal errors, compaction,
and prefix stability.
…ite them

Run DetectLoops outermost so compaction cannot clear the newest results
before they are recorded, read each response's results only once so a
response DetectLoops never saw cannot overwrite them, and make the DBOS test
exercise recovery.
@mpfaffenberger mpfaffenberger added final-review Run the CI Review agent on this PR's head and removed final-review Run the CI Review agent on this PR's head labels Oct 27, 2026
@mpfaffenberger
mpfaffenberger added this pull request to the merge queue Oct 27, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 27, 2026
…prefix

The new FunctionModel prefix check on main flags the two tests that pair
DetectLoops with ClearToolResults and SlidingWindowCompaction; they rewrite
history on purpose to show detection survives it.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at e87faa645e743dd9681fc663e40446a03aeed0d3.

All 22 changed files were reviewed: the capability core (detect_loops/_capability.py), its tests across the main, DBOS, Prefect, and Temporal suites, the docs page and package README twins, and the skill/navigation/parity/exports updates. The detection state machine, result-pairing fix, nudge metadata/scope, sanitization, telemetry, config validation, and documented defaults all check out; the single inline comment is a non-blocking docs completeness nitpick. No blocking findings.

Generated by 🔎 CI Review for #11471

| `alternating_calls` | Two different tool calls are made in turn, A-B-A-B, for this many A-B cycles. | `alternating_calls_threshold` (`3`) |
| `repeated_text` | This many consecutive responses have no tool calls and the same text, ignoring case and whitespace. This happens when [output validation](../output.md#output-validator-functions) keeps asking the model to retry. | `repeated_text_threshold` (`3`) |

`window` (default `10`) is how many of the most recent tool calls the three call-based checks look at. Set a threshold to `None` to turn that check off. A threshold must be at least 2, and a call-based threshold has to fit in `window` (`alternating_calls_threshold` needs twice its value), so a check that could never fire raises `ValueError` when the capability is created.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The validation paragraph enumerates the constructor's rules but omits the one on window itself: __post_init__ rejects window < 2 even when every call-based threshold is None, so a text-only config such as DetectLoops(window=1, repeated_call_threshold=None, repeated_result_threshold=None, alternating_calls_threshold=None) raises ValueError with no hint from this page. Add it to the list:

@mpfaffenberger
mpfaffenberger added this pull request to the merge queue Oct 27, 2026
Merged via the queue into main with commit 44eaa96 Oct 27, 2026
99 checks passed
@mpfaffenberger
mpfaffenberger deleted the agent/clai2-9008 branch October 27, 2026 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:durable-exec Durable execution: `durable_exec`, `@durable_operation`, replay, every engine in core or Harness. feature New feature request, or PR implementing a feature final-review Run the CI Review agent on this PR's head new-capability Requests and PRs adding a first-party capability or a new provider-backed implementation. pkg:harness pydantic-ai-harness (src/pydantic_ai_harness) size: L Large PR (501-1500 weighted lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a DetectLoops capability to interrupt repeated-action loops

1 participant