fix(agents): block message-tool spam loops defeated by volatile message ids#89109
Conversation
|
Codex review: needs maintainer review before merge. Reviewed June 5, 2026, 2:13 AM ET / 06:13 UTC. Summary PR surface: Source +99, Tests +339. Total +438 across 2 files. Reproducibility: yes. Current main hashes volatile send IDs in the resultHash while the critical blockers require identical resultHash values, and the PR adds deterministic regression cases for that exact source-level failure path. Review metrics: none identified. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Risk before merge
Maintainer options:
Next step before merge
Security Review detailsBest possible solution: Land exactly one canonical volatile-send hashing fix for the shared loop detector, ideally with Gateway/qa-channel or equivalent runtime proof, then close the linked bug and close or supersede the duplicate PR. Do we have a high-confidence way to reproduce the issue? Yes. Current main hashes volatile send IDs in the resultHash while the critical blockers require identical resultHash values, and the PR adds deterministic regression cases for that exact source-level failure path. Is this the best way to solve the issue? Yes, fixing the shared outcome hash layer is a maintainable solution for this bug, but this branch is not clearly the sole best landing path while the sibling PR with broader runtime proof remains open. AGENTS.md: found and applied where relevant. Codex review notes: model gpt-5.5, reasoning high; reviewed against d7759c6a35ac. Label changesLabel justifications:
Evidence reviewedPR surface: Source +99, Tests +339. Total +438 across 2 files. View PR surface stats
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
7b0ec30 to
f4a29c9
Compare
ede8650 to
59b2c14
Compare
59b2c14 to
93816f1
Compare
8913738 to
fdd7479
Compare
fdd7479 to
428e585
Compare
428e585 to
6dd37c8
Compare
d2399da to
99b9c0c
Compare
7456425 to
1f4246e
Compare
loopDetection only warned on repeated message.send calls and could never escalate to a block: each send returns a fresh messageId, so the full result hash differed every call and the no-progress / critical paths (global circuit breaker, ping_pong critical, generic no-progress) never fired. A looping model could spam identical messages until runRetries.max. Hash message-tool outcomes from stable delivery facts only, dropping volatile id keys, mirroring the existing exec / process stable-hash cases. The args hash already separates genuinely distinct sends, so this cannot mask a different message. Fixes openclaw#89090
1f4246e to
7b65fee
Compare
|
Land-ready after maintainer fixup. What changed after review:
Local proof:
GitHub proof on
Known proof gaps: no additional live Feishu/Lark/Telegram send was run in this session; the PR has hash-layer coverage, wrapped message-tool runtime coverage, and the repository real-behavior proof check is green. |
Summary
loopDetectiondetects repeatedmessage.sendcalls (ping_pong, warning level) but can never escalate to a block. A model stuck in a tool-call loop keeps sending duplicate messages — 30+ duplicates reported — untilrunRetries.max(default 160) is exhausted; the user only sees a flood of identical messages and the warning the model ignores.tool-loop-detection.ts,hashToolOutcomehashes the full result for the send tools. A successful send returns throughjsonResult(payload), so the per-call delivery ids land in both the resultdetails(e.g.messageId,receipt, the gatewayrunId) and the resulttext(the stringified payload). The hash therefore differs on every call. All three critical-block paths — the global circuit breaker,ping_pongcritical, and generic no-progress — require consecutive identicalresultHashvalues (getNoProgressStreak/noProgressEvidence), which is structurally impossible for a tool whose result carries a unique id. Detection fires only at warning level because it also keys onargsHash, which is stable.sessions_sendand the outbound-message actions (send,broadcast,reply,thread-reply,sendWithEffect,sendAttachment,upload-file,sticker,poll) of the multi-actionmessagetool.isVolatileSendResultgates the path:sessions_sendand the outbound-message actions above on the multi-actionmessagetool get the stripped hash, whilemessage's mutate/query actions (read,search,edit,react,unsend,list-pins,thread-list, …) keep full-result hashing because their ids/timestamps are real progress. Provider-docked send tools (channel plugins registered as named tools, e.g.telegram/discord) return the same volatile-id shape and are matched via the sharedisMessagingToolSendActiondetector, so loops on them escalate too;SEND_LIKE_MESSAGE_ACTIONSstays intentionally broader than that helper's terminal-send set (broadcast/reply/sticker/pollare volatile but not terminal sends).stripVolatileSendIdsrecursively removesVOLATILE_SEND_RESULT_KEYS(messageId,message_id,messageIds,platformMessageId,platformMessageIds,fileId,fileKey,pollId,receipt,runId,idempotencyKey,ts,timestamp,sentAt,deliveredAt,createdAt) fromdetailsat any depth — so it also neutralizes the nested ids in abroadcastpayload (results[].result.messageId).id(detected viaisMessageDeliveryObject: a stringid+textplusconversation/direction/senderId/accountId) so plugin-shaped delivery results normalize too, while a stable nested route/conversation.idis preserved.textis not hashed for delivery results (it is the payload restated and carries the same volatile ids); the stable delivery facts (ok/ status / channel / target) remain.buildBlockedToolResultwithdeniedReason: "tool-loop") gets no result hash, so a critical block stays sticky instead of the block resetting the streak. Other blocked results (plugin/approval vetoes) keep their hash, so repeated identical denials still escalate normally.hashToolOutcomelayer, so it covers themessagetool,sessions_send, and provider-docked channel tools uniformly (direct, gateway, and plugin-handled send results). Repeated identical-args sends now produce a stableresultHash, so the no-progress / critical paths fire and the loop is blocked.argsHashandresultHashto repeat. The args hash already carries the message content and target, so two genuinely different messages have differentargsHashand are never flagged — dropping the volatile ids only restores detection for true same-args resends.src/agents/tool-loop-detection.ts— addSEND_LIKE_MESSAGE_ACTIONS,VOLATILE_SEND_RESULT_KEYS,stripVolatileSendIds,isVolatileSendResult,isMessageDeliveryObject, andisLoopVetoResult; branch delivery results to the id-stripped hash insidehashToolOutcomeand give blocked/veto results no hash.isVolatileSendResultdelegates provider-docked tool names to the sharedisMessagingToolSendAction(fromembedded-agent-messaging.ts), unifying the send-action source of truth; the denylist also coversplatformMessageId/platformMessageIds.src/agents/tool-loop-detection.test.ts— fifteen regression cases on the realjsonResultresult shape (stable hash across ids in details+text, nested broadcast ids, escalation to critical formessagesend, siblingreplyandupload-file(fresh file id) actions,sessions_send, and a provider-dockedtelegramsend, no false-positive on distinct messages, a non-sendmessagereadwhose id/timestamp progress must stay distinct, a stable nested route id that must survive the strip, and a critical block that stays sticky after the veto result is recorded).messageaction hashing, thresholds, and retry caps are untouched — nothing was loosened or raised; this restores the existing block behavior for delivery results only.docs/reference/config).extensions/*api/runtime-api, registry/loader).Reproduction
message.sendwith identical arguments.ping_pongfires at warning level but never reaches critical/block; the user receives 30+ duplicate messages until run-retries are exhausted.Deterministic source reproduction (added as regression tests): record repeated sends with identical args but changing delivery ids in the result, then call
detectToolCallLoop— before the fix it can never reach a critical branch.Real behavior proof
Behavior addressed (#89090 — send loops can now be blocked, not just warned): repeated identical
message.send(andsessions_send) escalates to a critical block instead of spamming until run-retry exhaustion.Real environment tested (Linux, Node — Vitest against the production loop-detection module using the real
jsonResultresult shape; no live channel run): verified viatool-loop-detection.test.ts, plus core production type-check and a scoped test type-check over the two changed files, plus format check. A live multi-channel gateway loop was not run.Exact steps or command run after this patch (repo-root):
node scripts/run-vitest.mjs src/agents/tool-loop-detection.test.ts;node scripts/run-tsgo.mjs -p tsconfig.core.json(core types) and a scopedtsgoover the changed production + test files (test types);oxfmt --checkon the changed files.Evidence after fix (Vitest output for the touched test file):
Fifteen new cases pass, all built on the real
jsonResultresult shape (volatile id present in bothdetailsand the stringifiedtext): a stable result hash across two sends whose ids differ; stable hash acrossbroadcastresults with per-call nested ids; escalation tocriticalformessagesend,reply,upload-file,sessions_send, and a provider-dockedtelegramsend despite every id differing; a plugin-shaped result whose message object carries a bare per-sendid; nostuckresult when the sends carry distinct content (different args); a non-sendmessagereadwhose results differ only by id/timestamp staying distinct (full hashing preserved); a stable nested route id that survives the strip; a changingconversation.idthat stays below critical; and a critical block that stays sticky after the veto result is recorded.Observed result after fix (identical-args send loops reach critical/block):
detectToolCallLoopreportsstuckatcriticalfor repeated identical sends, while distinct-content sends remain unflagged.What was not tested (live channel/gateway runaway loop): a real Feishu/gateway loop session was not reproduced live; the proof is the deterministic module-level reproduction the issue itself describes.
Repro confirmation: with the production delivery branch removed on an otherwise-identical tree, the four send behavior cases fail (stable-hash, nested-broadcast, and both escalation cases); and when the branch is instead broadened to every
messageaction (dropping theparams.actiongate), the non-sendreadcase fails — so both the fix and its action-scoping are covered by tests that fail on the shape the runtime actually produces rather than passing vacuously. All cases pass with the scoped fix.Risk / Mitigation
argsHashgate remain, so distinct messages stay distinguishable (covered by the no-false-positive regression test).Change Type (select all)
Scope (select all touched areas)
Linked Issue/PR
Fixes #89090