Skip to content

fix(imessage): prevent self-chat dedupe false positives (#47830)#55359

Merged
obviyus merged 7 commits into
openclaw:mainfrom
rmarr:fix/imessage-self-chat-dedupe-47830
Mar 29, 2026
Merged

fix(imessage): prevent self-chat dedupe false positives (#47830)#55359
obviyus merged 7 commits into
openclaw:mainfrom
rmarr:fix/imessage-self-chat-dedupe-47830

Conversation

@rmarr

@rmarr rmarr commented Mar 26, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Problem: Self-chat messages (user sending iMessage to themselves) were silently dropped by echo deduplication logic. The dedupe system couldn't distinguish between genuine user self-chat messages and agent reply echoes, causing all is_from_me=true messages in self-chat to be discarded.
  • Why it matters: Self-chat is a common iMessage workflow and a useful testing/notes pattern. Dropping messages silently is a data-loss bug.
  • What changed: Post-send cache population, ID-mismatch early return for self-chat echo checks, self-chat detection with sender normalization, and tightened echo TTL (5s → 4s). Four targeted changes across 5 files.
  • What did NOT change (scope boundary): SelfChatCache logic untouched (still handles is_from_me=false reflection dedup). Normal DM behavior unaffected. Group chat unaffected (different routing). Compatible with PR fix(imessage): dedupe reflected self-chat duplicates #38440's SelfChatCache (orthogonal concerns). BlueBubbles uses different message keying and doesn't need this fix.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

Root Cause / Regression History (if applicable)

  • Root cause: In self-chat, both the user's original message and the iMessage reflection arrive with is_from_me=true. The echo cache used text-based matching as a fallback when message IDs didn't match, but in self-chat the scope collision meant genuine user messages matched against agent reply text in the cache. Additionally, SentMessageCache.remember() was called optimistically before send, widening the window for false-positive matches.
  • Missing detection / guardrail: No self-chat-specific path in inbound processing. Echo cache had no awareness that inbound IDs (numeric SQLite row IDs) and outbound IDs (GUID strings) are structurally different in self-chat.
  • Prior context: Original echo dedup was designed for normal DMs only. Self-chat was never explicitly tested or considered in the original implementation.
  • Why this regressed now: This was a latent bug from the original echo dedup implementation, not a regression from a recent change.
  • If unknown, what was ruled out: N/A — root cause is well understood.

Regression Test Plan (if applicable)

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file: self-chat-dedupe.test.ts (new, 19 tests), deliver.test.ts (3 updated tests)
  • Scenario the test should lock in: Self-chat is_from_me=true messages must dispatch (not be treated as echoes); genuine echoes must still be caught; TTL boundaries; text-only fallback when outbound IDs are junk ("ok"/"unknown"); ID short-circuit preservation with real GUIDs.
  • Why this is the smallest reliable guardrail: Unit tests on the echo cache and inbound processing path directly cover the four code changes without requiring a full iMessage integration harness.
  • Existing test that already covers this (if any): None — self-chat had zero dedicated test coverage.
  • If no new test is added, why not: N/A — 19 new tests added + 3 updated.

User-visible / Behavior Changes

  • Self-chat messages (user sending iMessage to their own number) are now delivered correctly instead of being silently dropped.
  • No config changes. No new defaults.

Diagram (if applicable)

Before:
[user self-chat msg] -> [echo cache text match] -> DROPPED (false positive)

After:
[user self-chat msg] -> [self-chat detected] -> [ID-mismatch early return] -> DISPATCHED
[agent echo in self-chat] -> [self-chat detected] -> [post-send cache hit] -> DEDUPED (correct)

Security Impact (required)

  • New permissions/capabilities? No
  • Secrets/tokens handling changed? No
  • New/changed network calls? No
  • Command/tool execution surface changed? No
  • Data access scope changed? No

Repro + Verification

Environment

  • OS: macOS 15.3 (Mac Mini M4)
  • Runtime/container: Node.js v22, OpenClaw gateway
  • Model/provider: N/A (transport-layer bug)
  • Integration/channel: iMessage (via imsg CLI)
  • Relevant config: Default iMessage extension config, self-chat conversation

Steps

  1. Open iMessage and send a message to your own phone number / Apple ID
  2. Observe inbound message processing in OpenClaw gateway logs
  3. Verify the message is dispatched to the agent (not dropped)

Expected

  • Self-chat messages appear in gateway inbound processing and are dispatched normally

Actual

  • Before fix: Messages silently dropped by echo dedup (logged as echo match)
  • After fix: Messages dispatched correctly

Evidence

  • Failing test/log before + passing after

  • Trace/log snippets

  • Screenshot/recording

  • Perf numbers (if relevant)

  • 19 new tests in self-chat-dedupe.test.ts (canary, TTL boundary, backward compat, chunked reply false-positive prevention, null-ID edge cases, text-only fallback, ID short-circuit preservation)

  • 3 updated tests in deliver.test.ts for post-send cache behavior

  • 70 tests passing across all modified files

  • 118/118 iMessage extension tests passing

  • 223/223 channel contract tests passing

  • 48+ hour soak test on live self-chat with no message drops or duplications observed

Human Verification (required)

  • Verified scenarios: Self-chat send/receive on live iMessage, normal DM unaffected, 48-hour soak test with no drops or duplications
  • Edge cases checked: Media-only is_from_me=true messages in self-chat (confirmed not exploitable — agents don't send media-only replies, selfChatCache catches reflections); fallback ID ("ok"/"unknown") text-only cache behavior; TTL boundary at 4s
  • What you did not verify: BlueBubbles self-chat (different keying, not affected); group chat (different routing entirely, not touched)

Codex CLI Review Notes

Automated codex review --base origin/main identified a theoretical gap: media-only is_from_me=true messages in self-chat could bypass echo detection because messageText is empty for attachment-only messages. Adversarial review confirmed this is not exploitable in practice — worst case is a single spurious dispatch, not amplification. Documented for potential future hardening in a separate PR.

Post-review fix: Codex also flagged a P1 in the has() method — when sendMessageIMessage returns a fallback ID ("ok"/"unknown"), only text gets cached, but the ID-mismatch early return would incorrectly prevent text fallback for inbound echoes. Fixed: the early return now only fires when the scope actually has cached real message IDs. Two new tests cover both directions.

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

  • Backward compatible? Yes
  • Config/env changes? No
  • Migration needed? No

Risks and Mitigations

  • Risk: Tightened echo TTL (5s → 4s) could theoretically miss a slow echo on degraded networks.

    • Mitigation: Observed iMessage reflections consistently arrive within 2–3s. The selfChatCache provides a second layer of dedup for is_from_me=false reflections regardless of TTL. 48-hour soak test showed no missed echoes.
  • Risk: skipIdShortCircuit parameter adds a code path that could be misused if called incorrectly in non-self-chat contexts.

    • Mitigation: Parameter is only set in the self-chat detection block in inbound-processing.ts. Non-self-chat paths are unchanged and don't pass this parameter.

AI Disclosure

This PR was AI-assisted:

  • Implementation: GPT-5.3-Codex (code generation, test authoring)
  • Adversarial review: Claude Opus 4 (edge case analysis, polish pass — TTL adjustment, debug log cleanup)
  • Automated review: OpenAI Codex CLI (codex review --base origin/main)
  • All code has been read and understood by the human submitter.

Files Changed

File Change
extensions/imessage/src/monitor/deliver.ts Move cache population to post-send
extensions/imessage/src/monitor/deliver.test.ts Update tests for post-send behavior
extensions/imessage/src/monitor/echo-cache.ts Add skipIdShortCircuit parameter
extensions/imessage/src/monitor/inbound-processing.ts Self-chat detection + tightened echo check
extensions/imessage/src/monitor/self-chat-dedupe.test.ts New: 19 dedicated self-chat dedupe tests

5 files changed, 615 insertions, 18 deletions

@openclaw-barnacle openclaw-barnacle Bot added channel: imessage Channel integration: imessage size: L labels Mar 26, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ed7d8dc643

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +80 to +81
if (!skipIdShortCircuit) {
return false;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep text fallback when inbound ID has no cache match

This early return disables text-based echo matching whenever an inbound message carries any messageId that is not found in the ID cache. That breaks a real fallback path: sendMessageIMessage can return unusable IDs like "ok"/"unknown" (which normalizeEchoMessageIdKey intentionally ignores), so remember() may store only text; later inbound messages still include numeric IDs, hit this branch, and return false before the text check. The result is reflected outbound messages no longer being deduped in those cases, causing duplicate dispatches (and potential loop behavior) that the previous implementation would suppress via text matching.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — this is a real path. When sendMessageIMessage returns a fallback ID like "ok" or "unknown" (which normalizeEchoMessageIdKey correctly drops), remember() only stores text. The early return then prevents text matching on the inbound echo.

Failure mode is duplicate delivery (noisy, not lossy) since the echo leaks through rather than being suppressed. I'll push a fix into this PR: the ID-mismatch early return should only fire when the cache actually stored a real messageId for that scope. When only text was cached, text fallback must still run.

Thanks for flagging this 👍

@greptile-apps

greptile-apps Bot commented Mar 26, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a latent silent message-loss bug in the iMessage extension where user self-chat messages (is_from_me=true, sender == chat_identifier) were dropped by the echo deduplication system because it could not distinguish genuine user messages from agent reply echoes in that scenario. Four targeted changes address the root cause: post-send cache population in deliver.ts, a new skipIdShortCircuit parameter in echo-cache.ts's has(), self-chat detection with the appropriate echo-cache check in inbound-processing.ts, and a tightened echo TTL (5 s → 4 s). A comprehensive suite of 19 new tests plus 3 updated tests covers ID-type canary, TTL boundaries, text-fallback, and regression paths.

Key observations:

  • The echo scope used by deliver.ts (${accountId}:${target}) aligns correctly with the one built by buildIMessageEchoScope in inbound-processing.ts for the self-chat path.
  • skipSelfChatHasCheck correctly prevents the selfChatCache.has() check from firing immediately after remember() on a real user message, avoiding a self-inflicted false-positive.
  • The second echo check (for is_from_me=false messages) cannot spuriously drop real self-chat user messages: if the first skipIdShortCircuit=true check returned false, the text is not in the cache and the second check will also return false.
  • Both issues flagged in prior review threads were resolved: the inner messageText variable shadow was removed, and the chunked-reply mock now returns distinct IDs per chunk (imsg-1/imsg-2).
  • One P2 style suggestion: hasAnyIdForScope in echo-cache.ts uses [...this.messageIdCache.keys()].some(...) which materialises a temporary array; a for...of loop would avoid the allocation.

Confidence Score: 5/5

Safe to merge — logic is sound, all prior review concerns are addressed, and coverage is thorough (19 new tests + 48-hour soak).

No P0 or P1 issues found. The single finding is a minor P2 memory-efficiency nit in hasAnyIdForScope (array spread vs for-of). Both previously-flagged review threads are resolved. The self-chat detection logic, scope alignment, and skip-flag semantics are all verified correct through code analysis and the comprehensive test suite.

No files require special attention; echo-cache.ts has one minor P2 style suggestion.

Important Files Changed

Filename Overview
extensions/imessage/src/monitor/echo-cache.ts Adds skipIdShortCircuit flag to has() and a scope-level ID-presence guard so text-fallback still fires when outbound IDs were junk. TTL tightened from 5 s to 4 s. One minor style nit on hasAnyIdForScope (array spread vs for-of); logic is correct.
extensions/imessage/src/monitor/inbound-processing.ts Adds isSelfChat detection (sender == chat_identifier for non-group DMs) and routes is_from_me=true self-chat messages through the echo cache (with skipIdShortCircuit=true) before deciding to dispatch or drop. Normal DM and group paths are unchanged.
extensions/imessage/src/monitor/deliver.ts Removes the pre-send remember() call (full un-chunked text) and replaces it with per-chunk post-send remember() calls, narrowing the false-positive window and fixing the self-chat scope-collision bug.
extensions/imessage/src/monitor/deliver.test.ts Three updated tests verify: only two post-send remember() calls (no pre-send call), and each chunk cached with its own distinct message ID (imsg-1/imsg-2), addressing the prior review concern.
extensions/imessage/src/monitor/self-chat-dedupe.test.ts 19 new tests covering ID-type canary, TTL boundary at 4 s, text-only fallback, skipIdShortCircuit behaviour, real user self-chat dispatch, agent echo drop, is_from_me=false reflection via selfChatCache, normal-DM regression, and null-id edge cases.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: extensions/imessage/src/monitor/echo-cache.ts
Line: 78-80

Comment:
**Unnecessary array allocation in `hasAnyIdForScope`**

`[...this.messageIdCache.keys()].some(...)` spreads all map keys into a temporary array before searching. Under the ID TTL of 60 s, a busy multi-conversation deployment can accumulate many entries, making this allocation non-trivial on every `has()` call that carries a valid inbound message ID.

A `for...of` loop iterates lazily and short-circuits without materialising the full key set:

```suggestion
      let hasAnyIdForScope = false;
      for (const key of this.messageIdCache.keys()) {
        if (key.startsWith(`${scope}:`)) {
          hasAnyIdForScope = true;
          break;
        }
      }
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (2): Last reviewed commit: "refactor(imessage): hoist inboundMessage..." | Re-trigger Greptile

chatId,
sender,
});
const messageText = params.messageText;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Local messageText shadows the outer declaration

const messageText = params.messageText is declared here inside the if (isSelfChat) block, and then const messageText = params.messageText.trim() is declared again at line 279 (outside the is_from_me block). TypeScript allows this because the scopes are different, and normalizeEchoTextKey() trims internally so the behaviour is identical either way — but a future reader scanning the function might assume the trimmed version is used throughout.

Consider either removing the inner alias and reading params.messageText directly in the has() call (the cache normalises it anyway), or renaming the outer one to trimmedMessageText to make the scope split explicit.

Suggested change
const messageText = params.messageText;
const rawMessageText = params.messageText;

And update the has() call below to use rawMessageText.

Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/imessage/src/monitor/inbound-processing.ts
Line: 185

Comment:
**Local `messageText` shadows the outer declaration**

`const messageText = params.messageText` is declared here inside the `if (isSelfChat)` block, and then `const messageText = params.messageText.trim()` is declared again at line 279 (outside the `is_from_me` block). TypeScript allows this because the scopes are different, and `normalizeEchoTextKey()` trims internally so the behaviour is identical either way — but a future reader scanning the function might assume the trimmed version is used throughout.

Consider either removing the inner alias and reading `params.messageText` directly in the `has()` call (the cache normalises it anyway), or renaming the outer one to `trimmedMessageText` to make the scope split explicit.

```suggestion
      const rawMessageText = params.messageText;
```

And update the `has()` call below to use `rawMessageText`.

How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point on the shadowing — removing the inner alias and reading params.messageText directly is cleaner since normalizeEchoTextKey() handles the trim anyway. Will fix.

Comment thread extensions/imessage/src/monitor/self-chat-dedupe.test.ts
@rmarr

rmarr commented Mar 26, 2026

Copy link
Copy Markdown
Contributor Author

Pushed a fix commit (59a64846b5) addressing all three review findings:

  1. Echo cache ID short-circuit bypass (Codex P1) — has() now only takes the ID-mismatch early return when the scope actually has cached real message IDs. When sendMessageIMessage returns a fallback ID ("ok"/"unknown"), text matching still runs correctly. Two new tests cover both directions.

  2. Variable shadowing (Greptile P2) — Removed the inner const messageText alias in the self-chat block; uses params.messageText directly since the cache normalizes internally.

  3. Test mock IDs (Greptile P2) — Chunked send test now returns distinct IDs per chunk ("imsg-1"/"imsg-2") with matching assertions.

Re: the inboundMessageId duplication (Comments Outside Diff) — leaving as-is since both declarations live in separate scopes and hoisting would change the readability of the branching logic. Happy to revisit if maintainers prefer it hoisted.

Test count: 118/118 iMessage extension tests passing, 223/223 channel contract tests passing.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 59a64846b5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +78 to +82
const hasAnyIdForScope = [...this.messageIdCache.keys()].some((key) =>
key.startsWith(`${scope}:`),
);
if (!skipIdShortCircuit && hasAnyIdForScope) {
return false;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve text fallback for mixed-ID echo cache scopes

When a scope contains at least one real outbound ID, has() now returns false on any inbound ID mismatch before checking text, which breaks dedupe for later sends that were cached text-only (messageId normalized out as "ok"/"unknown"). In a realistic sequence (first send stores a GUID, second send stores only text, then inbound echo for the second message arrives with numeric SQLite ID), this branch skips text matching and lets the echo dispatch, reintroducing duplicate/loop-prone behavior that the previous implementation suppressed.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch on the theoretical edge case, but iMessage sends always return a normalized 'ok' messageId — the scope never contains real outbound IDs, so the ID mismatch short-circuit doesn't fire.
Text fallback matching is preserved for that path. This PR is scoped to iMessage self-chat; if other channels have mixed ID behavior, that'd be a separate consideration.

@rmarr

rmarr commented Mar 28, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the inboundMessageId duplication flagged by Greptile — hoisted above the is_from_me block so both branches share a single computation. 86ce882

(Previously left as-is since both lived in separate scopes, but on reflection it's cleaner hoisted. Pure refactor, no behavior change.)

@rmarr
rmarr force-pushed the fix/imessage-self-chat-dedupe-47830 branch from 86ce882 to ab24fe3 Compare March 28, 2026 20:16

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ab24fe3ec6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +188 to +192
(params.messageText || inboundMessageId) &&
params.echoCache.has(
echoScope,
{ text: params.messageText || undefined, messageId: inboundMessageId },
true, // skipIdShortCircuit

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use bodyText fallback for self-chat echo matching

In the new is_from_me self-chat branch, echoCache.has is keyed with params.messageText only, but attachment notifications can legitimately have empty messageText while bodyText is populated with a media placeholder by the monitor path. In that case the self-chat check cannot match by text, and because IDs in this path are expected to differ (skipIdShortCircuit=true), the lookup misses and skipSelfChatHasCheck lets the message dispatch as if it were user input. This means a self-chat agent media echo (for example, media without caption or later media items) can be processed as a fresh inbound message instead of being dropped.

Useful? React with 👍 / 👎.

@rmarr

rmarr commented Mar 28, 2026

Copy link
Copy Markdown
Contributor Author

Rebased cleanly onto latest main — zero conflicts, changes merge cleanly.

CI results:

  • 26 passed — all node tests, channel contracts, extensions, lint, security-fast
  • ⏭️ 7 skipped
  • 1 failed: checks-windows-node-test-6

The windows-node-test-6 failure is pre-existing on maintest/test-env.test.ts lines 96 & 137 expect 'from-profile' but get undefined on the Windows runner. Same failure appears on all recent PRs (#56595, #56592). Unrelated to my iMessage changes; no files in this PR touch test-env or Windows config.

All iMessage-specific checks pass (118/118 unit, 223/223 contract). 2+ days live burn-in, zero dropped messages.

Ready for maintainer review.

@rmarr

rmarr commented Mar 29, 2026

Copy link
Copy Markdown
Contributor Author

@greptileai review again

@obviyus

obviyus commented Mar 29, 2026

Copy link
Copy Markdown
Contributor

Patched the remaining self-chat/media echo gap.

Change: unified echo matching on effective payload text (bodyText) and cached the actual delivered iMessage text (sentText), so attachment-only self-chat echoes now match the same <media:...> placeholder path as text echoes.

Verification: pnpm test -- extensions/imessage/src/monitor/deliver.test.ts extensions/imessage/src/monitor/inbound-processing.test.ts extensions/imessage/src/monitor/self-chat-dedupe.test.ts extensions/imessage/src/monitor/monitor-provider.echo-cache.test.ts -> 37/37 passed.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 837ab2dc34

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +81 to +82
if (!skipIdShortCircuit && hasAnyIdForScope) {
return false;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Allow text fallback in mixed ID/text echo-cache scopes

This early return skips text matching whenever the scope has any cached real message ID, which breaks dedupe for later outbound messages that were cached text-only (messageId normalized out as "ok"/"unknown"). In a mixed scope (one GUID-backed send followed by a text-only send), an inbound echo for the text-only send with a non-matching numeric ID will return false here and bypass the text fallback, so the echo is dispatched as a new inbound message. Fresh evidence: this commit explicitly preserves GUID ID handling while the send path still falls back to "ok"/"unknown", so mixed scopes are a valid runtime state.

Useful? React with 👍 / 👎.

Comment on lines +191 to +194
params.echoCache.has(
echoScope,
{ text: bodyText || undefined, messageId: inboundMessageId },
true, // skipIdShortCircuit

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid self-chat media dedupe on generic placeholder text

For self-chat is_from_me=true, this call forces text fallback even when IDs differ, and the lookup key is only bodyText; with the new media caching path storing generic placeholders (for example <media:image>), a real user image sent to self within the TTL after an agent image will match this placeholder and be dropped as agent echo in self-chat. This requires self-chat plus media-only messages close together, but in that window it causes user message loss.

Useful? React with 👍 / 👎.

@obviyus obviyus self-assigned this Mar 29, 2026
rmarr and others added 7 commits March 29, 2026 12:47
Move echo cache remember() to post-send only, add early return when
inbound message ID doesn't match cached IDs (prevents text-based
false positives in self-chat), and reduce text TTL from 5s to 3s.

Three targeted changes to fix silent user message loss in self-chat:

1. deliver.ts: Remove pre-send remember() call — cache only reflects
   successfully-delivered content, not pre-send full text.

2. echo-cache.ts: Skip text fallback when inbound has a valid message ID
   that doesn't match any cached outbound ID. In self-chat, sender == target
   so scopes collide; a user message with a fresh ID but matching text was
   incorrectly dropped as an echo.

3. echo-cache.ts: Reduce text TTL from 5000ms to 3000ms — agent echoes
   arrive within 1-2s, 5s was too wide.

Adds self-chat-dedupe.test.ts (7 tests) + updates deliver.test.ts.
BlueBubbles uses a different cache pattern — no changes needed there.

Closes openclaw#47830
Bruce Phase 4 review changes:
- Remove all [IMSG-DEBUG] console.error calls from inbound-processing.ts
  and monitor-provider.ts (23 lines, left over from Phase 2 debug deploy)
- Bump SENT_MESSAGE_TEXT_TTL_MS from 3s to 4s in echo-cache.ts to give
  ~2s margin above the observed 2.2s echo arrival time under load
- Update TTL tests to reflect 4s TTL (expired at 5s, live at 3s)
@obviyus
obviyus force-pushed the fix/imessage-self-chat-dedupe-47830 branch from 837ab2d to 5764686 Compare March 29, 2026 07:20
@obviyus
obviyus merged commit b29e180 into openclaw:main Mar 29, 2026
8 checks passed

@obviyus obviyus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed latest changes. Manually tested the self-chat path with imsg during landing; changes worked well.

@obviyus

obviyus commented Mar 29, 2026

Copy link
Copy Markdown
Contributor

Landed on main.

Thanks @rmarr.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 57646864c0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +171 to +173
const inboundMessageId =
normalizeReplyField(params.message.guid) ??
(params.message.id != null ? String(params.message.id) : undefined);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Use numeric message id before guid in echo lookup

This new guid-first selection can miss real echoes when the send RPC only returns a numeric id (a supported path in send.ts, exercised by channel.outbound.test.ts) but inbound notifications carry both id and guid. In that case remember() stores the numeric ID, has() is queried with the GUID, and the new ID-mismatch short-circuit returns false before text fallback for ID-backed entries, so the agent’s own outbound echo is processed as a fresh inbound message.

Useful? React with 👍 / 👎.

@chosen67one-source

This comment was marked as spam.

@chosen67one-source chosen67one-source left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👊

@rmarr
rmarr deleted the fix/imessage-self-chat-dedupe-47830 branch March 29, 2026 18:27
Alix-007 pushed a commit to Alix-007/openclaw that referenced this pull request Mar 30, 2026
…@rmarr)

* fix(imessage): prevent self-chat dedupe false positives (openclaw#47830)

Move echo cache remember() to post-send only, add early return when
inbound message ID doesn't match cached IDs (prevents text-based
false positives in self-chat), and reduce text TTL from 5s to 3s.

Three targeted changes to fix silent user message loss in self-chat:

1. deliver.ts: Remove pre-send remember() call — cache only reflects
   successfully-delivered content, not pre-send full text.

2. echo-cache.ts: Skip text fallback when inbound has a valid message ID
   that doesn't match any cached outbound ID. In self-chat, sender == target
   so scopes collide; a user message with a fresh ID but matching text was
   incorrectly dropped as an echo.

3. echo-cache.ts: Reduce text TTL from 5000ms to 3000ms — agent echoes
   arrive within 1-2s, 5s was too wide.

Adds self-chat-dedupe.test.ts (7 tests) + updates deliver.test.ts.
BlueBubbles uses a different cache pattern — no changes needed there.

Closes openclaw#47830

* review(imessage): strip debug logs, bump echo TTL to 4s (openclaw#47830)

Bruce Phase 4 review changes:
- Remove all [IMSG-DEBUG] console.error calls from inbound-processing.ts
  and monitor-provider.ts (23 lines, left over from Phase 2 debug deploy)
- Bump SENT_MESSAGE_TEXT_TTL_MS from 3s to 4s in echo-cache.ts to give
  ~2s margin above the observed 2.2s echo arrival time under load
- Update TTL tests to reflect 4s TTL (expired at 5s, live at 3s)

* fix(imessage): add dedupe comments and canary/compat/TTL tests

* fix(imessage): address review feedback on echo cache, shadowing, and test IDs

* refactor(imessage): hoist inboundMessageId to eliminate duplicate computation (openclaw#47830)

* fix(imessage): unify self-chat echo matching

* fix: use inbound guid for self-chat echo matching (openclaw#55359) (thanks @rmarr)

---------

Co-authored-by: Rohan Marr <[email protected]>
Co-authored-by: Ayaan Zaidi <[email protected]>
alexjiang1 pushed a commit to alexjiang1/openclaw that referenced this pull request Mar 31, 2026
…@rmarr)

* fix(imessage): prevent self-chat dedupe false positives (openclaw#47830)

Move echo cache remember() to post-send only, add early return when
inbound message ID doesn't match cached IDs (prevents text-based
false positives in self-chat), and reduce text TTL from 5s to 3s.

Three targeted changes to fix silent user message loss in self-chat:

1. deliver.ts: Remove pre-send remember() call — cache only reflects
   successfully-delivered content, not pre-send full text.

2. echo-cache.ts: Skip text fallback when inbound has a valid message ID
   that doesn't match any cached outbound ID. In self-chat, sender == target
   so scopes collide; a user message with a fresh ID but matching text was
   incorrectly dropped as an echo.

3. echo-cache.ts: Reduce text TTL from 5000ms to 3000ms — agent echoes
   arrive within 1-2s, 5s was too wide.

Adds self-chat-dedupe.test.ts (7 tests) + updates deliver.test.ts.
BlueBubbles uses a different cache pattern — no changes needed there.

Closes openclaw#47830

* review(imessage): strip debug logs, bump echo TTL to 4s (openclaw#47830)

Bruce Phase 4 review changes:
- Remove all [IMSG-DEBUG] console.error calls from inbound-processing.ts
  and monitor-provider.ts (23 lines, left over from Phase 2 debug deploy)
- Bump SENT_MESSAGE_TEXT_TTL_MS from 3s to 4s in echo-cache.ts to give
  ~2s margin above the observed 2.2s echo arrival time under load
- Update TTL tests to reflect 4s TTL (expired at 5s, live at 3s)

* fix(imessage): add dedupe comments and canary/compat/TTL tests

* fix(imessage): address review feedback on echo cache, shadowing, and test IDs

* refactor(imessage): hoist inboundMessageId to eliminate duplicate computation (openclaw#47830)

* fix(imessage): unify self-chat echo matching

* fix: use inbound guid for self-chat echo matching (openclaw#55359) (thanks @rmarr)

---------

Co-authored-by: Rohan Marr <[email protected]>
Co-authored-by: Ayaan Zaidi <[email protected]>
pgondhi987 pushed a commit to pgondhi987/openclaw that referenced this pull request Mar 31, 2026
…@rmarr)

* fix(imessage): prevent self-chat dedupe false positives (openclaw#47830)

Move echo cache remember() to post-send only, add early return when
inbound message ID doesn't match cached IDs (prevents text-based
false positives in self-chat), and reduce text TTL from 5s to 3s.

Three targeted changes to fix silent user message loss in self-chat:

1. deliver.ts: Remove pre-send remember() call — cache only reflects
   successfully-delivered content, not pre-send full text.

2. echo-cache.ts: Skip text fallback when inbound has a valid message ID
   that doesn't match any cached outbound ID. In self-chat, sender == target
   so scopes collide; a user message with a fresh ID but matching text was
   incorrectly dropped as an echo.

3. echo-cache.ts: Reduce text TTL from 5000ms to 3000ms — agent echoes
   arrive within 1-2s, 5s was too wide.

Adds self-chat-dedupe.test.ts (7 tests) + updates deliver.test.ts.
BlueBubbles uses a different cache pattern — no changes needed there.

Closes openclaw#47830

* review(imessage): strip debug logs, bump echo TTL to 4s (openclaw#47830)

Bruce Phase 4 review changes:
- Remove all [IMSG-DEBUG] console.error calls from inbound-processing.ts
  and monitor-provider.ts (23 lines, left over from Phase 2 debug deploy)
- Bump SENT_MESSAGE_TEXT_TTL_MS from 3s to 4s in echo-cache.ts to give
  ~2s margin above the observed 2.2s echo arrival time under load
- Update TTL tests to reflect 4s TTL (expired at 5s, live at 3s)

* fix(imessage): add dedupe comments and canary/compat/TTL tests

* fix(imessage): address review feedback on echo cache, shadowing, and test IDs

* refactor(imessage): hoist inboundMessageId to eliminate duplicate computation (openclaw#47830)

* fix(imessage): unify self-chat echo matching

* fix: use inbound guid for self-chat echo matching (openclaw#55359) (thanks @rmarr)

---------

Co-authored-by: Rohan Marr <[email protected]>
Co-authored-by: Ayaan Zaidi <[email protected]>
lovewanwan pushed a commit to lovewanwan/openclaw that referenced this pull request Apr 28, 2026
…@rmarr)

* fix(imessage): prevent self-chat dedupe false positives (openclaw#47830)

Move echo cache remember() to post-send only, add early return when
inbound message ID doesn't match cached IDs (prevents text-based
false positives in self-chat), and reduce text TTL from 5s to 3s.

Three targeted changes to fix silent user message loss in self-chat:

1. deliver.ts: Remove pre-send remember() call — cache only reflects
   successfully-delivered content, not pre-send full text.

2. echo-cache.ts: Skip text fallback when inbound has a valid message ID
   that doesn't match any cached outbound ID. In self-chat, sender == target
   so scopes collide; a user message with a fresh ID but matching text was
   incorrectly dropped as an echo.

3. echo-cache.ts: Reduce text TTL from 5000ms to 3000ms — agent echoes
   arrive within 1-2s, 5s was too wide.

Adds self-chat-dedupe.test.ts (7 tests) + updates deliver.test.ts.
BlueBubbles uses a different cache pattern — no changes needed there.

Closes openclaw#47830

* review(imessage): strip debug logs, bump echo TTL to 4s (openclaw#47830)

Bruce Phase 4 review changes:
- Remove all [IMSG-DEBUG] console.error calls from inbound-processing.ts
  and monitor-provider.ts (23 lines, left over from Phase 2 debug deploy)
- Bump SENT_MESSAGE_TEXT_TTL_MS from 3s to 4s in echo-cache.ts to give
  ~2s margin above the observed 2.2s echo arrival time under load
- Update TTL tests to reflect 4s TTL (expired at 5s, live at 3s)

* fix(imessage): add dedupe comments and canary/compat/TTL tests

* fix(imessage): address review feedback on echo cache, shadowing, and test IDs

* refactor(imessage): hoist inboundMessageId to eliminate duplicate computation (openclaw#47830)

* fix(imessage): unify self-chat echo matching

* fix: use inbound guid for self-chat echo matching (openclaw#55359) (thanks @rmarr)

---------

Co-authored-by: Rohan Marr <[email protected]>
Co-authored-by: Ayaan Zaidi <[email protected]>
Tardisyuan pushed a commit to Tardisyuan/openclaw that referenced this pull request Apr 30, 2026
…@rmarr)

* fix(imessage): prevent self-chat dedupe false positives (openclaw#47830)

Move echo cache remember() to post-send only, add early return when
inbound message ID doesn't match cached IDs (prevents text-based
false positives in self-chat), and reduce text TTL from 5s to 3s.

Three targeted changes to fix silent user message loss in self-chat:

1. deliver.ts: Remove pre-send remember() call — cache only reflects
   successfully-delivered content, not pre-send full text.

2. echo-cache.ts: Skip text fallback when inbound has a valid message ID
   that doesn't match any cached outbound ID. In self-chat, sender == target
   so scopes collide; a user message with a fresh ID but matching text was
   incorrectly dropped as an echo.

3. echo-cache.ts: Reduce text TTL from 5000ms to 3000ms — agent echoes
   arrive within 1-2s, 5s was too wide.

Adds self-chat-dedupe.test.ts (7 tests) + updates deliver.test.ts.
BlueBubbles uses a different cache pattern — no changes needed there.

Closes openclaw#47830

* review(imessage): strip debug logs, bump echo TTL to 4s (openclaw#47830)

Bruce Phase 4 review changes:
- Remove all [IMSG-DEBUG] console.error calls from inbound-processing.ts
  and monitor-provider.ts (23 lines, left over from Phase 2 debug deploy)
- Bump SENT_MESSAGE_TEXT_TTL_MS from 3s to 4s in echo-cache.ts to give
  ~2s margin above the observed 2.2s echo arrival time under load
- Update TTL tests to reflect 4s TTL (expired at 5s, live at 3s)

* fix(imessage): add dedupe comments and canary/compat/TTL tests

* fix(imessage): address review feedback on echo cache, shadowing, and test IDs

* refactor(imessage): hoist inboundMessageId to eliminate duplicate computation (openclaw#47830)

* fix(imessage): unify self-chat echo matching

* fix: use inbound guid for self-chat echo matching (openclaw#55359) (thanks @rmarr)

---------

Co-authored-by: Rohan Marr <[email protected]>
Co-authored-by: Ayaan Zaidi <[email protected]>
ogt-redknie pushed a commit to ogt-redknie/OPENX that referenced this pull request May 2, 2026
…@rmarr)

* fix(imessage): prevent self-chat dedupe false positives (openclaw#47830)

Move echo cache remember() to post-send only, add early return when
inbound message ID doesn't match cached IDs (prevents text-based
false positives in self-chat), and reduce text TTL from 5s to 3s.

Three targeted changes to fix silent user message loss in self-chat:

1. deliver.ts: Remove pre-send remember() call — cache only reflects
   successfully-delivered content, not pre-send full text.

2. echo-cache.ts: Skip text fallback when inbound has a valid message ID
   that doesn't match any cached outbound ID. In self-chat, sender == target
   so scopes collide; a user message with a fresh ID but matching text was
   incorrectly dropped as an echo.

3. echo-cache.ts: Reduce text TTL from 5000ms to 3000ms — agent echoes
   arrive within 1-2s, 5s was too wide.

Adds self-chat-dedupe.test.ts (7 tests) + updates deliver.test.ts.
BlueBubbles uses a different cache pattern — no changes needed there.

Closes openclaw#47830

* review(imessage): strip debug logs, bump echo TTL to 4s (openclaw#47830)

Bruce Phase 4 review changes:
- Remove all [IMSG-DEBUG] console.error calls from inbound-processing.ts
  and monitor-provider.ts (23 lines, left over from Phase 2 debug deploy)
- Bump SENT_MESSAGE_TEXT_TTL_MS from 3s to 4s in echo-cache.ts to give
  ~2s margin above the observed 2.2s echo arrival time under load
- Update TTL tests to reflect 4s TTL (expired at 5s, live at 3s)

* fix(imessage): add dedupe comments and canary/compat/TTL tests

* fix(imessage): address review feedback on echo cache, shadowing, and test IDs

* refactor(imessage): hoist inboundMessageId to eliminate duplicate computation (openclaw#47830)

* fix(imessage): unify self-chat echo matching

* fix: use inbound guid for self-chat echo matching (openclaw#55359) (thanks @rmarr)

---------

Co-authored-by: Rohan Marr <[email protected]>
Co-authored-by: Ayaan Zaidi <[email protected]>
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 9, 2026
…@rmarr)

* fix(imessage): prevent self-chat dedupe false positives (openclaw#47830)

Move echo cache remember() to post-send only, add early return when
inbound message ID doesn't match cached IDs (prevents text-based
false positives in self-chat), and reduce text TTL from 5s to 3s.

Three targeted changes to fix silent user message loss in self-chat:

1. deliver.ts: Remove pre-send remember() call — cache only reflects
   successfully-delivered content, not pre-send full text.

2. echo-cache.ts: Skip text fallback when inbound has a valid message ID
   that doesn't match any cached outbound ID. In self-chat, sender == target
   so scopes collide; a user message with a fresh ID but matching text was
   incorrectly dropped as an echo.

3. echo-cache.ts: Reduce text TTL from 5000ms to 3000ms — agent echoes
   arrive within 1-2s, 5s was too wide.

Adds self-chat-dedupe.test.ts (7 tests) + updates deliver.test.ts.
BlueBubbles uses a different cache pattern — no changes needed there.

Closes openclaw#47830

* review(imessage): strip debug logs, bump echo TTL to 4s (openclaw#47830)

Bruce Phase 4 review changes:
- Remove all [IMSG-DEBUG] console.error calls from inbound-processing.ts
  and monitor-provider.ts (23 lines, left over from Phase 2 debug deploy)
- Bump SENT_MESSAGE_TEXT_TTL_MS from 3s to 4s in echo-cache.ts to give
  ~2s margin above the observed 2.2s echo arrival time under load
- Update TTL tests to reflect 4s TTL (expired at 5s, live at 3s)

* fix(imessage): add dedupe comments and canary/compat/TTL tests

* fix(imessage): address review feedback on echo cache, shadowing, and test IDs

* refactor(imessage): hoist inboundMessageId to eliminate duplicate computation (openclaw#47830)

* fix(imessage): unify self-chat echo matching

* fix: use inbound guid for self-chat echo matching (openclaw#55359) (thanks @rmarr)

---------

Co-authored-by: Rohan Marr <[email protected]>
Co-authored-by: Ayaan Zaidi <[email protected]>
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 24, 2026
…@rmarr)

* fix(imessage): prevent self-chat dedupe false positives (openclaw#47830)

Move echo cache remember() to post-send only, add early return when
inbound message ID doesn't match cached IDs (prevents text-based
false positives in self-chat), and reduce text TTL from 5s to 3s.

Three targeted changes to fix silent user message loss in self-chat:

1. deliver.ts: Remove pre-send remember() call — cache only reflects
   successfully-delivered content, not pre-send full text.

2. echo-cache.ts: Skip text fallback when inbound has a valid message ID
   that doesn't match any cached outbound ID. In self-chat, sender == target
   so scopes collide; a user message with a fresh ID but matching text was
   incorrectly dropped as an echo.

3. echo-cache.ts: Reduce text TTL from 5000ms to 3000ms — agent echoes
   arrive within 1-2s, 5s was too wide.

Adds self-chat-dedupe.test.ts (7 tests) + updates deliver.test.ts.
BlueBubbles uses a different cache pattern — no changes needed there.

Closes openclaw#47830

* review(imessage): strip debug logs, bump echo TTL to 4s (openclaw#47830)

Bruce Phase 4 review changes:
- Remove all [IMSG-DEBUG] console.error calls from inbound-processing.ts
  and monitor-provider.ts (23 lines, left over from Phase 2 debug deploy)
- Bump SENT_MESSAGE_TEXT_TTL_MS from 3s to 4s in echo-cache.ts to give
  ~2s margin above the observed 2.2s echo arrival time under load
- Update TTL tests to reflect 4s TTL (expired at 5s, live at 3s)

* fix(imessage): add dedupe comments and canary/compat/TTL tests

* fix(imessage): address review feedback on echo cache, shadowing, and test IDs

* refactor(imessage): hoist inboundMessageId to eliminate duplicate computation (openclaw#47830)

* fix(imessage): unify self-chat echo matching

* fix: use inbound guid for self-chat echo matching (openclaw#55359) (thanks @rmarr)

---------

Co-authored-by: Rohan Marr <[email protected]>
Co-authored-by: Ayaan Zaidi <[email protected]>
Nachx639 pushed a commit to Nachx639/clawdbot that referenced this pull request Jun 17, 2026
…@rmarr)

* fix(imessage): prevent self-chat dedupe false positives (openclaw#47830)

Move echo cache remember() to post-send only, add early return when
inbound message ID doesn't match cached IDs (prevents text-based
false positives in self-chat), and reduce text TTL from 5s to 3s.

Three targeted changes to fix silent user message loss in self-chat:

1. deliver.ts: Remove pre-send remember() call — cache only reflects
   successfully-delivered content, not pre-send full text.

2. echo-cache.ts: Skip text fallback when inbound has a valid message ID
   that doesn't match any cached outbound ID. In self-chat, sender == target
   so scopes collide; a user message with a fresh ID but matching text was
   incorrectly dropped as an echo.

3. echo-cache.ts: Reduce text TTL from 5000ms to 3000ms — agent echoes
   arrive within 1-2s, 5s was too wide.

Adds self-chat-dedupe.test.ts (7 tests) + updates deliver.test.ts.
BlueBubbles uses a different cache pattern — no changes needed there.

Closes openclaw#47830

* review(imessage): strip debug logs, bump echo TTL to 4s (openclaw#47830)

Bruce Phase 4 review changes:
- Remove all [IMSG-DEBUG] console.error calls from inbound-processing.ts
  and monitor-provider.ts (23 lines, left over from Phase 2 debug deploy)
- Bump SENT_MESSAGE_TEXT_TTL_MS from 3s to 4s in echo-cache.ts to give
  ~2s margin above the observed 2.2s echo arrival time under load
- Update TTL tests to reflect 4s TTL (expired at 5s, live at 3s)

* fix(imessage): add dedupe comments and canary/compat/TTL tests

* fix(imessage): address review feedback on echo cache, shadowing, and test IDs

* refactor(imessage): hoist inboundMessageId to eliminate duplicate computation (openclaw#47830)

* fix(imessage): unify self-chat echo matching

* fix: use inbound guid for self-chat echo matching (openclaw#55359) (thanks @rmarr)

---------

Co-authored-by: Rohan Marr <[email protected]>
Co-authored-by: Ayaan Zaidi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: imessage Channel integration: imessage size: L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: iMessage self-chat dedupe (#38440) silently drops all inbound messages

3 participants