Skip to content

[codex] fix(imessage): normalize attributed echo prefixes#66169

Closed
coder999999999 wants to merge 2 commits into
openclaw:mainfrom
coder999999999:codex/imessage-attributed-echo-prefix
Closed

[codex] fix(imessage): normalize attributed echo prefixes#66169
coder999999999 wants to merge 2 commits into
openclaw:mainfrom
coder999999999:codex/imessage-attributed-echo-prefix

Conversation

@coder999999999

@coder999999999 coder999999999 commented Apr 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Problem: iMessage reflected replies can carry short attributed-text prefix bytes like \x93b\x02, which makes echo-cache text fallback miss the agent's own outbound text.
  • Why it matters: when the reflected text misses echo matching, self-chat and direct-chat reply loops can re-enter the session and trigger duplicate or recursive replies.
  • What changed: extensions/imessage/src/monitor/echo-cache.ts now strips the leading attributed-text marker pattern before newline/trim normalization, and targeted regression tests were added for cache matching plus the self-chat inbound decision path.
  • What did NOT change (scope boundary): no routing changes, no TTL changes, no send-path changes, and no generic text sanitization outside the iMessage echo cache.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

Root Cause (if applicable)

  • Root cause: normalizeEchoTextKey() normalized newlines and trimming only, so reflected texts like "\x93b\x02Cached reply" did not match the clean outbound text stored in the echo cache.
  • Missing detection / guardrail: there was no regression test covering attributed-text prefix bytes in the iMessage echo fallback path.
  • Contributing context (if known): iMessage chat.db reflections can preserve short attributed-text marker bytes on some messages, especially richer/longer text.

Regression Test Plan (if applicable)

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file:
    • extensions/imessage/src/monitor/monitor-provider.echo-cache.test.ts
    • extensions/imessage/src/monitor/self-chat-dedupe.test.ts
  • Scenario the test should lock in: a reflected iMessage body with a leading attributed-text prefix should still match the clean outbound cached text and be dropped as an echo.
  • Why this is the smallest reliable guardrail: it exercises the exact normalization seam and the self-chat inbound decision seam without needing a live iMessage environment.
  • Existing test that already covers this (if any): existing echo-cache and self-chat tests covered clean text and control-character-ish cases, but not the attributed prefix byte shape from iMessage echo detection fails for messages with attributed text prefix bytes #64309.
  • If no new test is added, why not: N/A

User-visible / Behavior Changes

  • iMessage reflected replies with attributed-text prefix bytes should stop re-entering the session as fresh inbound messages.

Diagram (if applicable)

Before:
[agent reply] -> [iMessage reflection with prefix bytes] -> [text mismatch in echo cache] -> [loop / duplicate reply]

After:
[agent reply] -> [iMessage reflection with prefix bytes] -> [prefix stripped before echo match] -> [echo dropped]

Security Impact (required)

  • New permissions/capabilities? (No)
  • Secrets/tokens handling changed? (No)
  • New/changed network calls? (No)
  • Command/tool execution surface changed? (No)
  • Data access scope changed? (No)
  • If any Yes, explain risk + mitigation:

Repro + Verification

Environment

  • OS: macOS
  • Runtime/container: local Node/pnpm checkout
  • Model/provider: N/A
  • Integration/channel (if any): iMessage
  • Relevant config (redacted): iMessage echo-cache/self-chat test paths; local imsg access to Messages database

Steps

  1. Cache a clean outbound iMessage reply text.
  2. Simulate a reflected inbound message carrying a leading attributed-text prefix like \x93b\x02.
  3. Verify the reflected text still matches the cached outbound text and is dropped as an echo.
  4. Manually send live verification messages through imsg to a real iMessage target and confirm delivery on the recipient device.

Expected

  • The prefixed reflected text normalizes to the clean outbound text and is treated as an echo.
  • Live iMessage test messages deliver successfully to the target device.

Actual

  • Before this patch, the prefixed reflected text missed echo matching and could re-enter the session.
  • After this patch, the prefixed text matches in the targeted regression path, and the live verification sends delivered successfully.

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios: targeted echo-cache regression test, targeted self-chat inbound regression test, successful branch commit hook (pnpm check), and live iMessage test sends via imsg during validation.
  • Edge cases checked: prefixed reflected text with a GUID-backed outbound cache entry; self-chat is_from_me=true reflection path with the same prefix shape; real iMessage delivery path remained functional while validating on macOS.
  • What you did not verify: a full live OpenClaw+iMessage end-to-end loop reproduction with the branch wired into an enabled iMessage channel.

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

  • Backward compatible? (Yes)
  • Config/env changes? (No)
  • Migration needed? (No)
  • If yes, exact upgrade steps:

Risks and Mitigations

  • Risk: the prefix stripper could normalize an unusual legitimate leading control-byte sequence that was not an iMessage attributed-text marker.
    • Mitigation: the logic is intentionally narrow, only strips short leading marker-like runs, and is covered by targeted tests instead of broad text rewriting.

@openclaw-barnacle openclaw-barnacle Bot added channel: imessage Channel integration: imessage size: S labels Apr 13, 2026
@coder999999999
coder999999999 marked this pull request as ready for review April 14, 2026 18:07
Copilot AI review requested due to automatic review settings April 14, 2026 18:07
@greptile-apps

greptile-apps Bot commented Apr 14, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds stripLeadingIMessageAttributedTextPrefix in echo-cache.ts to normalize C1/attributed-text marker bytes (e.g. \x93b\x02) that iMessage chat.db can prepend to reflected messages, causing echo-cache text lookups to miss and re-enter the session as duplicates. Two targeted regression tests lock in the fix at the normalization seam and the self-chat inbound decision path. The only finding is a dead-code branch (offset += second ? 3 : 2 always adds 3).

Confidence Score: 5/5

  • Safe to merge — the change is narrowly scoped to echo-cache normalization with targeted regression tests and no routing or send-path changes.
  • The single finding (dead ternary branch in offset += second ? 3 : 2) is a P2 style issue with no functional impact. All logic, TTL semantics, and test coverage look correct.
  • No files require special attention.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: extensions/imessage/src/monitor/echo-cache.ts
Line: 63

Comment:
**Dead branch in ternary**

`offset += second ? 3 : 2` — the `: 2` arm is unreachable. Reaching this line requires `third &&` to be truthy, meaning `text[offset+2]` is a valid character. A valid string index always returns a non-empty single-character string (truthy), and a defined `text[offset+2]` implies `text[offset+1]` (`second`) is also defined and truthy. So `second` is always truthy here and the offset always increments by `3`.

```suggestion
      offset += 3;
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "fix(imessage): normalize attributed echo..." | Re-trigger Greptile

Comment thread extensions/imessage/src/monitor/echo-cache.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes iMessage echo-cache text matching when reflected messages include leading attributed-text marker bytes (e.g. \x93b\x02), preventing self-chat/direct-chat echo loops from re-entering the session.

Changes:

  • Add leading attributed-prefix stripping to normalizeEchoTextKey() before newline/trim normalization.
  • Add unit/regression tests covering attributed-prefix echo matching in the echo cache and the self-chat inbound decision path.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
extensions/imessage/src/monitor/echo-cache.ts Adds attributed-prefix/control-prefix normalization for echo text keys.
extensions/imessage/src/monitor/monitor-provider.echo-cache.test.ts Adds a regression test ensuring prefixed reflected text matches cached outbound text.
extensions/imessage/src/monitor/self-chat-dedupe.test.ts Adds a regression test ensuring self-chat is_from_me=true reflections with prefix bytes are dropped as agent echoes.

Comment thread extensions/imessage/src/monitor/echo-cache.ts Outdated
Comment thread extensions/imessage/src/monitor/echo-cache.ts Outdated
@clawsweeper

clawsweeper Bot commented Apr 27, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge.

Summary
The branch adds iMessage sent-message echo-cache prefix stripping for attributed/control-byte reflections, covers the echo-cache and self-chat drop paths, and adds a changelog entry.

Reproducibility: yes. at source level. Current main's normalizer leaves the reported U+0093 b U+0002 prefix on the echo-cache key, and the self-chat inbound drop path depends on that key matching bodyText.

Real behavior proof
Needs real behavior proof before merge: The PR body and comments narrate local tests/live sends, but no screenshot, terminal output, copied live output, recording, linked artifact, or redacted log shows the after-fix prefixed echo drop. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, ask a maintainer to comment @clawsweeper re-review.

Next step before merge
Human follow-up is needed because the external PR lacks inspectable real behavior proof and the branch is conflicting against current iMessage echo-cache changes.

Security
Cleared: The diff is limited to iMessage normalization, focused tests, and changelog text; it does not change dependencies, workflows, secrets, permissions, downloads, or package execution surfaces.

Review findings

  • [P2] Preserve the current marker normalizer — extensions/imessage/src/monitor/echo-cache.ts:34-39
  • [P2] Keep persisted echo matching on rebase — extensions/imessage/src/monitor/echo-cache.ts:126-128
Review details

Best possible solution:

Rebase the attributed-prefix normalizer onto current main, preserve current marker cleanup and persisted echo-cache behavior, keep focused tests for both prefix families, and add inspectable after-fix proof before merge.

Do we have a high-confidence way to reproduce the issue?

Yes, at source level. Current main's normalizer leaves the reported U+0093 b U+0002 prefix on the echo-cache key, and the self-chat inbound drop path depends on that key matching bodyText.

Is this the best way to solve the issue?

No, not as this branch currently stands. The echo-cache normalizer is the right narrow seam, but the PR must be rebased over current main so it adds C1 attributed-prefix support without losing marker cleanup or persisted echo lookup.

Full review comments:

  • [P2] Preserve the current marker normalizer — extensions/imessage/src/monitor/echo-cache.ts:34-39
    Current main already strips U+FEFF/U+FFFD/U+FFFE/U+FFFF leading attributedBody markers through the merged canonical marker fix. This branch's new predicate only covers U+FFFD and C0/C1-style bytes, so resolving the conflict by taking it as-is would reopen that landed echo miss; fold the current marker set and tests into the new prefix normalizer.
    Confidence: 0.9
  • [P2] Keep persisted echo matching on rebase — extensions/imessage/src/monitor/echo-cache.ts:126-128
    The PR head predates current main's hasPersistedIMessageEcho() call and goes straight to in-memory text/id matching. If this method wins during conflict resolution, restart/catchup sent-echo dedupe regresses; rebase the prefix stripping onto the current implementation instead of replacing has().
    Confidence: 0.88

Overall correctness: patch is incorrect
Overall confidence: 0.9

Acceptance criteria:

  • pnpm test extensions/imessage/src/monitor/monitor-provider.echo-cache.test.ts extensions/imessage/src/monitor/self-chat-dedupe.test.ts
  • pnpm check:changed

What I checked:

Likely related people:

  • rmarr: GitHub path history shows b29e180 introduced the central iMessage sent-message echo cache, post-send population behavior, text fallback safeguards, and self-chat tests. (role: introduced echo-cache/self-chat dedupe behavior; confidence: high; commits: b29e180ef407; files: extensions/imessage/src/monitor/echo-cache.ts, extensions/imessage/src/monitor/deliver.ts, extensions/imessage/src/monitor/self-chat-dedupe.test.ts)
  • vincentkoc: The PR discussion records a narrow repair pushed to this branch, and merged marker cleanup came through the same echo-cache/test surface in fix(imessage): normalize leading echoed text corruption #73942. (role: recent repair participant and canonical marker-fix author; confidence: high; commits: 780395a4c2ef, 11895810d60b; files: extensions/imessage/src/monitor/echo-cache.ts, extensions/imessage/src/monitor/monitor-provider.echo-cache.test.ts, CHANGELOG.md)
  • omarshahine: Recent iMessage private-API and catchup work added persisted echo-cache integration and tests that a rebase must preserve. (role: recent area contributor; confidence: high; commits: e259751ec9c9, 81e0a1a99bc4, 85ebb4c471a5; files: extensions/imessage/src/monitor/echo-cache.ts, extensions/imessage/src/monitor/persisted-echo-cache.ts, extensions/imessage/src/monitor/monitor-provider.echo-cache.test.ts)
  • obviyus: Recent merged iMessage commits touched the same inbound self-chat and echo matching decision path that depends on echo-cache text matching. (role: adjacent echo/self-chat contributor; confidence: high; commits: 4a5885df3a36, e3e2a19ab7f1, 1ee4a1606e9a; files: extensions/imessage/src/monitor/inbound-processing.ts, extensions/imessage/src/monitor/self-chat-dedupe.test.ts)

Remaining risk / open question:

  • The branch is currently conflicting, so conflict resolution could accidentally drop the landed marker normalizer or persisted sent-echo lookup.
  • The PR body narrates tests and live sends, but there is no inspectable after-fix artifact showing the prefixed iMessage echo being dropped in a real setup.
  • Current main still does not cover the exact U+0093 b U+0002 prefix family, so closing this PR as implemented would leave its central bug unresolved.

Codex review notes: model gpt-5.5, reasoning high; reviewed against f863447fe165.

@vincentkoc
vincentkoc force-pushed the codex/imessage-attributed-echo-prefix branch from 9d69f06 to 780395a Compare April 28, 2026 07:18
@vincentkoc

Copy link
Copy Markdown
Member

ProjectClownfish pushed a narrow repair to this branch so the original contributor path can stay canonical.

Source PR: #66169
Validation: pnpm test extensions/imessage/src/monitor/monitor-provider.echo-cache.test.ts extensions/imessage/src/monitor/self-chat-dedupe.test.ts; pnpm check:changed
Contributor credit is preserved in the branch history and PR context.

@omarshahine

Copy link
Copy Markdown
Contributor

Thanks! Closing as obsolete — covered upstream by #64000 (782b5622b6, strip wrapped imsg-rpc text fields) and the leading-echoed-text normalization in be445dd1c1. echo-cache now sees pre-cleaned text.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: imessage Channel integration: imessage clawsweeper Tracked by ClawSweeper automation size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

iMessage echo detection fails for messages with attributed text prefix bytes

4 participants