Skip to content

fix(imessage): strip length-prefixed UTF-8 from imsg rpc text#64000

Merged
obviyus merged 2 commits into
openclaw:mainfrom
neeravmakwana:fix/imessage-imsg-text-prefix
Apr 10, 2026
Merged

fix(imessage): strip length-prefixed UTF-8 from imsg rpc text#64000
obviyus merged 2 commits into
openclaw:mainfrom
neeravmakwana:fix/imessage-imsg-text-prefix

Conversation

@neeravmakwana

Copy link
Copy Markdown
Contributor

Summary

  • Problem: Inbound iMessage notifications from imsg rpc sometimes include a protobuf-style length-delimited UTF-8 blob inside the JSON text (and reply_to_text) string. Those bytes were passed through to the agent as literal prefix garbage.
  • Why it matters: Wastes tokens, breaks tools expecting clean text, and matches reports in [Bug]: iMessage (imsg) channel - binary length prefix leaking into message content #63868.
  • What changed: After validating the RPC payload, strip a leading varint length plus exactly that many following UTF-8 bytes only when they consume the entire field; otherwise leave the string unchanged. Applied to text and reply_to_text.
  • What did NOT change: JSON-RPC transport, outbound send path, and behavior when the field is already plain text.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

Root Cause (if applicable)

  • Root cause: Upstream imsg may embed a length-delimited UTF-8 payload inside the JSON string; OpenClaw previously forwarded that field with only trim().
  • Missing detection / guardrail: No normalization of text / reply_to_text after parse.
  • Contributing context (if known): Conservative strip only when varint + payload exactly matches full UTF-8 byte length of the string.

Regression Test Plan (if applicable)

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file: extensions/imessage/src/monitor/strip-imsg-length-prefixed-text.test.ts, extensions/imessage/src/monitor/parse-notification.test.ts
  • Scenario the test should lock in: Length-prefixed wrapper removed; plain text and non-matching layouts preserved.
  • Why this is the smallest reliable guardrail: Pure string transformation with no I/O.
  • Existing test that already covers this (if any): N/A
  • If no new test is added, why not: N/A — tests added.

User-visible / Behavior Changes

Inbound iMessage text and quoted reply text may no longer show a short binary or mojibake prefix when the imsg bridge included a length-delimited UTF-8 wrapper inside the JSON field.

Diagram (if applicable)

N/A

Security Impact (required)

  • New permissions/capabilities? (No)
  • New network endpoints or listeners? (No)
  • Touches auth/secrets/pairing? (No)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 01abbf1074

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread extensions/imessage/src/monitor/strip-imsg-length-prefixed-text.ts Outdated
@greptile-apps

greptile-apps Bot commented Apr 10, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a new stripImessageLengthPrefixedUtf8Text helper that removes a protobuf-style varint length header from inbound iMessage text and reply_to_text fields when the header + payload exactly exhausts the buffer, and wires it into parseIMessageNotification after type validation. The conservative exact-match guard makes false positives on normal text practically impossible.

Confidence Score: 5/5

Safe to merge — logic is correct and conservative; the only finding is a misleading test description with missing two-byte varint coverage.

All findings are P2. The implementation correctly handles the stated goal, the exact-length guard prevents false positives, and Unicode edge cases are safe. The test labeled "multi-byte varint" actually uses a single-byte varint (0x7F), leaving the two-byte decoding path unverified, but the production code logic is sound.

extensions/imessage/src/monitor/strip-imsg-length-prefixed-text.test.ts — misleading test name and missing two-byte varint case.

Prompt To Fix All With AI
This is a comment left during a code review.
Path: extensions/imessage/src/monitor/strip-imsg-length-prefixed-text.test.ts
Line: 10-15

Comment:
**Misleading test name — single-byte varint, not multi-byte**

`0x7F` has bit 7 clear, so it is the terminal byte of a **one-byte** varint with value 127, not a multi-byte varint. A genuinely two-byte varint (e.g. `0x80 0x01` for value 128) is not exercised here, leaving the multi-byte decoding path untested. Renaming the case and adding a companion case with a real two-byte prefix would lock in that path.

```suggestion
  it("removes a 127-byte payload behind a single-byte varint length", () => {
    const inner = "a".repeat(127);
    const buf = Buffer.allocUnsafe(1 + inner.length);
    buf.writeUInt8(0x7f, 0);
    buf.write(inner, 1, "utf8");
    expect(stripImessageLengthPrefixedUtf8Text(buf.toString("utf8"))).toBe(inner);
  });

  it("removes a 128-byte payload behind a two-byte varint length", () => {
    const inner = "a".repeat(128);
    // Varint for 128: 0x80 0x01
    const buf = Buffer.allocUnsafe(2 + inner.length);
    buf.writeUInt8(0x80, 0);
    buf.writeUInt8(0x01, 1);
    buf.write(inner, 2, "utf8");
    expect(stripImessageLengthPrefixedUtf8Text(buf.toString("utf8"))).toBe(inner);
  });
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "fix(imessage): strip length-prefixed UTF..." | Re-trigger Greptile

Comment thread extensions/imessage/src/monitor/strip-imsg-length-prefixed-text.test.ts Outdated
@neeravmakwana

Copy link
Copy Markdown
Contributor Author

Review feedback addressed (latest: 61f313ccd1)

chatgpt-codex-connector (P1) — false positive strip when one ASCII byte matches framing

  • Added a guard: if the length is encoded as a single-byte varint and that byte is in the ASCII printable range 0x20–0x7e, we do not strip (return original text). That blocks the reported case where the first character (e.g. A / 0x41) could be misread as length 65 for a 66-byte UTF-8 buffer.
  • Multibyte varints (continuation bytes) are unchanged: first varint byte has bit 7 set, so they never fall in the printable ASCII range and still strip when the payload consumes the full remainder.
  • Control / non-printable single-byte lengths (e.g. 0x05, 0x0b, 0x7f) still strip as before.

greptile-apps — test naming and two-byte varint coverage

  • Renamed the 127-byte case to state it uses a single-byte varint (0x7f).
  • Added a two-byte varint case for length 128 (0x80 0x01) using raw Buffer input via the new tryStripImessageLengthPrefixedUtf8Buffer helper. Raw bytes like 0x80 0x01 are not round-trippable through JS strings plus Buffer.from(text, "utf8"), so the buffer-level API documents that limitation and keeps the multibyte decode path covered in tests.
  • Added a regression test that a string starting with A + 65× b (66 UTF-8 bytes) is left unchanged.

Changelog

  • Unreleased entry updated to mention the ASCII-printable skip.

pnpm test:extension imessage and pnpm check are green on this branch.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1eac513d25

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread extensions/imessage/src/monitor/strip-imsg-length-prefixed-text.ts Outdated
@obviyus obviyus self-assigned this Apr 10, 2026
@obviyus
obviyus force-pushed the fix/imessage-imsg-text-prefix branch from 1eac513 to 6331065 Compare April 10, 2026 08:26
@aisle-research-bot

aisle-research-bot Bot commented Apr 10, 2026

Copy link
Copy Markdown

🔒 Aisle Security Analysis

We found 1 potential security issue(s) in this PR:

# Severity Title
1 🟡 Medium Unbounded UTF-8 Buffer conversion/decoding of inbound iMessage text can cause memory/CPU DoS
1. 🟡 Unbounded UTF-8 Buffer conversion/decoding of inbound iMessage text can cause memory/CPU DoS
Property Value
Severity Medium
CWE CWE-400
Location extensions/imessage/src/monitor/strip-imsg-length-prefixed-text.ts:48-59

Description

stripImessageLengthPrefixedUtf8Text() is applied to every inbound notification’s text and reply_to_text. It converts the entire untrusted string into a Node Buffer and then decodes bytes back to a string.

  • Input: message.text / message.reply_to_text from inbound iMessage RPC notifications (attacker-controlled by sending large messages)
  • Amplification: Buffer.from(text, "utf8") allocates/copies the entire string as bytes; TextDecoder.decode() then walks/decodes the resulting Uint8Array again.
  • No size limit / backpressure: there is no maximum length/byte check before allocating/decoding. Very large inbound messages can spike memory and CPU and potentially crash the extension process (availability impact).

Vulnerable code:

const stripped = tryStripImessageLengthPrefixedUtf8Buffer(Buffer.from(text, "utf8"));
...
const inner = utf8Decoder.decode(stripped);

Recommendation

Add an explicit size/byte limit (and ideally a try/catch) before converting/decoding untrusted text.

Example:

const MAX_STRIP_BYTES = 64 * 1024; // choose an appropriate bound

export function stripImessageLengthPrefixedUtf8Text(text: string): string {
  if (!text) return text;// Prevent large allocations/CPU on attacker-controlled input
  if (Buffer.byteLength(text, "utf8") > MAX_STRIP_BYTES) {
    return text;
  }

  try {
    const buf = Buffer.from(text, "utf8");
    const stripped = tryStripImessageLengthPrefixedUtf8Buffer(buf);
    if (!stripped) return text;

    const inner = utf8Decoder.decode(stripped);
    return inner.length > 0 ? inner : text;
  } catch {
    return text; // fail open; preserve original text
  }
}

If feasible, further reduce work by only examining a small prefix to decide whether stripping is needed before allocating/decoding the full payload.


Analyzed PR: #64000 at commit 4ba523d

Last updated on: 2026-04-10T08:36:08Z

@obviyus
obviyus force-pushed the fix/imessage-imsg-text-prefix branch from 6331065 to 4ba523d Compare April 10, 2026 08:29

@obviyus obviyus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the imessage inbound text corruption path and confirmed the fix now strips only the expected length-delimited wrapper shape before parseIMessageNotification forwards text and reply_to_text.

Maintainer follow-up: rebased onto latest main, replaced the printable-length heuristic with an explicit field-wrapper discriminator, and added regressions for ASCII-printable lengths plus non-wrapper plain text.

Local gate: pnpm test extensions/imessage/src/monitor.gating.test.ts extensions/imessage/src/monitor/strip-imsg-length-prefixed-text.test.ts extensions/imessage/src/monitor/parse-notification.test.ts and pnpm tsgo.

@obviyus
obviyus merged commit 782b562 into openclaw:main Apr 10, 2026
8 checks passed
@obviyus

obviyus commented Apr 10, 2026

Copy link
Copy Markdown
Contributor

Landed on main.

Thanks @neeravmakwana.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4ba523dc60

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +32 to +36
if (key.value !== 0x0a) {
return null;
}

const length = readVarint(buf, key.nextOffset);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Guard newline-prefixed plain text from wrapper stripping

Fresh evidence on this commit: "\na" + "x".repeat(97) is treated as a wrapped payload because 0x0a is both the protobuf field-1 tag and a literal newline byte, so the function drops the first two characters and returns only the tail. Since parseIMessageNotification now applies this helper to every inbound text and reply_to_text, legitimate user messages that start with a newline can be silently corrupted when the next byte happens to equal the remaining UTF-8 length; add a stronger discriminator before stripping.

Useful? React with 👍 / 👎.

steipete pushed a commit that referenced this pull request Apr 10, 2026
* fix(imessage): strip length-prefixed UTF-8 from imsg rpc text

* fix: strip wrapped imsg rpc text fields (#64000) (thanks @neeravmakwana)

---------

Co-authored-by: Ayaan Zaidi <[email protected]>
lovewanwan pushed a commit to lovewanwan/openclaw that referenced this pull request Apr 28, 2026
…ravmakwana)

* fix(imessage): strip length-prefixed UTF-8 from imsg rpc text

* fix: strip wrapped imsg rpc text fields (openclaw#64000) (thanks @neeravmakwana)

---------

Co-authored-by: Ayaan Zaidi <[email protected]>
ogt-redknie pushed a commit to ogt-redknie/OPENX that referenced this pull request May 2, 2026
…ravmakwana)

* fix(imessage): strip length-prefixed UTF-8 from imsg rpc text

* fix: strip wrapped imsg rpc text fields (openclaw#64000) (thanks @neeravmakwana)

---------

Co-authored-by: Ayaan Zaidi <[email protected]>
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 9, 2026
…ravmakwana)

* fix(imessage): strip length-prefixed UTF-8 from imsg rpc text

* fix: strip wrapped imsg rpc text fields (openclaw#64000) (thanks @neeravmakwana)

---------

Co-authored-by: Ayaan Zaidi <[email protected]>
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 24, 2026
…ravmakwana)

* fix(imessage): strip length-prefixed UTF-8 from imsg rpc text

* fix: strip wrapped imsg rpc text fields (openclaw#64000) (thanks @neeravmakwana)

---------

Co-authored-by: Ayaan Zaidi <[email protected]>
jameslcowan pushed a commit to jameslcowan/openclaw that referenced this pull request Jun 2, 2026
…ravmakwana)

* fix(imessage): strip length-prefixed UTF-8 from imsg rpc text

* fix: strip wrapped imsg rpc text fields (openclaw#64000) (thanks @neeravmakwana)

---------

Co-authored-by: Ayaan Zaidi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: imessage Channel integration: imessage size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: iMessage (imsg) channel - binary length prefix leaking into message content

2 participants