fix(pair): render /pair qr as media by BunsDev · Pull Request #70047 · openclaw/openclaw

BunsDev · 2026-04-22T07:33:16Z

Fixes /pair qr on internal chat surfaces where the QR was showing up as raw code/text instead of an actual scannable image ✨

The bug was not in QR generation itself. The device-pair command was returning the QR for webchat as markdown text with an inline data:image/... payload, and the gateway chat path only had special handling for audio media. That meant image-bearing replies could degrade into text-oriented output, so Control UI and related internal surfaces showed raw code-like content instead of a rendered QR image.

This change moves the pair QR flow onto the normal media path:

/pair qr now returns structured media for the generated QR instead of embedding it in markdown text
the gateway webchat/internal chat path now converts image-bearing assistant replies into real image content blocks
media-only final replies no longer collapse into MEDIA:... text for internal chat rendering
the user-facing pairing instructions remain as text alongside the rendered QR

Tests added/updated:

device-pair regression test for /pair qr returning structured media on webchat
gateway webchat media test for converting image data URLs into assistant image blocks
gateway non-streaming chat regression proving image replies render as image content instead of raw MEDIA: text

Validation:
node scripts/test-projects.mjs extensions/device-pair/index.test.ts src/gateway/server-methods/chat-webchat-media.test.ts src/gateway/server-methods/chat.directive-tags.test.ts

greptile-apps · 2026-04-22T07:37:02Z

Greptile Summary

This PR routes /pair qr through the structured media path so the QR image renders as a real image block on internal chat surfaces instead of raw markdown text. The gateway non-streaming path is updated to call the new buildWebchatAssistantMessageFromReplyPayloads helper, which handles both audio and image media, and tests are added/updated accordingly.

NO_REPLY leaks into content blocks: buildWebchatAssistantMessageFromReplyPayloads includes payload.text without the isSuppressedControlReplyText guard that buildTranscriptReplyText applies. A payload with text: \"NO_REPLY\" and a mediaUrl produces { type: \"text\", text: \"NO_REPLY\" } in the content array instead of the synthetic \"Image reply\" fallback — breaking the test at chat.directive-tags.test.ts:1914.
Reply-threading context dropped: replyToId / replyToCurrent directives are encoded by buildTranscriptReplyText but ignored in the new helper, so transcript message strings for media-bearing replies lose thread context.

Confidence Score: 4/5

Not safe to merge until the NO_REPLY / suppressed-token filtering is added and the replyToId regression is addressed.

Two P1 defects: the missing isSuppressedControlReplyText guard will cause visible NO_REPLY text in image-bearing messages and a test regression, and replyToId threading is silently dropped for all media replies. The core QR rendering fix is correct and well-structured, but these two gaps need to be resolved first.

src/gateway/server-methods/chat-webchat-media.ts (buildWebchatAssistantMessageFromReplyPayloads — missing suppressed-text filter and reply-threading directives)

Prompt To Fix All With AI

This is a comment left during a code review.
Path: src/gateway/server-methods/chat-webchat-media.ts
Line: 150-155

Comment:
**`NO_REPLY` / suppressed control tokens leak into content blocks**

`buildTranscriptReplyText` guards `payload.text` with `isSuppressedControlReplyText(text)` before including it, but this function does not. As a result, when a media-bearing payload carries `text: "NO_REPLY"` (or `ANNOUNCE_SKIP` / `REPLY_SKIP`), the trimmed token is pushed into both `transcriptTextParts` and `content`, so `transcriptText` becomes `"NO_REPLY"` and the content block is `{ type: "text", text: "NO_REPLY" }` instead of the intended `"Image reply"` synthetic fallback.

The test at line 1914 of `chat.directive-tags.test.ts` sets exactly `{ text: "NO_REPLY", mediaUrl: "..." }` and expects `{ type: "text", text: "Image reply" }` — this assertion will fail with the code as written.

```ts
const text = payload.text?.trim();
if (text && !isSuppressedControlReplyText(text)) {
  transcriptTextParts.push(text);
  content.push({ type: "text", text });
}
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/gateway/server-methods/chat-webchat-media.ts
Line: 139-193

Comment:
**`replyToId` / `replyToCurrent` directives silently dropped for media replies**

`buildTranscriptReplyText` prepends `[[reply_to:...]]` / `[[reply_to_current]]` when `payload.replyToId` or `payload.replyToCurrent` are set. This function ignores those fields entirely, so whenever `mediaMessage` takes precedence over `combinedReply` in `chat.ts`, the threading context is lost from the stored transcript message. Consider mirroring the directive-tag handling here, or falling back to the `combinedReply`-derived message for the transcript `message` string while still using `mediaMessage.content` for the content blocks.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/gateway/server-methods/chat.ts
Line: 2091

Comment:
**`appendedWebchatAgentAudio` variable name is stale**

The variable `appendedWebchatAgentAudio` (declared earlier in scope, unchanged in this PR) is now used to gate both audio and image media appends. The name is misleading and could cause confusion during future work on this guard. Consider renaming it to `appendedWebchatAgentMedia` for consistency with the other identifiers renamed in this PR (`appendWebchatAgentMediaTranscriptIfNeeded`, `assistant-media`).

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (1): Last reviewed commit: "Merge branch 'main' into fix/pair-qr-ren..." | Re-trigger Greptile}

BunsDev · 2026-04-22T07:57:58Z

Resolved in 038118d50e.

A note on scope: the linked Aisle comment currently shows 3 findings as of April 22, 2026, but I hardened the full 5-objection boundary from the earlier review cycle so this path does not come back with a revised variant of the same issue class.

Arbitrary local file read via assistant-controlled mediaUrl
Fixed by requiring explicit trust before the webchat path will read and embed a local file. chat-webchat-media.ts now only resolves local audio when trustedLocalMedia === true, and that flag is only set by known-good TTS producers (commands-tts.ts and tts-tool.ts).
Proof:

src/gateway/server-methods/chat-webchat-media.test.ts: rejects untrusted local audio paths
src/agents/tools/tts-tool.test.ts: trusted local media flag is present on TTS output

Pairing QR persisted into transcript / broadcast state
Fixed by marking /pair qr output as sensitiveMedia and changing the gateway transcript append path to persist only the text portion for sensitive media while still sending the live image in the outbound assistant message.
Proof:

extensions/device-pair/index.ts: webchat QR payload now sets sensitiveMedia: true
src/gateway/server-methods/chat.directive-tags.test.ts: does not persist sensitive image media into transcript updates

Unbounded data:image URL accepted into assistant content
Fixed by replacing the permissive image check with a strict data:image/...;base64,... validator that:

only allows safe raster formats
caps total URL size
caps decoded payload size
Proof:
src/gateway/server-methods/chat-webchat-media.ts: MAX_WEBCHAT_IMAGE_DATA_URL_CHARS, MAX_WEBCHAT_IMAGE_DATA_BYTES, raster allowlist
src/gateway/server-methods/chat-webchat-media.test.ts: drops oversized data image URLs

Remote image URLs causing client-side fetch / SSRF / privacy leakage
Fixed by no longer embedding arbitrary remote image URLs at all in this webchat assistant-media path. Only bounded raster data URLs are accepted now.
Proof:

src/gateway/server-methods/chat-webchat-media.test.ts: rejects remote image URLs

Directive-tag injection through unvalidated replyToId
Fixed by adding shared reply-id sanitization and using it in both transcript builders before interpolating [[reply_to:...]]. Unsafe control characters and bracket characters are stripped, and length is capped.
Proof:

src/utils/directive-tags.ts: shared sanitizer
src/gateway/server-methods/chat-webchat-media.test.ts: sanitizes reply ids before embedding directive prefixes
src/gateway/server-methods/chat.directive-tags.test.ts: sanitizes replyToId before emitting inline reply directives

Additional hardening included in the same patch:

data:image/svg+xml is rejected
only trusted tool/media paths can promote local file embedding through the agent tool pipeline

Validation run:

node scripts/test-projects.mjs extensions/device-pair/index.test.ts src/auto-reply/reply/commands-tts.test.ts src/agents/tools/tts-tool.test.ts src/agents/pi-embedded-subscribe.tools.media.test.ts src/agents/pi-embedded-subscribe.handlers.messages.test.ts src/agents/pi-embedded-subscribe.handlers.tools.media.test.ts src/gateway/server-methods/chat-webchat-media.test.ts src/gateway/server-methods/chat.directive-tags.test.ts
node scripts/test-projects.mjs src/utils/directive-tags.test.ts src/agents/pi-embedded-subscribe.handlers.lifecycle.test.ts src/agents/pi-embedded-subscribe.handlers.tools.test.ts

aisle-research-bot · 2026-04-22T08:03:25Z

🔒 Aisle Security Analysis

We found 3 potential security issue(s) in this PR:

#	Severity	Title
1	🟠 High	Audit log bypass via untrusted `sensitiveMedia` flag suppressing transcript persistence
2	🟠 High	Internal-only trustedLocalMedia flag forwarded to outbound channel plugins via deliverOutboundPayloads
3	🟡 Medium	Potential client-side DoS via decompression-bomb image data URLs in webchat assistant messages

1. 🟠 Audit log bypass via untrusted `sensitiveMedia` flag suppressing transcript persistence

Property	Value
Severity	High
CWE	CWE-778
Location	`src/gateway/server-methods/chat.ts:217-240`

Description

The gateway uses payload.sensitiveMedia === true to suppress persisting media references/blocks into the transcript history.

Because sensitiveMedia is part of the public plugin SDK normalization surface, an untrusted/third‑party plugin (or tool emitting a loose payload object) can set sensitiveMedia: true on any media reply and cause the gateway to:

omit MEDIA: lines from the transcript text entirely
omit rich content blocks (e.g., embedded images/audio) from persisted transcript entries

This creates an audit/evidence gap: users still receive the media, but transcript storage does not retain what was sent.

Vulnerable behavior:

// buildTranscriptReplyText(): drops media URLs from persisted transcript text
for (const mediaUrl of parts.mediaUrls) {
  if (payload.sensitiveMedia === true) {
    continue;
  }
  lines.push(`MEDIA:${trimmed}`);
}

// appendAssistantTranscriptMessage(): drops persisted `content`
...(payload.sensitiveMedia === true ? {} : { content: mediaMessage.content })

Additionally, the plugin SDK explicitly preserves sensitiveMedia during normalization:

// plugin-sdk/reply-payload.ts
const sensitiveMedia = payload.sensitiveMedia === true ? true : undefined;

so callers outside core can trigger the suppression.

Recommendation

Treat sensitiveMedia as a privileged/internal-only flag rather than a user/plugin-controlled field.

Recommended fixes (pick one):

Remove sensitiveMedia from the public plugin SDK surface and strip it during normalization, similar to trustedLocalMedia:

// src/plugin-sdk/reply-payload.ts
export type ReplyPayload = Omit<InternalReplyPayload, "trustedLocalMedia" | "sensitiveMedia">;

export function normalizeOutboundReplyPayload(payload: Record<string, unknown>): OutboundReplyPayload {
  // ...do NOT forward sensitiveMedia from untrusted payloads
  return { text, mediaUrls, mediaUrl, replyToId };
}

Gate it behind an explicit trust/capability check (e.g., only allow when the reply originates from core-owned extensions, signed plugins, or when a server-side policy allows it). Enforce this in the gateway before applying suppression:

const allowSensitive = isTrustedOrigin(ctx) && payload.sensitiveMedia === true;
if (!allowSensitive) payload = { ...payload, sensitiveMedia: undefined };

If you must keep the feature for privacy (e.g., QR codes), persist an auditable placeholder (hash/metadata) so transcripts reflect that sensitive media was sent, without storing the raw media.

This prevents malicious plugins from hiding delivered content from audit logs while preserving the intended privacy behavior for approved internal flows.

2. 🟠 Internal-only trustedLocalMedia flag forwarded to outbound channel plugins via deliverOutboundPayloads

Property	Value
Severity	High
CWE	CWE-201
Location	`src/infra/outbound/deliver.ts:834-854`

Description

Outbound delivery forwards the full internal ReplyPayload object (which may include the internal-only trustedLocalMedia flag) directly into the channel plugin adapter sendPayload().

ReplyPayload.trustedLocalMedia is intended as an internal trust signal for allowing local filesystem paths to be embedded/loaded by specific internal transports (e.g., gateway webchat).
In deliverOutboundPayloads, the plugin handler is invoked with handler.sendPayload(effectivePayload, ...) without stripping trustedLocalMedia.
Even if the plugin SDK TypeScript types omit trustedLocalMedia, the property still exists at runtime and can be accessed by adapters via any/unknown plumbing or property indexing.
A malicious or compromised outbound channel plugin could treat trustedLocalMedia as permission to read and exfiltrate local files (e.g., sending /etc/passwd or other sensitive local paths as media).

Vulnerable code (payload forwarded without boundary normalization):

const effectivePayload = await renderPresentationForDelivery(handler, hookResult.payload);
...
const delivery = await handler.sendPayload(effectivePayload, sendOverrides);

Recommendation

Enforce a runtime outbound plugin boundary before calling any outbound adapter methods:

Strip internal-only properties (at minimum trustedLocalMedia) from the payload object passed to plugin adapters.
Prefer an allowlist-based projection (similar to normalizeOutboundReplyPayload) rather than deleting keys.
Apply the normalization consistently for all adapter entry points (sendPayload, sendMedia, sendText, formatted senders, approval flows, etc.).

Example (conceptual):

import { normalizeOutboundReplyPayload } from "../../plugin-sdk/reply-payload";

// before calling plugin adapter
const outbound = normalizeOutboundReplyPayload(effectivePayload as Record<string, unknown>);
await handler.sendPayload(outbound, sendOverrides);

If internal delivery paths need trustedLocalMedia, keep that flag only within trusted internal transports and never pass it into externally-supplied plugin code.

3. 🟡 Potential client-side DoS via decompression-bomb image data URLs in webchat assistant messages

Property	Value
Severity	Medium
CWE	CWE-409
Location	`src/gateway/server-methods/chat-webchat-media.ts:121-142`

Description

resolveEmbeddableImageUrl() allows data:image/*;base64,... URLs to be forwarded into webchat assistant message content as { type: "input_image", image_url: ... }.

Although the code caps encoded size (1.5MB decoded base64 estimate), it does not validate image dimensions/pixel count before the web client decodes/renders it. Many image formats (notably GIF, BMP, and even PNG) can be crafted as decompression bombs where a relatively small encoded payload expands into extremely large pixel buffers when decoded, causing:

high memory consumption in the browser/webview
UI freezes / tab crashes

This is a security issue when an attacker can cause such a payload to be delivered to other users via chat/websocket (availability impact).

Vulnerable code:

const match = /^data:(image\/[a-z0-9.+-]+);base64,([a-z0-9+/=\s]+)$/i.exec(trimmed);
...
if (!ALLOWED_WEBCHAT_DATA_IMAGE_MEDIA_TYPES.has(mediaType)) {
  return null;
}
if (estimateBase64DecodedBytes(base64Data) > MAX_WEBCHAT_IMAGE_DATA_BYTES) {
  return null;
}
return trimmed;

Recommendation

Validate images server-side before forwarding to the webchat client.

Options (prefer the first):

Decode and inspect dimensions / pixel count (and reject animations / large frames) using a safe image parser (e.g., sharp, image-size, or equivalent) and enforce a strict maximum, e.g. total pixels <= 4–16MP.
Re-encode the image server-side into a safe, bounded format (e.g., JPEG/PNG) after decoding, stripping animation and metadata.
Tighten the allowlist (e.g., consider dropping image/bmp and image/gif if not required).

Example sketch (dimension check):

import sharp from "sharp";

async function validateImage(base64Data: string) {
  const buf = Buffer.from(base64Data.replace(/\s+/g, ""), "base64");
  const meta = await sharp(buf, { animated: false, limitInputPixels: 16_000_000 }).metadata();
  if (!meta.width || !meta.height) throw new Error("invalid image");
  if (meta.width * meta.height > 16_000_000) throw new Error("image too large");
}

This prevents small encoded payloads from causing large decode-time allocations in the client.

Analyzed PR: #70047 at commit 007c511

_{Last updated on: 2026-04-22T08:34:22Z}

BunsDev · 2026-04-22T08:20:25Z

Explicit resolution for #70047 (comment):

Fixed in ad7cc1a0ab.

trustedLocalMedia is no longer part of the public OutboundReplyPayload surface.
Public plugin SDK ReplyPayload now omits trustedLocalMedia.
normalizeOutboundReplyPayload(...) strips any caller-supplied trustedLocalMedia bit, so loose/external payload objects cannot promote local file embedding.
The trust bit remains internal-only on core reply payloads, and the internal TTS producers still set it where the gateway needs it.

Validation:

pnpm test src/plugin-sdk/reply-payload.test.ts src/gateway/server-methods/chat-webchat-media.test.ts src/agents/tools/tts-tool.test.ts
pnpm plugin-sdk:api:check

I also ran pnpm check:changed. The remaining failure is an unrelated extension typecheck in unchanged extensions/telegram/src/bot.ts (AbortSignal mismatch; the failing lines match origin/main), so it is not introduced by this fix.

Updated changelog to include recent fixes and enhancements.

* fix(pair): render pair qr as media * fix(gateway): preserve media reply threading * fix(gateway): harden webchat media replies * fix(plugin-sdk): keep trustedLocalMedia internal * docs(changelog): note pair qr media fix * Update CHANGELOG with recent fixes and enhancements Updated changelog to include recent fixes and enhancements.

BunsDev and others added 2 commits April 22, 2026 02:30

fix(pair): render pair qr as media

4eab411

Merge branch 'main' into fix/pair-qr-render

f4ae985

openclaw-barnacle Bot added app: web-ui App: web-ui gateway Gateway runtime extensions: device-pair size: S maintainer Maintainer-authored PR labels Apr 22, 2026

greptile-apps Bot reviewed Apr 22, 2026

View reviewed changes

Comment thread src/gateway/server-methods/chat-webchat-media.ts Outdated

Comment thread src/gateway/server-methods/chat-webchat-media.ts

Comment thread src/gateway/server-methods/chat.ts

fix(gateway): preserve media reply threading

2d727af

openclaw-barnacle Bot added size: M and removed size: S labels Apr 22, 2026

fix(gateway): harden webchat media replies

038118d

openclaw-barnacle Bot added agents Agent runtime and tooling size: L and removed size: M labels Apr 22, 2026

Merge branch 'main' into fix/pair-qr-render

12deeaa

fix(plugin-sdk): keep trustedLocalMedia internal

ad7cc1a

openclaw-barnacle Bot added the docs Improvements or additions to documentation label Apr 22, 2026

BunsDev and others added 3 commits April 22, 2026 03:21

Merge branch 'main' into fix/pair-qr-render

d6b9c47

docs(changelog): note pair qr media fix

007c511

Update CHANGELOG with recent fixes and enhancements

0e9772d

Updated changelog to include recent fixes and enhancements.

BunsDev merged commit 43a941b into main Apr 22, 2026
94 checks passed

BunsDev deleted the fix/pair-qr-render branch April 22, 2026 08:31

github-actions Bot mentioned this pull request Apr 22, 2026

📡 Upstream Digest — 2026-04-22 09:12 UTC curtismercier/openclaw-mods#643

Open

perlowja mentioned this pull request Apr 23, 2026

[Bug] Agent tool-use regressed for all providers between v2026.4.20 and main — 0/6 models invoke tools on a cold-start prompt; multiple model families converge on fabricated "Exec is blocked on this session" phrase #70456

Closed

This was referenced Jun 28, 2026

fix(utils): keep reply directive ids unicode-safe #96938

Merged

fix: /pair qr fails to show QR code in chat surfaces #97933

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(pair): render /pair qr as media#70047

fix(pair): render /pair qr as media#70047
BunsDev merged 9 commits into
mainfrom
fix/pair-qr-render

BunsDev commented Apr 22, 2026

Uh oh!

greptile-apps Bot commented Apr 22, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

BunsDev commented Apr 22, 2026

Uh oh!

aisle-research-bot Bot commented Apr 22, 2026 •

edited

Loading

Description

Recommendation

Description

Recommendation

Description

Recommendation

Uh oh!

BunsDev commented Apr 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

BunsDev commented Apr 22, 2026

Uh oh!

greptile-apps Bot commented Apr 22, 2026

Greptile Summary

Confidence Score: 4/5

Uh oh!

Uh oh!

Uh oh!

Uh oh!

BunsDev commented Apr 22, 2026

Uh oh!

aisle-research-bot Bot commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔒 Aisle Security Analysis

Description

Recommendation

Description

Recommendation

Description

Recommendation

Uh oh!

BunsDev commented Apr 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

aisle-research-bot Bot commented Apr 22, 2026 •

edited

Loading