Skip to content

Reply: fix generated image delivery to Discord#52489

Merged
scoootscooob merged 1 commit intoopenclaw:mainfrom
scoootscooob:codex/discord-image-reply-fixes
Mar 22, 2026
Merged

Reply: fix generated image delivery to Discord#52489
scoootscooob merged 1 commit intoopenclaw:mainfrom
scoootscooob:codex/discord-image-reply-fixes

Conversation

@scoootscooob
Copy link
Copy Markdown
Contributor

Summary

  • Problem: Discord image requests could succeed at generation time but still respond with text only because the generated media never reached the outbound reply path.
  • Why it matters: Clover could say an image was generated while users received no attachment in Discord DMs or channels.
  • What changed: fixed Discord outbound filename propagation, mapped OpenAI image aspect ratios to supported sizes, and delivered media-bearing block replies even when block streaming is off.
  • What did NOT change (scope boundary): no Discord-only special casing for specific users, channels, or prompts; this stays in shared reply/runtime plumbing.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

  • Closes #
  • Related #

User-visible / Behavior Changes

  • Discord image attachments keep their filename through the send path, improving inline rendering.
  • OpenAI-backed image_generate requests no longer fail when the agent chooses a supported aspect ratio.
  • Generated images attached to tool/media block replies are now delivered even when text block streaming is disabled.

Security Impact (required)

  • New permissions/capabilities? (No)
  • Secrets/tokens handling changed? (No)
  • New/changed network calls? (No)
  • Command/tool execution surface changed? (No)
  • Data access scope changed? (No)
  • If any Yes, explain risk + mitigation:

Repro + Verification

Environment

  • OS: macOS
  • Runtime/container: Node 25 / local gateway
  • Model/provider: anthropic/claude-sonnet-4-6 with openai/gpt-image-1.5
  • Integration/channel (if any): Discord DM and guild channel
  • Relevant config (redacted): local OpenClaw gateway with Discord + OpenAI image generation configured

Steps

  1. Send Clover a Discord prompt asking it to generate an image.
  2. Observe image_generate succeed in the session transcript with details.media.mediaUrls populated.
  3. Observe only a text reply arrive in Discord.

Expected

  • The generated image is delivered to Discord along with the assistant reply flow.

Actual

  • The image was generated locally but dropped before outbound delivery; only text reached Discord.

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios:
    • direct OpenAI Images API generation succeeded with the configured local provider key
    • Discord outbound DM send from the local Clover instance succeeded
    • session transcripts showed image_generate succeeded and stored a PNG path under details.media.mediaUrls
    • after the patch, focused reply-delivery and dispatch tests passed
  • Edge cases checked:
    • OpenAI aspect-ratio handling maps to supported sizes instead of failing
    • media-only tool results still forward when tool summaries are suppressed
    • text-only block replies remain buffered when block streaming is disabled
  • What you did not verify:
    • a final post-patch Discord roundtrip screenshot after this commit

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

If a bot review conversation is addressed by this PR, resolve that conversation yourself. Do not leave bot review conversation cleanup for maintainers.

Compatibility / Migration

  • Backward compatible? (Yes)
  • Config/env changes? (No)
  • Migration needed? (No)
  • If yes, exact upgrade steps:

Failure Recovery (if this breaks)

  • How to disable/revert this change quickly: revert commit 1838119db6778022b0e06ae480f1fae56bd41808
  • Files/config to restore: shared reply delivery and OpenAI image provider paths in this PR
  • Known bad symptoms reviewers should watch for: duplicate media sends, missing final text after media-only block replies, or unexpected OpenAI size selection

Risks and Mitigations

  • Risk: media-bearing block replies could send earlier than some channels expect when block streaming is disabled.
    • Mitigation: the new path only triggers for payloads that already contain media; text-only block replies still follow the prior final-reply accumulation path.
  • Risk: OpenAI aspect-ratio mapping could choose an unexpected size.
    • Mitigation: explicit size still wins, and coverage was added for aspect-ratio mapping behavior.

Notes

  • pnpm build passed.
  • Focused tests passed:
    • pnpm test -- src/auto-reply/reply/reply-delivery.test.ts
    • pnpm test -- src/auto-reply/reply/dispatch-from-config.test.ts -t "suppresses group tool summaries but still forwards tool media"
    • earlier in this branch, focused Discord send tests and OpenAI image-provider tests also passed.
  • Full hook-driven pnpm check is currently blocked by unrelated existing TypeScript failures in untouched files:
    • src/cron/service.issue-regressions.test-helpers.ts
    • src/cron/service.issue-regressions.test.ts

@aisle-research-bot
Copy link
Copy Markdown

aisle-research-bot bot commented Mar 22, 2026

🔒 Aisle Security Analysis

We found 1 potential security issue(s) in this PR:

# Severity Title
1 🔵 Low OpenAI image provider accepts arbitrary size strings without validation

1. 🔵 OpenAI image provider accepts arbitrary size strings without validation

Property Value
Severity Low
CWE CWE-20
Location src/image-generation/providers/openai.ts:34-38

Description

The OpenAI image-generation provider returns a caller-supplied size string directly (only trimmed) and forwards it to the OpenAI API request body.

  • req.size can originate from higher-level runtime/tool inputs.
  • The provider advertises OPENAI_SUPPORTED_SIZES in capabilities metadata, but does not enforce that the runtime request uses one of these allowed presets.
  • This can lead to:
    • unexpected provider behavior / API errors (reliability impact)
    • bypass of intended geometry/cost controls if upstream validation is skipped or another caller uses the runtime directly
    • potential resource issues if an extremely large size string is provided (oversized request/log payloads)

Vulnerable code:

const explicitSize = params.size?.trim();
if (explicitSize) {
  return explicitSize;
}

Recommendation

Enforce provider geometry server-side before sending requests to OpenAI.

Option A (strict): reject unsupported sizes.

const OPENAI_SUPPORTED_SIZES = ["1024x1024", "1024x1536", "1536x1024"] as const;
const OPENAI_SUPPORTED_SIZE_SET = new Set<string>(OPENAI_SUPPORTED_SIZES);

function resolveOpenAISize(params: { size?: string; aspectRatio?: string }): string {
  const explicitSize = params.size?.trim();
  if (explicitSize) {
    if (!OPENAI_SUPPORTED_SIZE_SET.has(explicitSize)) {
      throw new Error(
        `OpenAI image size must be one of ${OPENAI_SUPPORTED_SIZES.join(", ")}`,
      );
    }
    return explicitSize;
  }// existing aspectRatio mapping...
}

Option B (lenient): if unsupported, ignore and fall back to aspectRatio/default (but consider logging a warning).

Also consider adding a reasonable max length check (e.g. explicitSize.length <= 32) to prevent oversized payloads.


Analyzed PR: #52489 at commit 604a0f5

Last updated on: 2026-03-22T23:28:41Z

@openclaw-barnacle openclaw-barnacle bot added channel: discord Channel integration: discord agents Agent runtime and tooling size: M maintainer Maintainer-authored PR labels Mar 22, 2026
@scoootscooob scoootscooob self-assigned this Mar 22, 2026
@scoootscooob scoootscooob force-pushed the codex/discord-image-reply-fixes branch from 1838119 to 604a0f5 Compare March 22, 2026 22:17
@scoootscooob scoootscooob merged commit 24032dc into openclaw:main Mar 22, 2026
14 checks passed
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 22, 2026

Greptile Summary

This PR fixes a three-part bug chain that caused Discord image attachments to be silently dropped: the filename field was never forwarded through the Discord send path, the OpenAI image provider rejected valid aspect-ratio requests, and media-bearing block replies were swallowed when block streaming was disabled. All three fixes are focused and well-scoped.

Key changes

  • send.shared.ts / send.outbound.ts / runtime.messaging.ts: threads an optional filename parameter end-to-end through the Discord send path, with a sensible fallback chain (caller name → media filename → MIME-derived name → "upload").
  • openai.ts: introduces resolveOpenAISize to map aspect-ratio strings onto OpenAI's three available size presets, with explicit size taking precedence; updates supportsAspectRatio to true and populates geometry.aspectRatios.
  • reply-delivery.ts: adds an else if (blockHasMedia) branch so media-bearing blocks are delivered immediately (with text stripped) when block streaming is disabled, letting text arrive via the normal final-reply path instead of being lost.

Minor observations

  • The new extensionForMime import in send.shared.ts is a separate statement rather than being merged with the two existing openclaw/plugin-sdk/media-runtime imports.
  • OPENAI_SUPPORTED_ASPECT_RATIOS includes ratios like "21:9" and "4:3" that are approximate matches to the three available size presets; a clarifying comment would help future readers understand this is best-effort mapping.

Confidence Score: 4/5

  • Safe to merge with at most one minor follow-up (consolidating the split import). The core delivery fix is well-reasoned, tested, and scoped.
  • All three bug-fix paths are coherent and covered by focused tests. The else if (blockHasMedia) design in reply-delivery.ts is intentional and internally consistent with how runResult.payloads propagates media. The only outstanding items are non-blocking style suggestions (split import, aspect-ratio approximation comment).
  • extensions/discord/src/send.shared.ts — split import from openclaw/plugin-sdk/media-runtime should be merged. src/image-generation/providers/openai.ts — aspect-ratio list could benefit from a comment explaining best-effort size mapping.

Comments Outside Diff (1)

  1. extensions/discord/src/send.shared.ts, line 14-20 (link)

    P2 Duplicate import from same module

    The new extensionForMime import was added as a separate statement instead of being merged with the two existing imports from "openclaw/plugin-sdk/media-runtime". All three should be consolidated into one import.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: extensions/discord/src/send.shared.ts
    Line: 14-20
    
    Comment:
    **Duplicate import from same module**
    
    The new `extensionForMime` import was added as a separate statement instead of being merged with the two existing imports from `"openclaw/plugin-sdk/media-runtime"`. All three should be consolidated into one import.
    
    
    
    How can I resolve this? If you propose a fix, please make it concise.

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Prompt To Fix All With AI
This is a comment left during a code review.
Path: extensions/discord/src/send.shared.ts
Line: 14-20

Comment:
**Duplicate import from same module**

The new `extensionForMime` import was added as a separate statement instead of being merged with the two existing imports from `"openclaw/plugin-sdk/media-runtime"`. All three should be consolidated into one import.

```suggestion
import {
  buildOutboundMediaLoadOptions,
  extensionForMime,
  normalizePollDurationHours,
  normalizePollInput,
  type PollInput,
} from "openclaw/plugin-sdk/media-runtime";
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/image-generation/providers/openai.ts
Line: 9-20

Comment:
**Aspect-ratio list claims coverage OpenAI doesn't support natively**

`OPENAI_SUPPORTED_ASPECT_RATIOS` is advertised to callers (via `capabilities.geometry.aspectRatios`) as ratios the provider "supports," but several entries are pure approximations. For example, `"21:9"` (~2.33:1) maps to `1536x1024` (~1.5:1), and `"4:3"` maps to the same `1536x1024` (3:2). Callers that trust the list to contain exact matches may be surprised.

A comment on the constant (or in `resolveOpenAISize`) explaining that these are best-effort mappings to the three available size presets would prevent future confusion:

```typescript
// These are best-effort mappings to OpenAI's three supported size presets
// (1024x1024, 1024x1536, 1536x1024). Ratios like "21:9" or "4:3" are
// approximated to the nearest available size.
const OPENAI_SUPPORTED_ASPECT_RATIOS = [
```

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "Reply: fix generated image delivery to D..." | Re-trigger Greptile

Comment on lines +9 to +20
const OPENAI_SUPPORTED_ASPECT_RATIOS = [
"1:1",
"2:3",
"3:2",
"3:4",
"4:3",
"4:5",
"5:4",
"9:16",
"16:9",
"21:9",
] as const;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Aspect-ratio list claims coverage OpenAI doesn't support natively

OPENAI_SUPPORTED_ASPECT_RATIOS is advertised to callers (via capabilities.geometry.aspectRatios) as ratios the provider "supports," but several entries are pure approximations. For example, "21:9" (~2.33:1) maps to 1536x1024 (~1.5:1), and "4:3" maps to the same 1536x1024 (3:2). Callers that trust the list to contain exact matches may be surprised.

A comment on the constant (or in resolveOpenAISize) explaining that these are best-effort mappings to the three available size presets would prevent future confusion:

// These are best-effort mappings to OpenAI's three supported size presets
// (1024x1024, 1024x1536, 1536x1024). Ratios like "21:9" or "4:3" are
// approximated to the nearest available size.
const OPENAI_SUPPORTED_ASPECT_RATIOS = [
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/image-generation/providers/openai.ts
Line: 9-20

Comment:
**Aspect-ratio list claims coverage OpenAI doesn't support natively**

`OPENAI_SUPPORTED_ASPECT_RATIOS` is advertised to callers (via `capabilities.geometry.aspectRatios`) as ratios the provider "supports," but several entries are pure approximations. For example, `"21:9"` (~2.33:1) maps to `1536x1024` (~1.5:1), and `"4:3"` maps to the same `1536x1024` (3:2). Callers that trust the list to contain exact matches may be surprised.

A comment on the constant (or in `resolveOpenAISize`) explaining that these are best-effort mappings to the three available size presets would prevent future confusion:

```typescript
// These are best-effort mappings to OpenAI's three supported size presets
// (1024x1024, 1024x1536, 1536x1024). Ratios like "21:9" or "4:3" are
// approximated to the nearest available size.
const OPENAI_SUPPORTED_ASPECT_RATIOS = [
```

How can I resolve this? If you propose a fix, please make it concise.

jacobtomlinson pushed a commit to jacobtomlinson/openclaw that referenced this pull request Mar 22, 2026
frankekn pushed a commit to artwalker/openclaw that referenced this pull request Mar 23, 2026
furaul pushed a commit to furaul/openclaw that referenced this pull request Mar 24, 2026
alexey-pelykh pushed a commit to remoteclaw/remoteclaw that referenced this pull request Mar 28, 2026
(cherry picked from commit 24032dc)

# Conflicts:
#	extensions/discord/src/send.sends-basic-channel-messages.test.ts
#	extensions/discord/src/send.shared.ts
#	src/agents/tools/discord-actions.test.ts
#	src/agents/tools/image-generate-tool.test.ts
#	src/image-generation/providers/openai.test.ts
#	src/image-generation/providers/openai.ts
alexey-pelykh pushed a commit to remoteclaw/remoteclaw that referenced this pull request Mar 28, 2026
(cherry picked from commit 24032dc)

# Conflicts:
#	src/agents/tools/image-generate-tool.test.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling channel: discord Channel integration: discord maintainer Maintainer-authored PR size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant