Skip to content

feat(channels): batch 1 producers drop media placeholder bodies#111447

Merged
steipete merged 2 commits into
mainfrom
claude/media-placeholder-batch1
Jul 19, 2026
Merged

feat(channels): batch 1 producers drop media placeholder bodies#111447
steipete merged 2 commits into
mainfrom
claude/media-placeholder-batch1

Conversation

@steipete

Copy link
Copy Markdown
Contributor

What Problem This Solves

Batch 1 of the media-placeholder elimination program (hard-delete end state, producers migrate before core consumer deletion; PR #111315 was the program's PR1). Four channel plugins — Google Chat, Zalo, LINE, Mattermost — minted <media:kind> placeholder bodies for media-only inbound messages, mixing model/transport scaffolding into user-authored text that core then had to regex-un-parse.

Why This Change Was Made

  • Media-only messages now carry an empty caption as primary/agent/command text plus one structured fact per native attachment. Downloads that fail or are rejected contribute type-only facts (kind/contentType, no path), so payload positions, counts, and kind signals stay aligned — same contract Discord established in fix(media): gate inbound audio on structured media facts #111315 for its failed-download audio case. Command routing is unaffected: empty input is non-command by the shared detector contract (traced per channel).
  • The frozen shared formatter lands with its first consumers: formatMediaPlaceholderText(facts) in src/channels/inbound-event/media.ts, exported via the existing openclaw/plugin-sdk/channel-inbound subpath. Kind inference reuses media-core (kind → MIME → URL/path extension → attachment); existing count vocabulary; mixed kinds collapse; no filename annotations. Mattermost's pending-room text (a text-only carrier) is its first consumer.
  • Deleted per channel: placeholder builders, string-sentinel comparisons for unavailable media, and Mattermost's expected-count side channel (now derived from fact-array lengths). Channel production code is net −25 lines.
  • SDK surface growth (+2 public, +1 callable) follows the budget-ledger process with regenerated API baseline; docs updated in docs/plugins/sdk-channel-inbound.md.

User Impact

Media-only messages from these four channels no longer inject <media:image>-style text into transcripts, previews, or model prompts; captions are exactly what the user typed. Failed downloads still surface their existing unavailable notices, and (new) their attachment kind/count still reaches the model context instead of vanishing.

Evidence

  • Focused suites via node scripts/run-vitest.mjs (each file green): formatter 8/8, googlechat monitor 17/17, zalo polling lifecycle 15/15, LINE bot-message-context 19/19, mattermost monitor-resources 18/18 + monitor 54/54, plus core consumers unchanged: inbound-media 6/6, user-turn-transcript 31/31, session-memory transcript 3/3, media-understanding apply 58/58.
  • Remote full changed-surface proof on Blacksmith Testbox (pre-rebase content): check:changed exit 0 — API manifest/exports/surface, four tsgo lanes, core/extensions/scripts lint, plugin boundaries, formatting, import-cycle guards (run 29688888129).
  • Post-rebase: API baseline --check green, formatter suite rerun green; surface budget re-derived against main's harvested base (8182 + 2 = 8184).
  • Production placeholder scan across the four migrated modules: zero <media:*, mediaPlaceholder, or expectedCount matches.
  • Autoreview (Codex gpt-5.6-sol): two review-fix cycles (Mattermost dispatched payload dropped failed attachments; URL-rejection path skipped its alignment fact — both fixed with regression tests) → final clean, "patch is correct (0.9)".
  • Known scoped gap: Google Chat still downloads only its first attachment; later attachments are preserved as type-only facts (count/kind reach the model; binaries do not). Multi-attachment download support remains deliberately out of scope.

@steipete
steipete marked this pull request as ready for review July 19, 2026 14:39
@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation channel: googlechat Channel integration: googlechat channel: line Channel integration: line channel: mattermost Channel integration: mattermost channel: zalo Channel integration: zalo scripts Repository scripts size: L maintainer Maintainer-authored PR labels Jul 19, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: abbcd6d81f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +275 to +277
const mediaInputs: ChannelInboundMediaInput[] = attachments.map((attachment) => ({
contentType: attachment.contentType,
}));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Exclude type-only facts from prompt attachment lines

When a Google Chat message has multiple attachments, every item after the first is deliberately left type-only here. buildChannelInboundEventContext projects those facts into index-aligned MediaPaths entries containing "" (src/channels/inbound-event/context.ts:352 and src/channels/inbound-event/media.ts:161), but buildInboundMediaNote counts and renders every MediaPaths entry (src/auto-reply/media-note.ts:134), so a normal two-attachment turn tells the model that an undownloaded second file is attached and emits a blank attachment path; Mattermost mixed download failures now hit the same path. Preserve index alignment for structured processing, but exclude facts without a path or URL from prompt attachment lines.

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 231e43f1ab

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

: nativeMediaKind
? [{ kind: nativeMediaKind }]
: [];
const rawBody = textContent;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep non-image media-only turns dispatchable to ACP

When a captionless LINE video or document is successfully downloaded, this leaves BodyForAgent empty while retaining only structured media. Bound ACP dispatch only converts image attachments (src/auto-reply/reply/agent-turn-attachments.ts:101-107), and if media understanding produces no text or extracted image, tryDispatchAcpReply classifies the empty prompt plus zero supported attachments as acp_empty_prompt and completes without running the model (src/auto-reply/reply/dispatch-acp.ts:685-736). The changed Mattermost path has the same regression for captionless non-image files. Preserve a downstream text fallback derived from the structured media facts when ACP cannot pass the attachment itself.

Useful? React with 👍 / 👎.

@openclaw-barnacle openclaw-barnacle Bot reopened this Jul 19, 2026
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. labels Jul 19, 2026
@clawsweeper

clawsweeper Bot commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 19, 2026, 1:12 PM ET / 17:12 UTC.

Summary
The branch stops Google Chat, LINE, Mattermost, and Zalo from placing <media:…> placeholder text in primary inbound bodies, preserves structured media facts, and adds a formatter for text-only channel carriers.

PR surface: Source +104, Tests +294, Docs +8, Generated 0, Other +2. Total +408 across 18 files.

Reproducibility: no. live reproduction was supplied in the reviewed materials, but the failure paths are high-confidence from source: captionless non-image facts yield no ACP attachments and no prompt text, while type-only MediaPaths entries reach prompt-note rendering.

Review metrics: 1 noteworthy metric.

  • Plugin SDK surface: 2 public exports added, 1 callable formatter added. The new public structured-media formatter and type become a compatibility contract for channel plugins.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🦪 silver shellfish
Result: blocked until real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P2] Fix the ACP fallback for captionless non-image media-only turns.
  • Filter pathless/URL-less type-only facts out of prompt attachment notes while retaining structured alignment.
  • [P1] Add redacted after-fix channel output or runtime logs for successful and failed media-only events; updating the PR body should trigger ClawSweeper re-review, or a maintainer can comment @clawsweeper re-review.

Proof guidance:

  • [P1] Needs real behavior proof before merge: The PR body reports tests and a changed-surface check, but the supplied review context contains no inspectable after-fix real channel run, terminal capture, runtime log, recording, or artifact proving successful and failed media-only behavior. Add redacted live evidence after the two correctness fixes; redact private endpoints, identifiers, and credentials. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

  • [P2] Merging without an ACP fallback can silently complete captionless LINE or Mattermost document/video turns without invoking the model, because ACP only materializes supported image attachments.
  • [P1] Merging with type-only facts in MediaPaths can expose blank attachment lines and incorrect attachment counts in model-visible prompt notes for failed or intentionally unmaterialized downloads.
  • [P1] The branch adds a public plugin-SDK formatter and type contract; the compatibility behavior should be made coherent with the downstream consumer fixes before that surface is accepted.

Maintainer options:

  1. Repair both downstream consumers before merge (recommended)
    Preserve structured-fact alignment, omit type-only entries from prompt attachment lines, and provide a text fallback when ACP cannot pass a captionless non-image attachment.
  2. Pause the producer batch
    Do not merge this batch until the shared ACP and prompt-context behavior has a compatible contract for type-only and unsupported media facts.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Repair the ACP empty-turn and blank prompt-attachment regressions with focused coverage for successful and failed captionless non-image media.

Next step before merge

  • [P1] The two blockers are mechanically understandable, but the protected maintainer label requires explicit maintainer handling rather than an automated repair lane.

Security
Cleared: The diff changes channel/media handling and SDK documentation only; no dependency source, workflow permission, secret, package-resolution, or supply-chain concern was identified.

Review findings

  • [P1] Preserve a fallback for non-image ACP media turns — extensions/line/src/bot-message-context.ts:225-244
  • [P2] Exclude type-only facts from prompt attachment paths — extensions/googlechat/src/monitor.ts:272-285
Review details

Best possible solution:

Keep structured media as the canonical channel contract, but add a downstream ACP-safe text fallback for unsupported captionless media and have prompt-note rendering omit facts with neither a usable path nor URL; then show redacted after-fix channel evidence for successful and failed media-only cases.

Do we have a high-confidence way to reproduce the issue?

No live reproduction was supplied in the reviewed materials, but the failure paths are high-confidence from source: captionless non-image facts yield no ACP attachments and no prompt text, while type-only MediaPaths entries reach prompt-note rendering.

Is this the best way to solve the issue?

No. Replacing producer placeholders is directionally sound, but doing so without repairing ACP’s unsupported-media fallback and the prompt-note consumer creates two downstream regressions; the shared consumer fixes are the narrower durable solution.

Full review comments:

  • [P1] Preserve a fallback for non-image ACP media turns — extensions/line/src/bot-message-context.ts:225-244
    This change removes the only caption text for successful media-only document and video events. ACP only converts materialized image attachments; its empty-text/zero-supported-attachment branch completes the message as acp_empty_prompt, so these inbound turns never reach the model. Keep structured facts, but derive a downstream fallback only when ACP cannot pass the attachment.
    Confidence: 0.94
  • [P2] Exclude type-only facts from prompt attachment paths — extensions/googlechat/src/monitor.ts:272-285
    This branch intentionally preserves failed or undispatched attachments as type-only facts, which creates aligned empty MediaPaths entries. buildInboundMediaNote renders every entry, so the model receives blank attachment lines and an inflated file count. Retain facts for structured processing, but filter entries without a path or URL before prompt rendering.
    Confidence: 0.96

Overall correctness: patch is incorrect
Overall confidence: 0.93

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against d4c7549d1ca7.

Label changes

Label changes:

  • add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦪 silver shellfish.
  • remove rating: 🦪 silver shellfish: Current PR rating is rating: 🧂 unranked krab, so this older rating label is no longer current.

Label justifications:

  • P1: The proposed behavior can cause real captionless inbound media turns to be silently completed without an ACP model run.
  • merge-risk: 🚨 compatibility: The PR changes shared plugin-SDK media semantics and makes existing media-only channel input empty-bodied.
  • merge-risk: 🚨 message-delivery: The ACP empty-turn path can suppress the agent response for affected incoming media messages.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦪 silver shellfish.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body reports tests and a changed-surface check, but the supplied review context contains no inspectable after-fix real channel run, terminal capture, runtime log, recording, or artifact proving successful and failed media-only behavior. Add redacted live evidence after the two correctness fixes; redact private endpoints, identifiers, and credentials. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed

PR surface:

Source +104, Tests +294, Docs +8, Generated 0, Other +2. Total +408 across 18 files.

View PR surface stats
Area Files Added Removed Net
Source 10 218 114 +104
Tests 5 312 18 +294
Docs 1 8 0 +8
Config 0 0 0 0
Generated 1 1 1 0
Other 1 4 2 +2
Total 18 543 135 +408

What I checked:

  • Prior review blockers remain applicable: The supplied prior ClawSweeper review identified the ACP empty-turn and blank prompt-attachment-path regressions at the current reviewed head; the current PR diff still makes successful media-only bodies empty while retaining type-only facts. (extensions/line/src/bot-message-context.ts:225, 231e43f1ab23)
  • Prompt-note consumer does not exclude type-only entries: Current main's prompt-note builder uses every MediaPaths entry, including an empty string, to build attachment lines. The PR deliberately creates such index-aligned empty entries for undownloaded attachments, so this produces a blank attachment reference rather than only rendering materialized media. (src/auto-reply/media-note.ts:908, d4c7549d1ca7)
  • ACP accepts only image attachments: Current main resolves ACP turn attachments only for image MIME types with a local path, then completes a turn as acp_empty_prompt if both text and supported attachments are empty. The PR makes successful captionless document/video turns empty-bodied, exposing that path. (src/auto-reply/reply/agent-turn-attachments.ts:4, d4c7549d1ca7)
  • Related merged foundation: The merged predecessor fix(media): gate inbound audio on structured media facts #111315 moved inbound-audio detection to structured media facts. This PR is an adjacent producer-migration batch, not a replacement for that merged change. (src/channels/inbound-event/media.ts:1, e20d4cd450a4)
  • Maintainer review gate: The PR currently carries the protected maintainer label, so this workflow must leave it open for explicit maintainer handling.

Likely related people:

  • steipete: Authored the merged structured-media foundation in fix(media): gate inbound audio on structured media facts #111315 and the current producer-migration batch, connecting them to the active media-placeholder transition. (role: related feature author and recent area contributor; confidence: high; commits: e20d4cd450a4, abbcd6d81f9f; files: src/channels/inbound-event/media.ts, extensions/line/src/bot-message-context.ts, extensions/mattermost/src/mattermost/monitor.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (1 earlier review cycle)
  • reviewed 2026-07-19T16:41:48.567Z sha 231e43f :: needs real behavior proof before merge. :: [P1] Preserve a fallback for non-image ACP media turns | [P2] Exclude type-only facts from prompt attachment paths

@steipete steipete self-assigned this Jul 19, 2026
steipete added 2 commits July 19, 2026 10:10
Media-placeholder program batch 1: Google Chat, Zalo, LINE, and
Mattermost stop minting <media:kind> placeholder bodies. Media-only
messages carry an empty caption plus one structured fact per native
attachment (type-only when a download fails or is rejected, so payload
positions and kind signals stay aligned). The shared
formatMediaPlaceholderText SDK formatter renders text-only carriers
(Mattermost pending-room lines) from structured facts; per-channel
placeholder builders and the expected-count side channel are deleted.
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Jul 19, 2026
@steipete
steipete force-pushed the claude/media-placeholder-batch1 branch from 231e43f to 00432a7 Compare July 19, 2026 17:22

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 00432a7f65

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +275 to +277
const mediaInputs: ChannelInboundMediaInput[] = attachments.map((attachment) => ({
contentType: attachment.contentType,
}));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve a fallback for unavailable Google Chat attachments

When a captionless attachment lacks attachmentDataRef.resourceName—for example, the no-ref attachment already constructed in extensions/googlechat/src/monitor.test.ts:288downloadAttachment returns null, leaving this fact with only contentType while rawBody remains empty. buildChannelInboundMediaPayload then emits no MediaPath(s) or MediaUrl(s), so buildInboundMediaNote returns undefined (src/auto-reply/media-note.ts:136-144), and bound ACP completes the turn as acp_empty_prompt without invoking the model (src/auto-reply/reply/dispatch-acp.ts:731-736). Unlike the existing multi-attachment blank-path finding, this fresh single-attachment case loses the prior <media:attachment> signal entirely; retain an unavailable-media text fallback when no attachment can be materialized.

Useful? React with 👍 / 👎.

@steipete
steipete merged commit 83303f1 into main Jul 19, 2026
121 checks passed
@steipete
steipete deleted the claude/media-placeholder-batch1 branch July 19, 2026 17:32
@steipete

Copy link
Copy Markdown
Contributor Author

Merged via squash.

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 20, 2026
…claw#111447)

* feat(channels): batch 1 producers drop media placeholder bodies

Media-placeholder program batch 1: Google Chat, Zalo, LINE, and
Mattermost stop minting <media:kind> placeholder bodies. Media-only
messages carry an empty caption plus one structured fact per native
attachment (type-only when a download fails or is rejected, so payload
positions and kind signals stay aligned). The shared
formatMediaPlaceholderText SDK formatter renders text-only carriers
(Mattermost pending-room lines) from structured facts; per-channel
placeholder builders and the expected-count side channel are deleted.

* fix(mattermost): satisfy type, deadcode, and SDK manifest gates
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: googlechat Channel integration: googlechat channel: line Channel integration: line channel: mattermost Channel integration: mattermost channel: zalo Channel integration: zalo docs Improvements or additions to documentation maintainer Maintainer-authored PR merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. P1 High-priority user-facing bug, regression, or broken workflow. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. scripts Repository scripts size: L status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant