Skip to content

Bug: coalesced mixed-media batches lose non-image enrichment (audio transcript / doc extract) on the debounced path #6348

Description

@nevgenov

Symptom

On a channel bot with a coalesce window (message_coalesce_window_ms > 0), when an inbound batch contains an image together with other media (e.g. a photo + a voice note sent within the debounce window), the non-image media is delivered to the agent as a bare text placeholder — [Voice message (5s): <url>], [File (<name>): <url>] — instead of its transcript / extracted text. The agent never receives the voice transcription or document contents.

This is the same class of defect that #6321 fixed for images (PR #6323), but only images were made path-agnostic; audio/voice/document enrichment was not generalized.

Scope (precise)

The gap is narrow but real — it depends on whether the coalesced batch contains an image:

Case Path Result
Voice/doc alone, or in an image-free batch blocks == Nonedispatch_message transcribed / enriched ✅
Voice/doc coalesced with an image blocks == Somedispatch_with_blocks placeholder text, not transcribed/enriched ❌

Root cause (tag v2026.6.26-beta.24, crates/librefang-channels/src/bridge.rs)

  1. The debounce ingest pre-downloads only images into blocks; all other media is pushed as raw ChannelContent:
// bridge.rs:1527
let image_blocks = if let ChannelContent::Image { ref url, ref caption, ref mime_type } = message.content {
    // download_image_to_blocks(...)
} else {
    None
};
let pending = PendingMessage { message, image_blocks };
  1. On coalesce, MessageDebouncer::drain turns every non-image message into a text placeholder via content_to_text (bridge.rs:984-1020): [Voice message (Ns): url], [Audio (Ns): url], [File (name): url].

  2. The debounced flush only enriches images, then dispatches the assembled blocks:

// bridge.rs:1088  — image describe (the #6321 fix), no audio/doc step
let blocks = prepend_image_descriptions(&channel_handle, blocks).await;
// ... dispatch_with_blocks(blocks, ...)   // bridge.rs:~1228

So when blocks == Some (an image was in the batch), the path goes through dispatch_with_blocks, which runs only prepend_image_descriptions. The audio/doc enrichment steps live exclusively in dispatch_message and are never reached for that batch:

// dispatch_message — full per-type enrichment (the only place these run):
// bridge.rs:3947  prepend_image_descriptions(...)
// bridge.rs:4024  maybe_transcribe_inbound_audio(...)   // Voice arm
// bridge.rs:4087  maybe_transcribe_inbound_audio(...)   // Audio arm
// bridge.rs:5841  enrich_saved_file(...)                // File/doc arm

The else branch of the flush (no image in batch) calls dispatch_message directly, which is why standalone voice/docs still work — the gap is specifically the mixed image + other-media batch.

Why this is the same divergence #6321 warned about

#6321/#6323 fixed image parity by extracting prepend_image_descriptions and calling it from both the immediate (dispatch_message) and debounced (dispatch_with_blocks) paths — explicitly so the two paths cannot drift out of parity again. But enrichment for the other media types was left only on the immediate path, so the divergence persists for audio and documents.

Proposed fix

Generalize the #6321 approach to all media types: replace the image-only prepend_image_descriptions with a single, type-agnostic enrich_media_blocks(handle, blocks) that, for each media block in the final assembled list, runs the appropriate existing enrichment —

  • ImageFilemaybe_describe_inbound_image
  • audio/voice → maybe_transcribe_inbound_audio
  • document/file → enrich_saved_file

— and call it from the single shared point before dispatch_with_blocks, used by both paths. The underlying enrichment helpers already exist; this is a consolidation, not new capability. It also requires the debounce ingest (bridge.rs:1527) to download non-image media into blocks too (mirroring download_image_to_blocks), so coalesced audio/docs carry a file path rather than a bare URL placeholder.

Net effect: media is enriched identically whether coalesced or not, and a mixed photo+voice+pdf burst reaches the agent as one turn with every attachment turned into text — which also benefits text-only models.

Setup (for reference)

  • v2026.6.26-beta.24
  • Pattern: channel bot with message_coalesce_window_ms > 0; user sends an image and a voice note (or a document) within the coalesce window.
  • Repro: the voice arrives as [Voice message (Ns): <url>] with no transcript; the same voice sent alone is transcribed.

Happy to test a patch.

Refs: #6321 / #6323 (image parity fix this generalizes)

Metadata

Metadata

Assignees

No one assigned

    Labels

    has-prA pull request has been linked to this issueneeds-triageAuto-applied when the issue title/body matched no area label — needs maintainer review

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions