Skip to content

[Bug]: WhatsApp DM voice notes bypass STT pipeline entirely — audio arrives as raw <media:audio> with no transcription attempt #63677

Description

@Workz777

Bug type

Regression (worked before, now fails)

Beta release blocker

No

Summary

WhatsApp DM voice notes arrive as raw media:audio placeholders with zero transcription attempt — the STT pipeline is completely bypassed despite Groq Whisper being correctly configured, plugin enabled, and verified working via direct curl API call.

Steps to reproduce

  1. Configure Groq Whisper as the audio transcription provider in openclaw.json:

{
env: { vars: { GROQ_API_KEY: "gsk_..." } },
plugins: { entries: { groq: { enabled: true } } },
tools: {
media: {
audio: {
enabled: true,
scope: { default: "allow" },
models: [{ provider: "groq" }]
}
}
}
}

  1. Restart gateway (openclaw gateway restart).
  2. Send a voice note (audio/ogg; codecs=opus) via WhatsApp DM to the OpenClaw instance.
  3. Observe: audio arrives in the agent session as raw media:audio with no transcription. Groq dashboard confirms zero API requests. Verbose gateway logs show no media understanding step between inbound message receipt and agent session injection.

Expected behavior

Voice note should be transcribed via Groq Whisper and the transcript injected into the session as an [Audio] block with {{Transcript}} set, replacing media:audio. This is the documented behavior per /providers/groq and /nodes/audio, and matches how Telegram DM voice notes now work after the preflight transcription fix in #61008 (2026.4.5). Direct curl to the same Groq API with the same audio file and API key returns a successful transcription, confirming the provider and config are correct.

Actual behavior

Voice note arrives as raw media:audio placeholder with no transcription. Verbose gateway logs show the complete processing chain with no media understanding step:

  1. web-inbound receives message with body: "media:audio", mediaPath and mediaType present
  2. web-auto-reply/processMessage forwards message directly to agent session
  3. No transcription attempt occurs — no Groq API call, no provider error, no media understanding log entry

The STT pipeline is entirely bypassed. Groq dashboard confirms zero API requests from the instance. Direct curl to Groq API with the same audio file and GROQ_API_KEY returns successful transcription: {"text":"Okay test test we're testing if this audio message is getting to our dear friend Miles..."} — proving the issue is in the OpenClaw pipeline, not the provider.

OpenClaw version

Broken: 2026.3.31 (56b5ba0), 2026.4.1 (4f407d2), 2026.4.2 (f8e67ef), 2026.4.2-beta.1 (721cab2), 2026.4.6 (898579d), 2026.4.9 (97dfbe0). Working: 2026.3.14 (594920f) and older.

Operating system

macOS Monterey 12.7.x (x64)

Install method

pnpm dev

Model

anthropic/claude-opus-4.6 / anthropic/claude-sonnet-4.6 / google/gemini-3.1-pro-preview / google/gemini-3.1-pro-preview-customtools

Provider / routing chain

WhatsApp (Baileys) → OpenClaw gateway → tools.media.audio pipeline → Groq API (whisper-large-v3-turbo) — pipeline never reached; audio forwarded to agent session unprocessed. No proxies or intermediary gateways.

Additional provider/model setup details

Auth: GROQ_API_KEY set in env.vars (openclaw.json). No auth.profiles entry for Groq — key resolved via env var only. models.providers is empty — Groq is not registered as a chat model provider (only used for audio transcription).

Audio config:

  • tools.media.audio.enabled: true
  • tools.media.audio.scope: { default: "allow" }
  • tools.media.audio.timeoutSeconds: 300
  • tools.media.audio.models: [{ provider: "groq" }] (defaults to whisper-large-v3-turbo per docs)
  • plugins.entries.groq: { enabled: true }
  • No plugins.allow restriction set — all bundled plugins load by default

Also tested: auto-detect mode (no explicit audio.models array) — Groq is never selected, audio still arrives as raw media:audio.
Also tested: without plugins.entries.groq (relying on enabledByDefault from #59982) — same result.

Agent config: 5 agents defined. Default agent is "main" running anthropic/claude-opus-4-6. No per-agent audio overrides. Issue reproduces regardless of which agent handles the session.

WhatsApp config: channels.whatsapp.dmPolicy: "allowlist", blockStreaming: false, debounceMs: 0, healthMonitor.enabled: false. No WhatsApp-specific media or audio overrides.

No proxies, no baseUrl overrides, no per-agent audio config. Direct egress to api.groq.com (verified working via curl).

Logs, screenshots, and evidence

### Verbose log excerpt — inbound voice note processing chain

Complete processing chain for a WhatsApp DM voice note on 2026.4.9 (97dfbe0). Note the absence of any media understanding / transcription step:

json
// 1. WhatsApp channel receives audio
{"module":"web-inbound", "from":"+44REDACTED", "body":"<media:audio>", "mediaPath":"/Users/.../.openclaw/media/inbound/ad90f53d-...ogg", "mediaType":"audio/ogg; codecs=opus", "timestamp":1775583366000}

// 2. Message forwarded directly to agent session — no transcription step
{"module":"web-auto-reply","runId":"d93556b4-...","body":"[WhatsApp +44REDACTED +13s ...] <media:audio>", "mediaType":"audio/ogg; codecs=opus", "mediaPath":"/Users/.../.openclaw/media/inbound/ad90f53d-...ogg"}

// 3. Gateway inbound log — audio recognized but not processed
{"subsystem":"gateway/channels/whatsapp/inbound", "Inbound message +44REDACTED -> +44REDACTED (direct, audio/ogg; codecs=opus, 70 chars)"}

// No further audio/transcription/media-understanding/groq log entries exist


### Direct Groq API verification (same audio file, same API key)

bash
curl -X POST "https://api.groq.com/openai/v1/audio/transcriptions" \
  -H "Authorization: Bearer $GROQ_API_KEY" \
  -F "file=@/Users/.../.openclaw/media/inbound/ad90f53d-...ogg" \
  -F "model=whisper-large-v3-turbo"


Response:
json
{"text":"Okay test test we're testing if this audio message is getting to our dear friend Miles. Miles do you read this? Over.","x_groq":{"id":"req_01knmga450eewa8cpam12nwd08"}}


This proves the Groq API key, audio file, and Whisper model all work correctly. The issue is entirely within OpenClaw's pipeline.

### Configuration (redacted)

json5
{
  env: { vars: { GROQ_API_KEY: "gsk_..." } },
  plugins: { entries: { groq: { enabled: true } } },
  tools: {
    media: {
      audio: {
        enabled: true,
        scope: { default: "allow" },
        timeoutSeconds: 300,
        models: [{ provider: "groq" }]
      }
    }
  }
}


### Doctor output (relevant sections)

- Plugins: Loaded: 56, Imported: 56, Disabled: 41, Errors: 0
- WhatsApp: linked
- No Groq-related errors or warnings

### Related issues

- **#59875** — Groq Whisper STT broken since 2026.3.31 (provider resolution). Both PRs (#59926, #59982) merged in 2026.4.5. Provider resolution fixed, but WhatsApp DM transcription pipeline still never triggers.
- **#61008** — Telegram DM voice-note preflight transcription restored in 2026.4.5. WhatsApp DMs have the identical symptom (raw `<media:audio>` placeholder) but were not included in that fix.
- **#60392** — Auto-enable media provider plugins referenced from tools.media. Also merged, also did not resolve the WhatsApp DM pipeline issue.

Impact and severity

  • Affected: All WhatsApp DM users sending voice notes to OpenClaw instances with configured STT providers (Groq, Deepgram, or any audio transcription provider). WhatsApp is OpenClaw's most popular messaging channel.
  • Severity: Blocks workflow — voice notes are a primary WhatsApp input method and are completely non-functional. Agents receive raw media:audio placeholders instead of transcribed text, making voice-based interaction impossible.
  • Frequency: Always — 100% reproducible on every WhatsApp DM voice note tested across 6 versions (2026.3.31 through 2026.4.9). Zero successful transcriptions observed.
  • Consequence: Voice notes are silently dropped as untranscribed audio. Agents cannot read or respond to voice content. Users relying on voice input (hands-free, accessibility, mobile-first workflows) have no workaround other than typing. The failure is silent — no error is surfaced to the user or operator, making it difficult to diagnose without verbose log inspection. This regression has persisted since 2026.3.31 (approximately 10 days unfixed across 6+ releases).

Additional information

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingregressionBehavior that previously worked and now fails

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions