Skip to content

fix(media): m4a/AAC audio files from non-Apple sources are misidentified as video/mp4 and excluded from audio workflows#111177

Merged
steipete merged 2 commits into
openclaw:mainfrom
YangManBOBO:fix/m4a-audio-misdetection
Jul 20, 2026
Merged

fix(media): m4a/AAC audio files from non-Apple sources are misidentified as video/mp4 and excluded from audio workflows#111177
steipete merged 2 commits into
openclaw:mainfrom
YangManBOBO:fix/m4a-audio-misdetection

Conversation

@YangManBOBO

Copy link
Copy Markdown
Contributor

What Problem This Solves

Fixes an issue where non-Apple m4a/AAC audio files (Android voice recordings, ffmpeg-transcoded audio, Telegram/Signal voice messages) are misidentified as video/mp4 by the MIME detection pipeline, causing them to be silently excluded from audio-only workflows.

The root cause is that file-type (the npm dependency that sniffs magic bytes) detects ISO-BMFF containers with the "isom" or "mp42" ftyp brand as video/mp4 regardless of content. Only Apple-authored m4a files use the "M4A " brand, which file-type correctly reports as audio/mp4. The existing ambiguous-container rescue only covers audio/mp4 and audio/webm — three real-world audio hint aliases and the file-extension-only case were left unprotected.

Concrete user impact: voice message received via Telegram with Content-Type audio/x-m4a or audio/m4a → stored with contentType video/mp4hasAudio check in bot-message-context.body.ts fails → no preflight transcription; selectAttachments("audio") filters it out via kindFromMime routing it to video capability → user message is silently unhandled.

Why This Change Was Made

Three audio/* aliases (audio/x-m4a, audio/m4a, audio/aac) are added to the existing AMBIGUOUS_VIDEO_MIME_BY_AUDIO_MIME map, which rescues audio hints when file-type returns one of the ambiguous container MIMEs (video/mp4, video/webm). These are aliases acknowledged by the repo's own voiceCompatMimes list in src/media/audio.ts.

Additionally, the audioContainerHint search now falls back to the extension-derived MIME after transport-header-based hints. Without this, a bare .m4a file with no Content-Type header produces video/mp4 because only mimeHints (transport headers) were consulted — the independently-computed extMime was excluded from the rescue.

The fix is bounded: it only changes what alias strings trigger the existing rescue when file-type returns video/mp4. Genuine video files (.mp4 extension, video/mp4 header, .mp4 extension without header) continue to return video/mp4 unchanged.

User Impact

Voice messages from Android/ffmpeg-produced m4a files, AAC audio, and m4a files without a Content-Type header are now correctly identified as audio. They follow the audio path for storage, attachment selection, and preflight transcription — matching the existing behavior for Apple-authored "M4A "-brand m4a files.

Evidence

Live behavior proof (production detectMime, isom-brand buffer)

Driver: the production detectMime function from packages/media-core/src/mime.ts is called with a 28-byte isom-brand ftyp box buffer (the common ffmpeg/Android-produced m4a format) and real-world header/filename combinations.

Before (current main) — 3 audio hint aliases and the ext-only case all return video/mp4:

m4a (Android/ffmpeg): video/mp4 ✗
m4a alias:            video/mp4 ✗
AAC audio:            video/mp4 ✗
audio/mp4:            audio/mp4 ✓   (already covered)
no header (ext only): video/mp4 ✗
genuine video (.mp4): video/mp4 ✓   (correctly unchanged)
video ext only:       video/mp4 ✓   (correctly unchanged)

After (this patch) — all audio cases return their expected audio MIME:

m4a (Android/ffmpeg): audio/x-m4a ✓
m4a alias:            audio/m4a   ✓
AAC audio:            audio/aac   ✓
audio/mp4:            audio/mp4   ✓
no header (ext only): audio/x-m4a ✓
genuine video (.mp4): video/mp4   ✓
video ext only:       video/mp4   ✓

Regression coverage

  • packages/media-core/src/mime.test.ts (2 new): isom-brand buffer + audio/x-m4a header preserves the audio hint; isom-brand buffer + bare .m4a filename (no header) rescues via extension.
  • Against unfixed main, both new tests fail (isom-brand sniffs as video/mp4; the rescue map lacks the alias; the extMime fallback is absent). 140 existing tests pass unchanged.
  • node scripts/run-vitest.mjs packages/media-core/src/mime.test.ts → 142 passed.
  • git diff --check clean.

Sibling-surface context

The existing audio/mp4 and audio/webm rescue in the same map was added in the original ambiguous-container fix and is pinned by tests at mime.test.ts:199-228. The three added aliases follow the same contract. The media.store.ts callers consume detectMime's return value directly; kindFromMime and isAudioFileName derive their results from this return value. No config, schema, or storage migration is needed.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. labels Jul 19, 2026
@clawsweeper

clawsweeper Bot commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 20, 2026, 1:13 AM ET / 05:13 UTC.

Summary
The PR adds audio/x-m4a and audio/m4a aliases to ambiguous ISO-BMFF MIME rescue logic and uses filename-derived MIME as a fallback hint, with parameterized MIME regression tests.

PR surface: Source +3, Tests +48. Total +51 across 2 files.

Reproducibility: yes. at source level: current main prioritizes file-type’s video/mp4 result over M4A header and filename hints, and src/media/store.ts:17 passes those exact inputs into the package MIME detector. The PR body also supplies an after-fix production-function run, but this review did not execute the test locally.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: vector/embedding metadata: packages/media-core/src/mime.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • [P2] Narrow the title/body and after-fix claims to M4A aliases and extension-only M4A detection, or explicitly explain why contradictory audio/aac metadata remains video.

Risk before merge

  • [P1] The PR title and body still describe AAC audio as fixed, but the final patch deliberately preserves video/mp4 for an audio/aac header paired with ISO-BMFF bytes; merge communication should describe the M4A-only scope accurately.

Maintainer options:

  1. Decide the mitigation before merge
    Land the narrow M4A hint rescue after aligning the title and PR narrative with the final code: preserve M4A transport or filename hints for ambiguous ISO-BMFF bytes, while continuing to trust contradictory ISO-BMFF bytes over audio/aac metadata.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P2] No mechanical repair is identified; this is a focused contributor PR awaiting ordinary maintainer review, final CI completion, and accurate scope wording.

Security
Cleared: The diff changes only MIME alias selection and unit coverage; it adds no dependencies, executable paths, permissions, secrets handling, or supply-chain inputs.

Review details

Best possible solution:

Land the narrow M4A hint rescue after aligning the title and PR narrative with the final code: preserve M4A transport or filename hints for ambiguous ISO-BMFF bytes, while continuing to trust contradictory ISO-BMFF bytes over audio/aac metadata.

Do we have a high-confidence way to reproduce the issue?

Yes at source level: current main prioritizes file-type’s video/mp4 result over M4A header and filename hints, and src/media/store.ts:17 passes those exact inputs into the package MIME detector. The PR body also supplies an after-fix production-function run, but this review did not execute the test locally.

Is this the best way to solve the issue?

Yes for the M4A regression: extending the existing ambiguous-container exception is narrower and safer than broadly trusting all audio headers over sniffed container bytes. The proposed AAC wording is not the best description because the final regression test intentionally keeps contradictory audio/aac plus ISO-BMFF data classified as video.

AGENTS.md: unclear because the file could not be read completely.

Codex review notes: model internal, reasoning high; reviewed against 8f9e42f2a947.

Label changes

Label justifications:

  • P2: This is a bounded media-routing bug affecting M4A uploads and voice-media handling, with no evidence of broader runtime failure.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body provides before/after output from the production detectMime function using an ISO-BMFF buffer and reports focused regression coverage; the completed real-behavior-proof check supports that evidence.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body provides before/after output from the production detectMime function using an ISO-BMFF buffer and reports focused regression coverage; the completed real-behavior-proof check supports that evidence.
Evidence reviewed

PR surface:

Source +3, Tests +48. Total +51 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 6 3 +3
Tests 1 51 3 +48
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 57 6 +51

What I checked:

  • Current affected runtime path: Current main persists inbound media through saveMediaBuffer, which calls detectMime from @openclaw/media-core/mime with the buffer, transport MIME, and original filename; this makes package-level MIME classification directly user-visible to channel media handling. (src/media/store.ts:17, 8f9e42f2a947)
  • Current main still lacks the rescue: Current packages/media-core MIME detection returns a non-generic sniffed MIME ahead of extension or header MIME, so an isom buffer detected as video/mp4 still wins over an M4A filename or audio header. (packages/media-core/src/mime.ts:7, 8f9e42f2a947)
  • Patch is bounded to the existing decision point: The proposed change only extends the ambiguous video/mp4 rescue aliases for M4A and adds the already-computed extension MIME as a fallback; it does not alter generic-container handling, persisted media metadata shape, or channel configuration. (packages/media-core/src/mime.ts:246, 0029122bf1ac)
  • Regression coverage distinguishes the unsafe AAC case: The updated parameterized test preserves audio/x-m4a, audio/m4a, and extension-only M4A hints, while explicitly expecting video/mp4 for an audio/aac header paired with ISO-BMFF bytes. That prevents an elementary-AAC header from overriding contradictory container bytes. (packages/media-core/src/mime.test.ts:227, 0029122bf1ac)
  • Recent corrective provenance: The latest PR commit, 0029122bf1ac573ec0f3cf1c22cda5b3022eeb1e (fix(media): correct ambiguous M4A aliases), follows the initial MIME-rescue commit and aligns the tests with the narrower M4A-only behavior. (packages/media-core/src/mime.test.ts:227, 0029122bf1ac)

Likely related people:

  • steipete: Authored the latest commit narrowing the ambiguous-container behavior and updating the affected MIME tests on this PR. (role: latest corrective-commit author; confidence: medium; commits: 0029122bf1ac; files: packages/media-core/src/mime.ts, packages/media-core/src/mime.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (1 earlier review cycle)
  • reviewed 2026-07-19T03:40:20.444Z sha a69a3de :: needs maintainer review before merge. :: none

@steipete
steipete merged commit e45ebac into openclaw:main Jul 20, 2026
111 of 113 checks passed
@steipete

Copy link
Copy Markdown
Contributor

Merged via squash.

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Jul 20, 2026
…ied as video/mp4 and excluded from audio workflows (openclaw#111177)

* fix(media): preserve audio hints for isom-brand m4a/AAC files misidentified as video/mp4

* fix(media): correct ambiguous M4A aliases

Co-authored-by: 潘晓波0668000512 <[email protected]>

---------

Co-authored-by: Peter Steinberger <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants