Summary

When memorySearch.multimodal.enabled = true with Gemini embedding 2, image and audio files are properly indexed with valid embeddings. However, they never surface in memory_search results under the default hybrid search configuration.

Root Cause

Hybrid search computes: finalScore = vectorWeight × vectorScore + textWeight × textScore

Image/audio chunks have minimal text content (e.g., "Image file: generated/images/photo.png"), so their BM25 (text) score is near-zero for any natural language query. With default weights (0.7/0.3), the BM25 penalty is enough to push image results below text-only chunks that match both signals.

Reproduction

1. Configure multimodal memory search with Gemini embedding 2
2. Index image files via extraPaths
3. Run memory_search with a query describing image content (e.g., "lobster and dolphin underwater cartoon")
4. Observe: only markdown text results returned, zero images

Workaround

Setting vectorWeight: 0.9, textWeight: 0.1 allows image results to surface (tested — images jump to #1 and #2 in results).

Suggested Fix

The hybrid merge function should detect when a chunk's source modality is non-text (image/audio) and skip the BM25 component for those chunks, using vector score only. Something like:

if (chunk.modality === 'image' || chunk.modality === 'audio') {
  finalScore = vectorScore;  // BM25 is meaningless for binary content
} else {
  finalScore = vectorWeight * vectorScore + textWeight * textScore;
}

This would let multimodal and text results compete fairly without requiring users to weaken BM25 for text-on-text queries.

Environment

• OpenClaw 2026.3.11
• Provider: gemini, model: gemini-embedding-2-preview
• 296 indexed files (169 .md, 100 images, 27 audio)
• Default hybrid config (no custom weights) reproduces the issue