Summary

gemini-embedding-2-preview now supports multimodal embedding - text, images, video, audio, and PDFs all map to the same vector space. This enables cross-modal search (e.g., search images by text description).

Proposed Changes

1. Extend memorySearch to embed images/attachments alongside text when storing conversation history
2. Add config option for output dimensionality (outputDimensionality: 768 | 1536 | 3072)
3. Support mixed-modal queries - user can search with text and get relevant images from history

API Reference

curl ".../gemini-embedding-2-preview:embedContent" \
  -d '{"content": {"parts": [
    {"text": "description"},
    {"inline_data": {"mime_type": "image/png", "data": "base64..."}}
  ]}, "outputDimensionality": 768}'

Model Capabilities

Output: 3072 dims (default), scalable to 768/1536 via MRL.

Benefits

• Cross-modal RAG: "find that screenshot I sent last week"
• Unified semantic space for all conversation content
• Better context retrieval for image-heavy conversations

References

• Gemini Embedding 2 Announcement
• API Docs