Best Multimodal Embedding Models and APIs in 2026: Text, Image, Video and Audio

Multimodal embedding is the layer that lets one query reach every kind of content you own. Get the model choice wrong and you find out expensively: vectors from one model are meaningless to another, so switching means re-embedding your entire corpus.

The category changed shape in the last twelve months. Until recently, "multimodal embedding" effectively meant CLIP - a text tower and an image tower, aligned at the output layer, with nothing for video or audio. In 2026 there are three distinct architectures competing, video and audio are first-class inputs rather than a roadmap item, and the leading models put five modalities in a single vector space.

We compared 16 models - managed APIs and open weights - on cross-modal retrieval quality, modality coverage, cost at scale, latency and deployment constraints. Below is the full comparison table, the architecture distinction that decides most shortlists, what the benchmarks do and don't measure, and a five-step framework for narrowing to two or three candidates you can test on your own data.

Multimodal embedding models compared at a glance 

What is a multimodal embedding model?

A multimodal embedding model converts different kinds of content - a paragraph, a photograph, a video clip, a voice note - into vectors that live in the same space. Similarity between any two of them is then just geometry, usually cosine similarity.

That single property is what makes cross-modal search possible. Type a sentence, retrieve a video. Upload a screenshot, retrieve the paragraph that describes it. Play an audio clip, find the moment it came from. Without a shared space you would need a separate model per modality and no meaningful way to compare their outputs, because vectors from two different models are unrelated even when they have identical dimensionality.

The practical consequence is that the model determines the quality ceiling of everything built on top of it, and the switching cost is a full re-index.

A note on scope: if your corpus is images and only images, the trade-offs are different - vision-only models often beat cross-modal ones at pure image-to-image matching. That comparison lives in our guide to image embedding models. This article is about spanning modalities.

Cross-modal, any-to-any, and omni: the distinction that decides your shortlist

Almost every comparison treats these models as one category. They aren't, and the architecture difference constrains what you can build.

Dual-encoder cross-modal (CLIP, SigLIP 2, JinaCLIP v2, Nomic Embed Vision)

Two separate encoders: one for text, one for images, trained contrastively so their outputs align. Cheap, fast, extremely well understood. The limitation is structural: they handle text ↔ image and nothing else. An input that mixes text and image in one document has to be split, embedded separately and recombined by hand.

Any-to-any VLM-backbone (Jina v4, GME, Qwen3-VL-Embedding, Voyage Multimodal 3.5)

A vision-language model repurposed as an encoder. Text, an image, or interleaved text-and-image all pass through the same pathway and pool into one vector. This is what lets you embed a slide, an invoice, or a screenshot as it appears, preserving layout and figures rather than OCR-ing them into a text string first.

Voyage's argument for this design is worth understanding: separate towers create a modality gap, where a text query systematically scores text documents higher than equally relevant images simply because they came from the same encoder. A single encoder removes it.

Omni (Gemini Embedding 2, Nova 2, Marengo 3.0, jina-v5-omni, ImageBind)

Four or more modalities in one space, including audio and video as native inputs rather than sampled frames. This is the 2026 arrival. It matters when your corpus genuinely is mixed media - a support org with articles, screenshots and call recordings, or a broadcaster with footage, transcripts and stills.

Rule of thumb: text-and-images only, at scale, on a budget → dual encoder. Documents where layout carries meaning → any-to-any. Audio or video in the corpus → omni. Most teams over-buy here; most of the rest discover the gap eighteen months in.

How we evaluated the best multimodal embedding models in 2026

Five weighted criteria, applied consistently:

Best multimodal embedding APIs in 2026

1. Google Gemini Embedding 2

What it is. Google DeepMind's first embedding model built natively on the Gemini architecture, in public preview since March 2026 via the Gemini API and Vertex AI. It maps text, images, video, audio and PDFs into a single 3,072-dimensional space.

What sets it apart. Nothing else currently ingests this many modalities natively from one endpoint. The alternative is stitching CLIP for images, a speech model for audio and a text embedding model together, then discovering their vector spaces don't align. Google's published input limits are 8,192 text tokens, up to six images per request, around two minutes of video, native audio, and PDFs up to six pages — no format conversion.

Strengths

• Genuinely native across five modalities, not frame-sampled approximations
• Matryoshka truncation anywhere from 128 to 3,072 dimensions; Google recommends 768 as the quality-per-byte sweet spot
• Custom task instructions let you optimise embeddings for a specific retrieval goal
• Strong published video-retrieval results against Nova and Voyage

Limitations

• Public preview, not GA - confirm status and SLA before production commitment
• Vector space is incompatible with gemini-embedding-001; upgrading means re-embedding
• Audio and video pricing is materially higher than text, since those inputs tokenize heavily
• Multimodal support has rolled out unevenly across the Gemini API and Vertex AI surfaces

Choose this when your corpus spans more than two modalities and you want one endpoint rather than an alignment project. Skip this if you need GA stability guarantees today, or your workload is text-only, cheaper models win there.

Pricing: approximately $0.20 per 1M text tokens; separate rates for image, audio and video.

2. Voyage Multimodal 3.5

What it is. Released January 2026, Voyage's next-generation multimodal model - now part of MongoDB - built for retrieval over text, images and video, including content that interleaves all three.

What sets it apart. A single transformer encoder for every modality rather than separate towers. Voyage's stated reason is the modality gap: with dual encoders, a text query drifts toward text results even when an image is the better answer. It's also among the first production video embedding models to support Matryoshka dimensions, which matters enormously when your index is frames.

Strengths

• Handles interleaved inputs natively: PDF screenshots, slides, tables, figures, video frame sequences
• Matryoshka at 2,048 / 1,024 / 512 / 256 plus float32, int8, uint8 and binary quantization
• 32,000-token context, generous by embedding-model standards
• Substantial free tier before any billing starts

Limitations

• API-only, no self-hostable weights, which rules it out for air-gapped or strict-residency deployments
• Pixel-based pricing for visual input takes modelling to forecast
• No audio modality

Choose this when your documents are visually rich and your retrieval quality bar is high. Skip this if data residency or self-hosting is a hard requirement.

Pricing: roughly $0.60 per billion pixels for visual input plus token pricing for text; the first 200M text tokens and 150B pixels are free per account.

3. Cohere Embed v4

What it is. The strongest managed option for enterprise document corpora, taking text, images and mixed content through a single model with a 128K-token context window.

What sets it apart. Robustness on real business material. Tables, charts, diagrams, code and handwritten notes are processed without a preprocessing pipeline, and it tolerates spelling errors and inconsistent formatting. Coverage spans 100+ languages with tuning for finance, healthcare and manufacturing.

Strengths

• 128K context means long documents often need no chunking at all
• Matryoshka truncation plus byte and binary quantization for cheap storage trade-offs
• Available direct, on Amazon Bedrock, on Azure AI Foundry, and via private deployment
• Genuinely strong multilingual quality, not an afterthought

Limitations

• No video or audio
• Self-hostable weights require a separate commercial licence; they are not freely redistributable
• Higher per-token cost than open alternatives

Choose this when your corpus is documents and images, multilingual or regulated, and you would rather buy quality than tune it. Skip this if you need video or audio in the same space.

4. Amazon Nova 2 Multimodal Embeddings

What it is. AWS's unified embedding model on Bedrock, covering text, documents, images, video and audio in one model.

What sets it apart. Four Matryoshka output dimensions: 3,072, 1,024, 384 and 256; from a single embedding, so one model serves four different cost-accuracy points without re-embedding. Context runs to roughly 8,192 tokens across around 200 languages, with both synchronous and asynchronous APIs and built-in segmentation for long text, video or audio.

Strengths

• Five modalities under one Bedrock-native API with IAM, quotas and S3 integration
• Dimension flexibility that maps cleanly onto a two-stage retrieval cascade
• Competitive token pricing
• Async path for large video, sync for small inputs

Limitations

• Segment caps mean long media needs your own chunking and aggregation strategy
• Regional availability is narrower than established text models - confirm before designing around it
• Closed weights, AWS-only
• Vendor benchmark claims are contested by competitors' published numbers

Choose this when you are already on AWS and want one managed model across every modality you hold. Skip this if long-form video understanding is the core job, or you need open weights.

5. TwelveLabs Marengo 3.0

What it is. A video-native embedding model, generally available since December 2025 on both the TwelveLabs API and Amazon Bedrock. One 512-dimensional space covers video, audio, images and text, with support for videos up to four hours.

What sets it apart. Temporal understanding rather than frame averaging. Most "video support" in this category means sampling frames and pooling image embeddings, which discards motion. Marengo encodes the temporal structure, and does it in 512 dimensions, small enough to keep a whole media archive searchable.

Strengths

• Actual video-native encoding, not frame approximation
• Compact 512d embedding: a fraction of the storage cost of 3,072d alternatives
• Native audio retrieval in the same space - speech, music, effects
• Available on Bedrock with regional inference in the US and EU

Limitations

• Headline benchmark figures are vendor-published; independent verification is still thin
• Closed weights, API-only, and Bedrock invocation is asynchronous rather than a synchronous embed call
• Per-minute video pricing compounds quickly on large archives

Choose this when video is your primary modality and motion matters. Skip this if your content is mostly text and documents, or you need synchronous low-latency embedding.

6. Google Vertex AI multimodalembedding@001

What it is. The incumbent Vertex model, producing 1,408-dimensional vectors from images, text or video.

Solid, well documented, and the shortest path from image to indexed vector if your stack already lives on Google Cloud, sitting alongside Vertex AI Vector Search with no egress or auth friction. It is, however, being superseded: no Matryoshka truncation, narrower multilingual coverage, and a smaller modality span than Gemini Embedding 2.

Choose this when you are on GCP with an existing pipeline and no appetite to re-embed. Skip this if you are starting fresh, evaluate Gemini Embedding 2 first.

7. Amazon Titan Multimodal Embeddings G1

What it is. Amazon's first-generation multimodal model, converting images and short English text into a shared space for search by text, image, or a combination.

The constraint to plan around is the text cap, roughly 128 tokens. That is fine for product titles and short captions and inadequate for anything document-shaped. Output supports 1,024, 384 or 256 dimensions. English only.

Choose this when you are on AWS with a short-form catalogue and existing Titan integration. Skip this if your text side is longer than a caption, or you need languages beyond English: Nova 2 supersedes it on both counts.

8. Azure AI Vision multimodal embeddings

What it is. Vectorizes images and text queries into a shared 1,024-dimensional space, letting you search an image set with text without tags or metadata.

Competent and unremarkable. Its case is Azure-native integration - particularly with Azure AI Search - rather than benchmark leadership.

Choose this when you are committed to Azure and want first-party support. Skip this if retrieval quality is the deciding factor.

Best open-source multimodal embedding models in 2026

Self-hosting removes per-request cost and solves data residency, at the price of running GPU infrastructure. The crossover point is higher than most teams expect: below roughly 200–300 million tokens per month, an API is usually cheaper once engineering time and idle GPU capacity are counted honestly. Above it, the maths flips decisively.

9. jina-embeddings-v5-omni

Announced May 2026, this is the strongest open-weight omni family available: text, images, video and audio in one shared space, in two sizes (small at around 1.6B parameters, nano at around 0.9B).

The design choice that makes it interesting is compatibility. The omni models share a vector space with the text-only jina-embeddings-v5-text models, so an existing text index can stay in place while you start writing image, audio and video vectors into it. Multimodal retrieval becomes an upgrade rather than a migration. Matryoshka truncation runs down to 32 dimensions, video is processed as uniformly sampled frames, and the nano variant runs on commodity hardware without a GPU.

The catch: weights are distributed under CC BY-NC 4.0. Commercial use requires a separate arrangement. Budget for that conversation before you build on it.

10. Jina Embeddings v4

Built on a Qwen2.5-VL backbone, processing text and images through a shared pathway with task-specific LoRA adapters for retrieval, text-matching and code. Two output modes matter practically: single-vector at 2,048 dimensions truncatable to 128, and multi-vector at 128 dimensions per token for late-interaction retrieval, which performs notably better on visually complex documents. Accepts text, images and PDFs directly, available both as an API with a free tier and as open weights.

Choose this when you want API convenience now and the option to self-host later.

11. Qwen3-VL-Embedding and GME

Alibaba's line of VLM-derived encoders. GME turns Qwen2-VL-2B into a single-vector any-to-any embedder: a slide, scanned invoice or chart-heavy page embeds as an image, capturing layout without an OCR pre-step. In at least one independent 2026 benchmark, a 2B Qwen3-VL model outperformed several commercial APIs on cross-modal retrieval, which is a useful reminder that open weights no longer implies second tier.

Choose this when you can self-host a 2B VLM and your corpus is visually rich. Skip this if you need audio or video, or the lowest possible per-item latency.

12. ColPali / ColQwen2

A different architecture for a specific job. Instead of OCR → chunk → embed, ColPali embeds page images directly using late-interaction multi-vector retrieval, preserving layout, tables and figures that OCR destroys. If you are building RAG over PDFs, slide decks or scanned reports, this class of model is the current answer - see the section on multi-vector retrieval below for the trade-offs.

13. SigLIP 2

The open-weights default for text-image similarity. Sigmoid loss gives independent, well-calibrated per-pair scores rather than softmax rankings, and the SigLIP 2 recipe adds multilingual coverage across 109 languages plus self-distillation and masked prediction, improving localization and dense features over both the original SigLIP and CLIP. Open weights on Hugging Face from ViT-B up to So400m and a 1B variant.

Choose this when you are self-hosting a cross-modal model and text-plus-image is enough.

14. JinaCLIP v2

Adds two things CLIP lacks: real multilingual support and Matryoshka embeddings that shrink to 64 dimensions for dramatic storage savings. Its 8,192-token context is the longest of any CLIP-style model, making it viable for embedding whole documents alongside images without aggressive chunking.

Choose this over SigLIP 2 when your corpus has substantial non-English text.

15. Nomic Embed Vision

Apache-2.0 text and vision models with quality competitive against proprietary APIs, plus a hosted option with a free tier. The straightforward pick when you want open licensing for residency or cost reasons but don't want to stand up infrastructure on day one. English-centric; no video or audio.

16. ImageBind

Meta's six-modality model: vision, text, audio, depth, thermal and IMU in one space. Uniquely capable and honestly positioned as a research and prototyping tool rather than a production retrieval model; per-modality accuracy lags specialists. Reach for it when you need a modality pairing nothing else supports, such as sensor fusion in robotics.

Video-native open models worth watching

Not full entries, but named because they change what's possible: V-JEPA 2, VideoPrism and InternVideo2 embed clips with temporal structure rather than averaging frames. If you are self-hosting for video and Marengo's closed weights are a blocker, this is where to look.

Single-vector vs multi-vector: when late interaction wins

A single-vector model compresses an entire image, page or clip into one vector. Retrieval is a single dot product: cheap, and entirely adequate for "is this concept present" queries.

A multi-vector, late-interaction model like ColPali or ColQwen keeps one vector per patch or token and scores a query by summing each query token's best match against all document vectors, an operation usually called MaxSim. Nothing spatial or temporal gets averaged away, which is why multi-vector wins decisively on documents, screenshots, dense text-in-image and exact-moment retrieval.

The cost is storage and query complexity. A page that was one vector becomes hundreds, and not every vector database handles late interaction well.

The practical rule: single-vector for broad semantic recall; multi-vector when the system must point to the exact region, page or moment that answers the question. Many production stacks run both - a cheap single-vector first pass, then multi-vector reranking on the top candidates.

What the benchmarks actually measure

Three benchmark families matter here, and they measure different things.

MMEB (Massive Multimodal Embedding Benchmark), including MMEB-v2 with video tasks, is the leaderboard to check for cross-modal work. Its known weakness is hard negatives: the suite doesn't stress subtle distinctions heavily, so models can score well without proving they separate near-misses.

ViDoRe (Visual Document Retrieval) is the one that matters if your documents are visual — PDFs, slides, reports with tables and charts. It is the benchmark the ColPali family was built against, and it correlates far better with real multimodal RAG performance than general leaderboards.

MTEB / MMTEB is the most-cited leaderboard and the least relevant here. The core suite is predominantly text retrieval. A high MTEB score tells you almost nothing about cross-modal performance.

Independent 2026 evaluations have also highlighted dimensions no public leaderboard covers well: cross-lingual retrieval, long-document accuracy, and quality retention under dimension truncation. One vendor-run benchmark specifically constructed around those gaps found that no single model wins everything.

The honest position: if any of those describe your workload, published scores will mislead you. Build a small evaluation set from your own data. Two hundred labelled pairs is enough to separate the top candidates and costs an afternoon.

Multimodal embedding pricing compared

Pricing in this category is quoted in incompatible units: per text token, per image, per pixel, per video minute, per transaction - which makes direct comparison genuinely hard. Three worked examples, using the rates in the table above:

Scenario A: 500,000 mixed business documents (text + embedded figures). Cohere Embed v4's 128K context means most documents embed whole rather than chunked, which is where the saving is. Model this on document count, not page count.

Scenario B: 1,000 hours of video archive. Marengo's per-minute pricing dominates the calculation, but the 512-dimension output means storage stays modest. Compare against a frame-sampling approach on Nova 2 or Voyage 3.5, where you pay per frame as an image and store six times the vector volume.

Scenario C: 10 million short text queries per month against a mixed index. Indexing is a one-time cost; queries are forever. Query text is short and cheap on every model here. Model your query volume, not your corpus size - this is the number teams consistently get backwards.

And the number nobody quotes: storage. One billion 3,072-dimension float32 vectors is roughly 12TB before index overhead. At 768 dimensions it is 3TB. This is why Matryoshka support is a structural cost advantage that per-token pricing completely hides, and why Gemini Embedding 2, Nova 2, Voyage 3.5 and the Jina family have an edge that doesn't show up in a price-per-token table.

Video and audio embeddings: the fastest-moving corner

Until late 2025, embedding video meant sampling frames, running an image model over each, and averaging. That works for "is there a car in this clip" and fails completely for anything involving motion, sequence or causation.

Three approaches now exist:

1. Video-native encoders (Marengo 3.0, V-JEPA 2, VideoPrism, InternVideo2) model temporal structure directly. Best quality, highest cost.
2. Omni models with video input (Gemini Embedding 2, Nova 2, jina-v5-omni, Voyage 3.5) accept video but generally process it as ordered frame sequences with segment caps. Good enough for most retrieval, and it keeps everything in one space.
3. Frame sampling with an image model remains viable for short clips where nothing moves meaningfully.

Audio is less mature. Gemini Embedding 2, Nova 2, Marengo and jina-v5-omni put speech and non-speech audio in the shared space; most other models don't touch it. If audio retrieval matters - call recordings, podcasts, sound effects libraries - that requirement alone narrows the shortlist to four.

How to choose a multimodal embedding model

Work through this in order. Each answer eliminates options before you reach the harder questions.

1. Which modalities do you actually have? Text and images only → any model here. Add video → Gemini Embedding 2, Nova 2, Voyage 3.5, Marengo, jina-v5-omni. Add audio → Gemini Embedding 2, Nova 2, Marengo, jina-v5-omni, ImageBind. Visually complex documents → Cohere Embed v4, Jina v4, ColPali family.

2. Do you have a data-residency or self-hosting constraint? If data cannot leave your infrastructure: eliminate Voyage, Marengo and the Azure/Vertex/Bedrock models entirely, and check licensing carefully on what's left — Cohere requires a commercial licence for weights, jina-v5-omni is non-commercial by default.

3. Single-vector or multi-vector? If retrieval must identify an exact page, region or moment, you need late interaction, and that decides both the model and the vector database.

4. What's your storage budget at target scale? Large corpora → prioritise Matryoshka truncation and quantization. At billion-vector scale this constrains the shortlist harder than accuracy differences do.

5. Then benchmark the surviving two or three on your own data. Two hundred labelled pairs from your actual corpus will tell you more than any leaderboard on this page.

Multimodal embeddings with Eden AI

Choosing a model is one decision. Being able to change it is a different one - and since switching means re-embedding your corpus, the cost of being locked into the wrong choice compounds with every vector you write.

Eden AI provides a single API across multiple multimodal embedding providers with a standardised JSON response format, so switching providers is a parameter change rather than an integration rewrite.

import requests

url = "https://api.edenai.run/v3/embeddings"
headers = {
    "Authorization": "Bearer <your-api-key>",
    "Content-Type": "application/json",
}
payload = {
    "model": "vertex/gemini-embedding-001",
    "input": "a technician inspecting a wind turbine",
}

response = requests.post(url, headers=headers, json=payload)
print(response.json()["data"][0]["embedding"])

‍

In practice that gives you centralised billing across providers, side-by-side comparison of accuracy, latency and cost on your own data, automatic fallback if a provider has an outage, and no data retention, with the option to filter to GDPR-compliant engines only. For European teams, the EU endpoint keeps requests and data within Europe. 

FAQ - Multimodal embedding models and APIs