fastembed 6.1.0

Library for generating vector embeddings, reranking locally.
Documentation

Features

Not looking for Rust?

Supported Models

Text Embedding

Quantized versions are also available for several models above (append Q to the model enum variant, e.g., EmbeddingModel::BGESmallENV15Q). EmbeddingGemma additionally ships a 4-bit build as EmbeddingModel::EmbeddingGemma300MQ4.

Sparse Text Embedding

Image Embedding

Reranking

✊ Support

To support the library, please donate to our primary upstream dependency, ort - The Rust wrapper for the ONNX runtime.

Installation

Run the following in your project directory:

cargo add fastembed

Or add the following line to your Cargo.toml:

[dependencies]
fastembed = "5"

Text Embeddings

use fastembed::{TextEmbedding, TextInitOptions, EmbeddingModel};

// With default options
let mut model = TextEmbedding::try_new(Default::default())?;

// With custom options
let mut model = TextEmbedding::try_new(
    TextInitOptions::new(EmbeddingModel::AllMiniLML6V2).with_show_download_progress(true).with_intra_threads(4),
)?;

let documents = vec![
    "passage: Hello, World!",
    "query: Hello, World!",
    "passage: This is an example passage.",
    // You can leave out the prefix but it's recommended
    "fastembed-rs is licensed under Apache 2.0"
];

 // Generate embeddings with the default batch size, 256
 let embeddings = model.embed(documents, None)?;

 println!("Embeddings length: {}", embeddings.len()); // -> Embeddings length: 4
 println!("Embedding dimension: {}", embeddings[0].len()); // -> Embedding dimension: 384

Sparse Text Embeddings

use fastembed::{SparseEmbedding, SparseInitOptions, SparseModel, SparseTextEmbedding};

// With default options
let mut model = SparseTextEmbedding::try_new(Default::default())?;

// With custom options
let mut model = SparseTextEmbedding::try_new(
    SparseInitOptions::new(SparseModel::SPLADEPPV1).with_show_download_progress(true),
)?;

let documents = vec![
    "passage: Hello, World!",
    "query: Hello, World!",
    "passage: This is an example passage.",
    "fastembed-rs is licensed under Apache 2.0"
];

// Generate embeddings with the default batch size, 256
let embeddings: Vec<SparseEmbedding> = model.embed(documents, None)?;

Inference-free Sparse Embeddings

SparseModel::OpenSearchNeuralSparseDocV3Gte is asymmetric: documents are expanded by the ONNX encoder, while queries are embedded by query_embed from the tokenizer and a precomputed IDF table alone, without any inference. Both sides are compared with a dot product.

use fastembed::{SparseInitOptions, SparseModel, SparseTextEmbedding};

let mut model = SparseTextEmbedding::try_new(
    SparseInitOptions::new(SparseModel::OpenSearchNeuralSparseDocV3Gte).with_max_length(8192),
)?;

//  This model emits one score per vocabulary entry per token.
// So keep the batch size small.
let documents = model.embed(vec!["Hello World"], Some(4))?;

let queries = model.query_embed(vec!["Hello World"])?;

representation.

Image Embeddings

use fastembed::{ImageEmbedding, ImageInitOptions, ImageEmbeddingModel};

// With default options
let mut model = ImageEmbedding::try_new(Default::default())?;

// With custom options
let mut model = ImageEmbedding::try_new(
    ImageInitOptions::new(ImageEmbeddingModel::ClipVitB32).with_show_download_progress(true),
)?;

let images = vec!["assets/image_0.png", "assets/image_1.png"];

// Generate embeddings with the default batch size, 256
let embeddings = model.embed(images, None)?;

println!("Embeddings length: {}", embeddings.len()); // -> Embeddings length: 2
println!("Embedding dimension: {}", embeddings[0].len()); // -> Embedding dimension: 512

Candidates Reranking

use fastembed::{TextRerank, RerankInitOptions, RerankerModel};

// With default options
let mut model = TextRerank::try_new(Default::default())?;

// With custom options
let mut model = TextRerank::try_new(
    RerankInitOptions::new(RerankerModel::BGERerankerBase).with_show_download_progress(true),
)?;

let documents = vec![
    "hi",
    "The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear, is a bear species endemic to China.",
    "panda is animal",
    "i dont know",
    "kind of mammal",
];

// Rerank with the default batch size, 256 and return document contents
let results = model.rerank("what is panda?", documents, true, None)?;
println!("Rerank result: {:?}", results);

Locally Available Models

Alternatively, local model files can be used for inference via the try_new_from_user_defined(...) methods of respective structs.

Similarity Search

Helpers in the similarity module score and rank the vectors embed returns, so a quick in-memory search needs no extra crate:

use fastembed::similarity::{cosine_similarity, top_k};

// `embeddings` is the Vec<Embedding> from model.embed(...)
let query = &embeddings[0];

// Score two vectors directly ([-1.0, 1.0], higher = closer)
let score = cosine_similarity(query, &embeddings[1]);

// Or rank the corpus: (index, score) pairs, best first
let hits = top_k(query, &embeddings, 5);
println!("Closest: {:?}", hits);

For larger corpora or persistence, push the vectors to a vector search engine (e.g. Qdrant) and query there.

Qwen3 Embeddings

Qwen3 embedding models are available behind the qwen3 feature flag (candle backend).

[dependencies]
fastembed = { version = "5", features = ["qwen3"] }

use candle_core::{DType, Device};
use fastembed::Qwen3TextEmbedding;

let device = Device::Cpu;
let model = Qwen3TextEmbedding::from_hf(
    "Qwen/Qwen3-Embedding-0.6B",
    &device,
    DType::F32,
    512,
)?;

// Text-only usage with the Qwen3-VL embedding checkpoint is also supported:
// let model = Qwen3TextEmbedding::from_hf("Qwen/Qwen3-VL-Embedding-2B", &device, DType::F32, 512)?;

let embeddings = model.embed(&["query: ...", "passage: ..."])?;
println!("Embeddings length: {}", embeddings.len());

For multimodal text/image usage with Qwen/Qwen3-VL-Embedding-2B:

use candle_core::{DType, Device};
use fastembed::Qwen3VLEmbedding;

let device = Device::Cpu;
let model = Qwen3VLEmbedding::from_hf(
    "Qwen/Qwen3-VL-Embedding-2B",
    &device,
    DType::F32,
    2048,
)?;

let image_embeddings = model.embed_images(&["tests/assets/image_0.png", "tests/assets/image_1.png"])?;
let text_embeddings = model.embed_texts(&["query: blue cat", "query: red cat"])?;

println!("Image embeddings: {}", image_embeddings.len());
println!("Text embeddings: {}", text_embeddings.len());

Nomic Embed Text v2 MoE

The nomic-embed-text-v2-moe model is available behind the nomic-v2-moe feature flag (candle backend). First general-purpose MoE embedding model with 100+ language support.

[dependencies]
fastembed = { version = "5", features = ["nomic-v2-moe"] }

use candle_core::{DType, Device};
use fastembed::NomicV2MoeTextEmbedding;

let device = Device::Cpu;
let model = NomicV2MoeTextEmbedding::from_hf(
    "nomic-ai/nomic-embed-text-v2-moe",
    &device,
    DType::F32,
    512,
)?;

let embeddings = model.embed(&["search_query: ...", "search_document: ..."])?;
println!("Embeddings length: {}", embeddings.len());

BGE-M3 Joint Embeddings

The BGE-M3 model produces dense, sparse, and ColBERT embeddings simultaneously in a single forward pass.

use fastembed::{Bgem3Embedding, Bgem3InitOptions, Bgem3Model};

// With default options
let mut model = Bgem3Embedding::try_new(Default::default())?;

// With custom options (supporting custom max length up to 8192 tokens)
let mut model = Bgem3Embedding::try_new(
    Bgem3InitOptions::new(Bgem3Model::BGEM3Q)
        .with_max_length(1024)
        .with_show_download_progress(true),
)?;

let documents = vec![
    "Hello, World!",
    "This is an example passage.",
    "fastembed-rs is licensed under Apache 2.0",
    "i dont know"
];

// Generate all three representations in a single forward pass
let output = model.embed(documents, None)?;

println!("Dense dimension: {}", output.dense[0].len()); // -> Dense dimension: 1024

let sparse_emb = &output.sparse[0];
println!("Sparse non-zero tokens: {}", sparse_emb.indices.len());

println!("ColBERT token count: {}", output.colbert[0].len());

Model cache

Models download on first use and load from cache afterwards (no network needed at runtime once cached).

  • FASTEMBED_CACHE_DIR — cache location (default: .fastembed_cache). Equivalent to TextInitOptions::with_cache_dir.
  • HF_HOME — if set, takes precedence over the above.
  • HF_ENDPOINT — Hugging Face mirror base URL, for restricted networks.

DirectML (Windows)

To run models on a GPU via DirectML on Windows, enable the directml feature:

[dependencies]
fastembed = { version = "5", features = ["directml"] }

Then pass a DirectML execution provider when initializing a model:

use fastembed::{TextEmbedding, TextInitOptions, EmbeddingModel};
use ort::ep::DirectML;

let model = TextEmbedding::try_new(
    TextInitOptions::new(EmbeddingModel::AllMiniLML6V2)
        .with_execution_providers(vec![DirectML::default().into()]),
)?;

When DirectML is detected, fastembed automatically disables memory pattern optimization and parallel execution on the ONNX Runtime session, as required by the DirectML execution provider.

Error handling

Fastembed returns a typed fastembed::Error. The type is re-exported from the crate root. The enum is #[non_exhaustive]. New variants can be added in minor releases without breaking match arms.

use fastembed::{Error, Result, TextEmbedding};

fn load() -> Result<TextEmbedding> {
    let model = TextEmbedding::try_new(Default::default())?;
    Ok(model)
    // ...
}

To handle an error, match on the variant that applies:

use fastembed::{Error, TextEmbedding, TextInitOptions, EmbeddingModel};

match TextEmbedding::try_new(TextInitOptions::new(EmbeddingModel::AllMiniLML6V2)) {
    Ok(model) => { /* ... */ }
    Err(Error::ModelRetrieval { file, source }) => {
        eprintln!("could not fetch {file}: {source}");
    }
    /*
    ...
    */
    Err(Error::Ort(err)) => {
        eprintln!("ONNX runtime error: {err}");
    }
    Err(e) => eprintln!("{e}"),
}

LICENSE

Apache 2.0