fastembed-rs README

The project's README from github.com/anush008/fastembed-rs. As one markdown file: README.md.

FastEmbed-rs 🦀

Rust library for generating vector embeddings, reranking locally!

Powered by Atlas Cloud

Features

Not looking for Rust?

Supported Models

Text Embedding

Click to list models

Quantized versions are also available for several models above (append Q to the model enum variant, e.g., EmbeddingModel::BGESmallENV15Q). EmbeddingGemma additionally ships a 4-bit build as EmbeddingModel::EmbeddingGemma300MQ4.

Sparse Text Embedding

Click to list models

Image Embedding

Click to list models

Reranking

Click to list models

✊ Support

To support the library, please donate to our primary upstream dependency, ort - The Rust wrapper for the ONNX runtime.

Installation

Run the following in your project directory:

cargo add fastembed

Or add the following line to your Cargo.toml:

[dependencies]
fastembed = "7"

Text Embeddings

use fastembed::{TextEmbedding, TextInitOptions, EmbeddingModel};

// With default options
let mut model = TextEmbedding::try_new(Default::default())?;

// With custom options
let mut model = TextEmbedding::try_new(
    TextInitOptions::new(EmbeddingModel::AllMiniLML6V2).with_show_download_progress(true).with_intra_threads(4),
)?;

let documents = vec![
    "passage: Hello, World!",
    "query: Hello, World!",
    "passage: This is an example passage.",
    // You can leave out the prefix but it's recommended
    "fastembed-rs is licensed under Apache 2.0"
];

 // Generate embeddings with the default batch size, 256
 let embeddings = model.embed(documents, None)?;

 println!("Embeddings length: {}", embeddings.len()); // -> Embeddings length: 4
 println!("Embedding dimension: {}", embeddings[0].len()); // -> Embedding dimension: 384

Sparse Text Embeddings

use fastembed::{SparseEmbedding, SparseInitOptions, SparseModel, SparseTextEmbedding};

// With default options
let mut model = SparseTextEmbedding::try_new(Default::default())?;

// With custom options
let mut model = SparseTextEmbedding::try_new(
    SparseInitOptions::new(SparseModel::SPLADEPPV1).with_show_download_progress(true),
)?;

let documents = vec![
    "passage: Hello, World!",
    "query: Hello, World!",
    "passage: This is an example passage.",
    "fastembed-rs is licensed under Apache 2.0"
];

// Generate embeddings with the default batch size, 256
let embeddings: Vec<SparseEmbedding> = model.embed(documents, None)?;

Image Embeddings

use fastembed::{ImageEmbedding, ImageInitOptions, ImageEmbeddingModel};

// With default options
let mut model = ImageEmbedding::try_new(Default::default())?;

// With custom options
let mut model = ImageEmbedding::try_new(
    ImageInitOptions::new(ImageEmbeddingModel::ClipVitB32).with_show_download_progress(true),
)?;

let images = vec!["assets/image_0.png", "assets/image_1.png"];

// Generate embeddings with the default batch size, 256
let embeddings = model.embed(images, None)?;

println!("Embeddings length: {}", embeddings.len()); // -> Embeddings length: 2
println!("Embedding dimension: {}", embeddings[0].len()); // -> Embedding dimension: 512

Candidates Reranking

use fastembed::{TextRerank, RerankInitOptions, RerankerModel};

// With default options
let mut model = TextRerank::try_new(Default::default())?;

// With custom options
let mut model = TextRerank::try_new(
    RerankInitOptions::new(RerankerModel::BGERerankerBase).with_show_download_progress(true),
)?;

let documents = vec![
    "hi",
    "The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear, is a bear species endemic to China.",
    "panda is animal",
    "i dont know",
    "kind of mammal",
];

// Rerank with the default batch size, 256 and return document contents
let results = model.rerank("what is panda?", documents, true, None)?;
println!("Rerank result: {:?}", results);

Locally Available Models

Alternatively, local model files can be used for inference via the try_new_from_user_defined(...) methods of the respective structs.

Helpers in the similarity module score and rank the vectors embed returns, so a quick in-memory search needs no extra crate:

use fastembed::similarity::{cosine_similarity, top_k};

// `embeddings` is the Vec<Embedding> from model.embed(...)
let query = &embeddings[0];

// Score two vectors directly ([-1.0, 1.0], higher = closer)
let score = cosine_similarity(query, &embeddings[1]);

// Or rank the corpus: (index, score) pairs, best first
let hits = top_k(query, &embeddings, 5);
println!("Closest: {:?}", hits);

For larger corpora or persistence, push the vectors to a vector search engine (e.g. Qdrant) and query there.

Inference-free Sparse Embeddings

use fastembed::{SparseInitOptions, SparseModel, SparseTextEmbedding};

let mut model = SparseTextEmbedding::try_new(
    SparseInitOptions::new(SparseModel::OpenSearchNeuralSparseDocV3Gte).with_max_length(8192),
)?;

//  This model emits one score per vocabulary entry per token.
// So keep the batch size small.
let documents = model.embed(vec!["Hello World"], Some(4))?;

// Queries are from the tokenizer and a precomputed IDF table alone, without any inference.
let queries = model.query_embed(vec!["Hello World"])?;

Qwen3 Embeddings

Qwen3 embedding models are available behind the qwen3 feature flag (candle backend).

[dependencies]
fastembed = { version = "7", features = ["qwen3"] }

use candle_core::{DType, Device};
use fastembed::Qwen3TextEmbedding;

let device = Device::Cpu;
let model = Qwen3TextEmbedding::from_hf(
    "Qwen/Qwen3-Embedding-0.6B",
    &device,
    DType::F32,
    512,
)?;

// Text-only usage with the Qwen3-VL embedding checkpoint is also supported:
// let model = Qwen3TextEmbedding::from_hf("Qwen/Qwen3-VL-Embedding-2B", &device, DType::F32, 512)?;

let embeddings = model.embed(&["query: ...", "passage: ..."])?;
println!("Embeddings length: {}", embeddings.len());

For multimodal text/image usage with Qwen/Qwen3-VL-Embedding-2B:

use candle_core::{DType, Device};
use fastembed::Qwen3VLEmbedding;

let device = Device::Cpu;
let model = Qwen3VLEmbedding::from_hf(
    "Qwen/Qwen3-VL-Embedding-2B",
    &device,
    DType::F32,
    2048,
)?;

let image_embeddings = model.embed_images(&["tests/assets/image_0.png", "tests/assets/image_1.png"])?;
let text_embeddings = model.embed_texts(&["query: blue cat", "query: red cat"])?;

println!("Image embeddings: {}", image_embeddings.len());
println!("Text embeddings: {}", text_embeddings.len());

Nomic Embed Text v2 MoE

The nomic-embed-text-v2-moe model is available behind the nomic-v2-moe feature flag (candle backend). First general-purpose MoE embedding model with 100+ language support.

[dependencies]
fastembed = { version = "7", features = ["nomic-v2-moe"] }

use candle_core::{DType, Device};
use fastembed::NomicV2MoeTextEmbedding;

let device = Device::Cpu;
let model = NomicV2MoeTextEmbedding::from_hf(
    "nomic-ai/nomic-embed-text-v2-moe",
    &device,
    DType::F32,
    512,
)?;

let embeddings = model.embed(&["search_query: ...", "search_document: ..."])?;
println!("Embeddings length: {}", embeddings.len());

BGE-M3 Joint Embeddings

The BGE-M3 model produces dense, sparse, and ColBERT embeddings simultaneously in a single forward pass.

use fastembed::{Bgem3Embedding, Bgem3InitOptions, Bgem3Model};

// With default options
let mut model = Bgem3Embedding::try_new(Default::default())?;

// With custom options (supporting custom max length up to 8192 tokens)
let mut model = Bgem3Embedding::try_new(
    Bgem3InitOptions::new(Bgem3Model::BGEM3Q)
        .with_max_length(1024)
        .with_show_download_progress(true),
)?;

let documents = vec![
    "Hello, World!",
    "This is an example passage.",
    "fastembed-rs is licensed under Apache 2.0",
    "i dont know"
];

// Generate all three representations in a single forward pass
let output = model.embed(documents, None)?;

println!("Dense dimension: {}", output.dense[0].len()); // -> Dense dimension: 1024

let sparse_emb = &output.sparse[0];
println!("Sparse non-zero tokens: {}", sparse_emb.indices.len());

println!("ColBERT token count: {}", output.colbert[0].len());

Model cache

Models download on first use and load from cache afterwards (no network needed at runtime once cached).

DirectML (Windows)

To run models on a GPU via DirectML on Windows, enable the directml feature:

[dependencies]
fastembed = { version = "7", features = ["directml"] }

Then pass a DirectML execution provider when initializing a model:

use fastembed::{TextEmbedding, TextInitOptions, EmbeddingModel};
use ort::ep::DirectML;

let model = TextEmbedding::try_new(
    TextInitOptions::new(EmbeddingModel::AllMiniLML6V2)
        .with_execution_providers(vec![DirectML::default().into()]),
)?;

When DirectML is detected, fastembed automatically disables memory pattern optimization and parallel execution on the ONNX Runtime session, as required by the DirectML execution provider.

Error handling

Fastembed returns a typed fastembed::Error. The type is re-exported from the crate root. The enum is #[non_exhaustive]. New variants can be added in minor releases without breaking match arms.

use fastembed::{Error, Result, TextEmbedding};

fn load() -> Result<TextEmbedding> {
    let model = TextEmbedding::try_new(Default::default())?;
    Ok(model)
    // ...
}

To handle an error, match on the variant that applies:

use fastembed::{Error, TextEmbedding, TextInitOptions, EmbeddingModel};

match TextEmbedding::try_new(TextInitOptions::new(EmbeddingModel::AllMiniLML6V2)) {
    Ok(model) => { /* ... */ }
    Err(Error::ModelRetrieval { file, source }) => {
        eprintln!("could not fetch {file}: {source}");
    }
    /*
    ...
    */
    Err(Error::Ort(err)) => {
        eprintln!("ONNX runtime error: {err}");
    }
    Err(e) => eprintln!("{e}"),
}

LICENSE

Apache 2.0

fastembed-rs (anush008/fastembed-rs)

Searching Database…

1K ·GitHub
Fastembed-rs
What is this?A Rust library that generates vector embeddings and reranks results entirely locally — no external API calls. It covers dense and sparse text embeddings, image embeddings, multimodal models, and rerankers.
Is this a ready-to-run app or a library for my own code?A library (a Rust crate) you add to your project, not a standalone app. If you are not on Rust, sibling ports exist: fastembed for Python, fastembed-go, and fastembed-js.
How do I add it to a Rust project?Run 'cargo add fastembed' in your project directory (or add the fastembed line to Cargo.toml), then initialize a model like TextEmbedding. Weights are fetched automatically on first use.
Do I have to download model weights myself?No — models download from Hugging Face on first use and load from cache afterwards, so no network is needed at runtime. Cache location and HF mirror are controlled via env vars like FASTEMBED_CACHE_DIR, HF_HOME and HF_ENDPOINT.
Are all the listed models usable right away?Most work with default features, but the Qwen3 embedding family and nomic-embed-text-v2-moe are gated behind cargo feature flags (qwen3, nomic-v2-moe) that switch inference to the candle backend.
Can it run on my GPU instead of the CPU?Yes, it runs on ONNX Runtime, and on Windows you can enable GPU execution via DirectML by turning on the directml feature and passing a DirectML execution provider when initializing the model.
Why run embeddings locally instead of calling an API?Everything executes on your machine via ONNX Runtime — after the first model download it needs no network at runtime, your text and images never leave the box, and the library is free under Apache-2.0 with no usage limits.
What do I get by default if I configure nothing?Sensible defaults are built in: BAAI/bge-small-en-v1.5 for text embeddings, prithivida/Splade++ for sparse, Qdrant/clip-ViT-B-32-vision for images, and BAAI/bge-reranker-base for reranking.
Is it async? Do I need Tokio in my project?No — usage is fully synchronous with no dependency on Tokio. Under the hood it uses ort for ONNX inference and Hugging Face tokenizers for fast encodings.
Which models should I pick for multilingual or Chinese text?Multilingual coverage comes from intfloat/multilingual-e5, BAAI/bge-small-zh and bge-large-zh, paraphrase-multilingual-mpnet, and BGE-M3. nomic-embed-text-v2-moe is described as supporting 100+ languages.
What are similar projects?memvid, FlagEmbedding, LLMSearchRecommender, turbovec, txtai, WeMM-Embedding, GLiClass, clip-as-service
Which topics is it in?reranking, tokenizer, retrieval, retrieval-augmented-generation, vector-search, onnx, semantic-search, embeddings, quantization, nlp, rag, inference
Creator
anush008
Size
1 MB
Stars
1K
Updated
CommitOct 12Releasev7.1.1Oct 07
Age
3y
Commits · 90 Days