Struct EmbeddingClient 

pub struct EmbeddingClient { /* private fields */ }

The llama.cpp embedding client.

EmbeddingClient loads a GGUF embedding model on a dedicated worker thread and exposes it through Rig’s rig::embeddings::EmbeddingModel trait. Create one with EmbeddingClient::from_gguf.

use rig::embeddings::EmbeddingModel;

let client = rig_llama_cpp::EmbeddingClient::from_gguf(
    "path/to/embedding-model.gguf",
    99,   // n_gpu_layers
    8192, // n_ctx
)?;
let model = client.embedding_model("local");
let embedding = model.embed_text("Hello, world!").await?;
println!("dims: {}", embedding.vec.len());

Implementations

impl EmbeddingClient

pub fn from_gguf( model_path: impl Into<String>, n_gpu_layers: u32, n_ctx: u32, ) -> Result<Self, LoadError>

Load a GGUF embedding model and start the embedding worker thread.

Arguments

• model_path — Path to a .gguf embedding model file.
• n_gpu_layers — Number of layers to offload to the GPU (u32::MAX for all).
• n_ctx — Context window size in tokens.

Errors

Returns a LoadError if the backend fails to initialise or the model cannot be loaded.

pub fn embedding_model(&self, model: impl Into<String>) -> EmbeddingModelHandle

Create an embedding model handle from this client.

pub fn embedding_model_with_ndims( &self, model: impl Into<String>, ndims: usize, ) -> EmbeddingModelHandle

Create an embedding model handle with explicit dimensions.

Trait Implementations

impl Drop for EmbeddingClient

fn drop(&mut self)

Executes the destructor for this type.