Struct EmbeddingBackend 

pub struct EmbeddingBackend {
    pub model: EmbeddingModelWeights,
    pub tokenizer: Tokenizer,
    pub device: Device,
}

A loaded embedding model ready for inference.

Fields

model: EmbeddingModelWeights

tokenizer: Tokenizer

device: Device

Implementations

impl EmbeddingBackend

pub fn load(model_dir: &Path, device: Device) -> Result<Self, InferenceError>

Load a Qwen3-Embedding GGUF model + tokenizer from a directory.

pub fn encode(&self, text: &str) -> Result<Vec<u32>, InferenceError>

Encode text to token IDs.

pub fn embed_one(&mut self, text: &str) -> Result<Vec<f32>, InferenceError>

Embed a single text. Returns L2-normalized hidden state vector.

pub fn embed_batch( &mut self, texts: &[String], ) -> Result<Vec<Vec<f32>>, InferenceError>

Embed multiple texts. Processes sequentially (no batch padding needed).

pub fn embed_query( &mut self, text: &str, instruction: &str, ) -> Result<Vec<f32>, InferenceError>

Embed with instruction prefix (for queries). Format: “Instruct: {instruction}\nQuery:{text}”