Struct EmbeddingModel 

pub struct EmbeddingModel { /* private fields */ }

A loaded embedding model: tokenizer + encoder + the checkpoint’s own pooling rule.

Implementations

impl EmbeddingModel

pub fn from_gguf_path(path: impl AsRef<Path>) -> Result<Self, EmbedError>

Opens path and builds whichever embedding stack its general.architecture names, or refuses naming what is missing.

pub fn architecture(&self) -> &str

pub fn name(&self) -> &str

The checkpoint’s general.name, or its architecture when the file carries none. What /v1/embeddings reports as model.

pub fn n_embd(&self) -> usize

pub fn n_ctx_train(&self) -> usize

pub fn pooling_type(&self) -> PoolingType

pub fn token_ids(&self, text: &str) -> Vec<u32>

The exact ids the encoder will see for text: the tokenizer’s pieces wrapped in the model’s own special tokens. Public because /v1/embeddings has to report usage.prompt_tokens, and that number is this length — llama.cpp counts the specials too.

pub fn embed(&self, text: &str, normalize: bool) -> Result<Vec<f32>, EmbedError>

Pooled embedding for text. normalize applies L2 normalization, which is what an OpenAI-compatible /v1/embeddings response is expected to carry and what llama.cpp’s server does by default; the raw pooled vector is what the graph produced.

pub fn hidden_states(&self, text: &str) -> Result<Vec<f32>, EmbedError>

Un-pooled n_tokens × n_embd hidden states, for a caller that wants to pool differently (or not at all).

pub fn rank_head(&self) -> Option<&RankHead>

The checkpoint’s reranker classification head, or None for a plain embedding model. What /v1/rerank checks before it promises a caller a relevance score.

pub fn rerank_token_ids( &self, query: &str, document: &str, ) -> Result<Vec<u32>, EmbedError>

The exact ids Self::rerank_score will see for one (query, document) pair: [CLS] query [SEP] document [SEP].

Separate from the scoring call for the same reason Self::token_ids is separate from Self::embed — a route has to report usage.prompt_tokens, and that number is this length.

pub fn rerank_score(&self, pair_ids: &[u32]) -> Result<f32, EmbedError>

The head’s relevance score for a pair sequence built by Self::rerank_token_ids.

This is upstream’s RANK path in full: encode, take the CLS row, run the classification head, report output 0 (send_rerank’s embd[0]). The CLS row is taken here regardless of what {arch}.pooling_type says, because the head was trained on that position — pooling_type = RANK is the checkpoint declaring this path, not naming a pooling rule, which is why crate::pooling::pool still refuses RANK and must keep refusing it.

No L2 normalization and no sigmoid: upstream reports the raw logit, so a score is comparable only against other scores from the same head, and this must not quietly squash it into 0..1.