Expand description
Embeddings + reranker support: the llama_get_embeddings* accessors.
Embeddings are produced when the context is built with
LlamaContextParams::with_embeddings(true) and (usually) a pooling type set
via with_pooling_type. Fetch a per-token vector with
LlamaContext::embeddings_ith or a pooled per-sequence vector with
LlamaContext::embeddings_seq_ith; each slice is n_embd long.
A reranker is just an embedding model run with a rank/classification
pooling head: the pooled “embedding” of a query+document sequence is a
relevance score. Read it with LlamaContext::embeddings_seq_ith after a
decode (with a rank pooling type, the slice collapses to the score). ik does
not expose a dedicated LLAMA_POOLING_TYPE_RANK constant — the available
pooling types are NONE, MEAN, CLS, and LAST.
Enums§
- Embeddings
Error - Errors returned by the embedding accessors.