Struct LlamaModel 

pub struct LlamaModel { /* private fields */ }

A loaded ik_llama.cpp model.

Uses ik’s model-pointer tokenizer/vocab API (llama_tokenize(model, …), llama_token_bos(model), …), which differs from modern stock llama.cpp’s vocab-pointer API.

Implementations

impl LlamaModel

pub fn chat_template(&self, name: Option<&str>) -> Option<LlamaChatTemplate>

Get the chat template from the model by name. If the name parameter is None, the model’s default chat template will be returned.

You supply this into Self::apply_chat_template to get back a string with the appropriate template substitution applied to convert a list of messages into a prompt the LLM can use to complete the chat.

Returns None if the model has no chat template by that name (or if name contains an interior null byte).

You could also use an external jinja parser, like minijinja, to parse jinja templates not supported by the ik_llama.cpp template engine.

pub fn apply_chat_template( &self, tmpl: &LlamaChatTemplate, chat: &[LlamaChatMessage], add_assistant: bool, ) -> Result<String, ApplyChatTemplateError>

Apply a chat template to a list of messages, returning the formatted prompt.

Inspired by hf apply_chat_template() on python. Use Self::chat_template to retrieve the template baked into the model (this is the preferred path), or build one directly with LlamaChatTemplate::new.

add_assistant controls whether the prompt ends with the token(s) that indicate the start of an assistant message.

Errors

There are many ways this can fail. See ApplyChatTemplateError for more information.

impl LlamaModel

pub fn lora_adapter_init( &self, path: &Path, ) -> Result<LlamaLoraAdapter<'_>, LlamaLoraAdapterInitError>

Initializes a lora adapter from a file.

Errors

See LlamaLoraAdapterInitError for more information.

impl LlamaModel

pub fn meta_val_str(&self, key: &str) -> Option<String>

Value of a metadata key (e.g. "general.architecture"), or None if absent.

pub fn meta_count(&self) -> i32

Number of metadata key-value pairs.

pub fn meta_key_by_index(&self, index: i32) -> Option<String>

Metadata key name at index, or None if out of range.

pub fn meta_val_str_by_index(&self, index: i32) -> Option<String>

Metadata value at index, or None if out of range.

pub fn desc(&self) -> String

Human-readable model description.

pub fn size(&self) -> u64

Total model size in bytes.

pub fn n_params(&self) -> u64

Total number of parameters.

pub fn n_ctx_train(&self) -> i32

Training context length.

pub fn n_embd(&self) -> i32

Embedding dimension.

pub fn n_layer(&self) -> i32

Number of layers.

pub fn rope_type(&self) -> llama_rope_type

RoPE type (raw llama_rope_type).

impl LlamaModel

pub fn load_from_file( _backend: &LlamaBackend, path: impl AsRef<Path>, params: &LlamaModelParams, ) -> Result<Self, LlamaError>

Load a model from a single GGUF file (for split models, pass the merged file or the first shard — see the crate docs on SPECIAL_SPLIT).

Invariant

The LlamaBackend passed here (and any derived crate::LlamaContext) must outlive the returned model. Dropping the backend first runs llama_backend_free before the model/context are freed, which is unsound. The natural nested/reverse-drop order (backend → model → context declared outer-to-inner, dropped inner-to-outer) satisfies this. (This mirrors the llama-cpp-2 anchor’s contract.)

pub fn n_vocab(&self) -> i32

Number of vocabulary tokens.

pub fn n_nextn_layer(&self) -> i32

Number of MTP / NextN prediction layers (0 if the model has none).

pub fn token_bos(&self) -> LlamaToken

Beginning-of-sequence token.

pub fn token_eos(&self) -> LlamaToken

End-of-sequence token.

pub fn is_eog(&self, token: LlamaToken) -> bool

Whether token marks end-of-generation (EOS/EOT/etc.).

pub fn is_eog_token(&self, token: LlamaToken) -> bool

Whether token marks end-of-generation (alias of Self::is_eog, matching the llama-cpp-2 method name).

pub fn new_context<'a>( &'a self, _backend: &LlamaBackend, params: LlamaContextParams, ) -> Result<LlamaContext<'a>, LlamaError>

Create a context for this model (matches llama-cpp-2’s model.new_context(&backend, params)).

The LlamaBackend argument is accepted for API parity; the returned context borrows self, so the model must outlive it.

Errors

LlamaError::ContextCreation if llama_init_from_model returns null.

pub fn str_to_token( &self, text: &str, add_bos: AddBos, ) -> Result<Vec<LlamaToken>, LlamaError>

Tokenize text, choosing whether to prepend BOS via AddBos (matches llama-cpp-2’s str_to_token). Special tokens are parsed.

Errors

LlamaError::Nul on an interior NUL byte; LlamaError::Tokenize if tokenization fails.

pub fn tokenize( &self, text: &str, add_bos: bool, ) -> Result<Vec<LlamaToken>, LlamaError>

Tokenize text. add_bos prepends the BOS token; special tokens are parsed.

pub fn token_to_piece_bytes( &self, token: LlamaToken, buf_size: usize, special: bool, lstrip: Option<NonZeroU16>, ) -> Result<Vec<u8>, LlamaError>

Raw bytes of a single token’s piece.

special controls whether special/control tokens render as text; lstrip strips that many leading spaces. buf_size is the initial buffer guess — on overflow the call retries once with the exact size.

Errors

LlamaError::TokenOutOfRange if token is not in [0, n_vocab); LlamaError::TokenToPiece if the C conversion fails.

pub fn token_to_piece( &self, token: LlamaToken, decoder: &mut Decoder, special: bool, lstrip: Option<NonZeroU16>, ) -> Result<String, LlamaError>

Convert a single token to text, decoding its bytes incrementally through decoder (so a multi-byte UTF-8 sequence split across token boundaries is reassembled). Matches llama-cpp-2’s token_to_piece.

special renders special/control tokens as text; lstrip strips leading spaces. Use one decoder (encoding_rs::UTF_8.new_decoder()) per stream.

A multi-byte sequence split at the very end of the stream stays buffered in decoder (this never flushes with last = true); that matches the llama-cpp-2 anchor and is a non-issue for a stream ending on a token boundary.

Errors

LlamaError::TokenToPiece if the C conversion fails.

pub fn token_to_piece_lossy( &self, token: LlamaToken, ) -> Result<String, LlamaError>

Convert a single token to its text piece, lossily (UTF-8 replacement on invalid bytes). Convenience over Self::token_to_piece for callers not doing incremental streaming.

Errors

LlamaError::TokenToPiece if the C conversion fails.

pub fn detokenize(&self, tokens: &[LlamaToken]) -> Result<String, LlamaError>

Detokenize a slice of tokens into a string, decoding incrementally through a single UTF-8 decoder (special = false).

Note: this concatenates per-token pieces, so special tokens are dropped and spacing may differ slightly from a true detokenizer. ik only exposes the vocab-pointer llama_detokenize (the model-pointer overload is commented out in the header); a full detokenizer path is a follow-up.

Trait Implementations

impl Debug for LlamaModel

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Drop for LlamaModel

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.

impl Send for LlamaModel

impl Sync for LlamaModel