Struct MtmdContext 

pub struct MtmdContext {
    pub context: NonNull<mtmd_context>,
}

Safe wrapper around mtmd_context.

This represents an initialized multimodal context that can process text, images, and audio through llama.cpp’s multimodal interface.

Fields

context: NonNull<mtmd_context>

Raw pointer to the underlying mtmd_context.

Implementations

impl MtmdContext

pub fn init_from_file( mmproj_path: &str, text_model: &LlamaModel, params: &MtmdContextParams, ) -> Result<Self, MtmdInitError>

Initialize MTMD context from a multimodal projection file.

Errors

This function will return an error if:

• The path cannot be converted to a C string
• The underlying C function returns null (indicating initialization failure)

pub fn decode_use_non_causal(&self, chunk: &MtmdInputChunk) -> bool

Check whether non-causal attention mask is needed before llama_decode for the given input chunk.

pub fn decode_use_mrope(&self) -> bool

Check whether the current model uses M-RoPE for llama_decode.

pub fn support_vision(&self) -> bool

Check whether the current model supports vision input.

pub fn support_audio(&self) -> bool

Check whether the current model supports audio input.

pub fn get_audio_sample_rate(&self) -> Option<u32>

Get audio sample rate in Hz (e.g., 16000 for Whisper). Returns None if audio is not supported.

pub fn tokenize( &self, text: MtmdInputText, bitmaps: &[&MtmdBitmap], ) -> Result<MtmdInputChunks, MtmdTokenizeError>

Tokenize input text and bitmaps into chunks.

The input text must contain media markers (default: <__media__>) that will be replaced with the corresponding bitmap data from the bitmaps array. The number of bitmaps must equal the number of markers in the text.

Errors

• BitmapCountMismatch - Number of bitmaps doesn’t match number of markers
• ImagePreprocessingError - Error occurred during image preprocessing
• UnknownError - Other tokenization error occurred

Example

let text = MtmdInputText {
    text: "Here is an image: <__media__>\nDescribe it.".to_string(),
    add_special: true,
    parse_special: true,
};
let chunks = ctx.tokenize(text, &[bitmap])?;

pub fn encode_chunk( &self, chunk: &MtmdInputChunk, ) -> Result<(), MtmdEncodeError>

Encode a chunk for image/audio processing.

Errors

Returns MtmdEncodeError::EncodeFailure if encoding fails.

Trait Implementations

impl Debug for MtmdContext

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Drop for MtmdContext

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.

impl Send for MtmdContext

impl Sync for MtmdContext