Struct MtmdContextParams 

pub struct MtmdContextParams { /* private fields */ }

Parameters used when creating an MtmdContext.

Obtain a default-initialised instance via MtmdContextParams::default().

Implementations

impl MtmdContextParams

pub fn use_gpu(self, v: bool) -> Self

Whether to run the vision/audio encoder on the GPU (default: true).

pub fn print_timings(self, v: bool) -> Self

Whether to print timing info after each encode (default: false).

pub fn n_threads(self, n: i32) -> Self

Number of threads used for the vision encoder (default taken from mtmd_context_params_default).

pub fn warmup(self, v: bool) -> Self

Whether to run a warm-up encode pass after initialisation.

pub fn image_min_tokens(self, n: i32) -> Self

Minimum number of image tokens (0 = use model default).

pub fn image_max_tokens(self, n: i32) -> Self

Maximum number of image tokens (0 = use model default).

pub fn with_batch_max_tokens(self, n: i32) -> Self

Maximum number of multimodal output tokens per batch.

Maps to mtmd_context_params.batch_max_tokens. The upstream default is 1024. Increase for large images or long audio segments.

Examples

use llama_cpp_4::mtmd::MtmdContextParams;
let params = MtmdContextParams::default().with_batch_max_tokens(2048);
assert_eq!(params.batch_max_tokens(), 2048);

pub fn batch_max_tokens(&self) -> i32

Get the configured batch token cap (batch_max_tokens).

pub fn with_flash_attn_type(self, flash_attn_type: LlamaFlashAttnType) -> Self

Set flash-attention mode for the vision encoder.

Maps to mtmd_context_params.flash_attn_type. Uses the same crate::context::params::LlamaFlashAttnType enum as text contexts.

Examples

use llama_cpp_4::context::params::LlamaFlashAttnType;
use llama_cpp_4::mtmd::MtmdContextParams;
let params = MtmdContextParams::default()
    .with_flash_attn_type(LlamaFlashAttnType::Auto);
assert_eq!(params.flash_attn_type(), LlamaFlashAttnType::Auto);

pub fn flash_attn_type(&self) -> LlamaFlashAttnType

Get flash-attention mode for the vision encoder.

pub fn with_progress_callback( self, callback: Option<MtmdProgressCallback>, user_data: *mut c_void, ) -> Self

Register a callback invoked while mmproj weights load.

Maps to mtmd_context_params.progress_callback. Pass None to disable progress reporting. The callback may return false to abort loading early; see MtmdProgressCallback.

user_data is forwarded to each invocation and must remain valid until MtmdContext::init_from_file returns.

pub fn media_marker(self, marker: Option<&str>) -> Result<Self, MtmdError>

Override the media marker string (e.g. "<image>").

The provided string must not contain interior NUL bytes. Pass None to use the library default (mtmd_default_marker()).

Note: the CString is stored inside the params so the pointer remains valid as long as this MtmdContextParams lives.

Errors

Returns MtmdError if the marker string contains a NUL byte.

Trait Implementations

impl Debug for MtmdContextParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for MtmdContextParams

fn default() -> Self

Returns the “default value” for a type.