Struct ClientBuilder 

pub struct ClientBuilder { /* private fields */ }

Builder for Client.

Construct one with Client::builder, then chain optional setters and finish with ClientBuilder::build. Every field except model_path has a sensible default, so the minimal usage is:

let client = rig_llama_cpp::Client::builder("path/to/model.gguf").build()?;

The builder shape is forward-compatible: new optional knobs can be added without breaking existing call sites.

Implementations

impl ClientBuilder

pub fn n_ctx(self, n_ctx: u32) -> Self

Desired context window size in tokens. Defaults to 4096.

pub fn sampling(self, sampling: SamplingParams) -> Self

Token sampling parameters.

pub fn fit(self, fit: FitParams) -> Self

Automatic-fit parameters (per-device memory margins, minimum context).

pub fn kv_cache(self, kv_cache: KvCacheParams) -> Self

KV cache data-type configuration. Defaults to F16 / F16.

pub fn checkpoints(self, checkpoint: CheckpointParams) -> Self

In-memory state-checkpoint cache tunables (used by hybrid/recurrent models to preserve KV state across turns).

pub fn mmproj(self, mmproj_path: impl Into<String>) -> Self

Path to a multimodal projector (mmproj) GGUF file. Setting this switches the resulting Client into vision mode. Only available when the mtmd feature is enabled.

pub fn build(self) -> Result<Client, LoadError>

Spawn the inference worker thread, load the model, and return a ready-to-use Client.

Errors

Returns a LoadError if the backend fails to initialise, automatic fitting fails, the GGUF file cannot be loaded, or — when mmproj was set — the multimodal projector cannot be initialised.