Struct Model 

pub struct Model { /* private fields */ }

High-level model wrapper with builder pattern for text generation.

Use this when you need to:

• Generate multiple times with the same model
• Use streaming callbacks
• Maintain conversation history
• Access model metadata

Example

use oxide_rs::Model;

let mut model = Model::new("model.gguf")?
    .with_options(oxide_rs::GenerateOptions {
        max_tokens: 256,
        temperature: 0.7,
        ..Default::default()
    })
    .load()?;

let response = model.generate("Hello!")?;
println!("{}", response);

Implementations

impl Model

pub fn new<P: AsRef<Path>>(model_path: P) -> Result<Self, Box<dyn Error>>

Create a new Model instance.

This only creates the Model struct - use load() to actually load the model.

Arguments

• model_path - Path to a GGUF model file

Example

let model = Model::new("model.gguf")?;

pub fn with_options(self, options: GenerateOptions) -> Self

Set generation options.

Example

let model = Model::new("model.gguf")
    .with_options(GenerateOptions {
        max_tokens: 256,
        temperature: 0.8,
        ..Default::default()
    });

pub fn with_tokenizer<P: AsRef<Path>>(self, tokenizer_path: P) -> Self

Set a custom tokenizer path.

If not provided, the tokenizer will be extracted from the GGUF file.

Example

let model = Model::new("model.gguf")
    .with_tokenizer("tokenizer.json");

pub fn load(&mut self) -> Result<(), Box<dyn Error>>

Load the model into memory.

This must be called before generate().

Example

let mut model = Model::new("model.gguf")?.load()?;

pub fn generate(&mut self, prompt: &str) -> Result<String, Box<dyn Error>>

Generate text from a prompt.

Requires load() to be called first.

Arguments

• prompt - The input prompt

Example

let response = model.generate("What is Rust?")?;
println!("{}", response);

pub fn generate_stream<F>( &mut self, prompt: &str, callback: F, ) -> Result<String, Box<dyn Error>>

where F: FnMut(String),

Generate text with streaming callback.

Tokens are passed to the callback as they’re generated, enabling real-time output display.

Requires load() to be called first.

Arguments

• prompt - The input prompt
• callback - Function called for each generated token

Example

model.generate_stream("Tell me a story", |token| {
    print!("{}", token);
})?;

pub fn generate_batch( &mut self, prompts: Vec<String>, ) -> Result<Vec<String>, Box<dyn Error>>

Generate text from multiple prompts in batch.

Processes multiple prompts sequentially, sharing the loaded model for efficiency. Each prompt generates independently with its own output.

Requires load() to be called first.

Arguments

• prompts - Vector of input prompts

Example

let prompts = vec!["Hello!", "How are you?", "What's up?"];
let results = model.generate_batch(prompts)?;
for result in results {
    println!("{}", result);
}

pub fn warmup(&mut self, num_tokens: usize) -> Result<(), Box<dyn Error>>

Pre-compile compute kernels for faster first-token generation.

Call this after load() to warm up the model before first use.

Arguments

• num_tokens - Number of tokens to use for warmup (default: 128)

Example

model.load()?;
model.warmup(128)?;
// First generation will be faster

pub fn clear_history(&mut self)

Clear conversation history.

Removes all previous messages from the conversation context.

Example

model.generate("Hello")?;
model.clear_history();

pub fn metadata(&self) -> Option<&GgufMetadata>

Get model metadata.

Returns information about the loaded model including name, architecture, layer count, embedding size, etc.

Example

if let Some(meta) = model.metadata() {
    println!("Model: {}", meta.name);
    println!("Architecture: {}", meta.architecture);
}

pub fn context_used(&self) -> Option<usize>

Get current context usage.

Returns the number of tokens currently in the context.

Example

println!("Using {} tokens", model.context_used());

pub fn context_limit(&self) -> Option<usize>

Get context limit.

Returns the maximum context window size.

Example

println!("Context limit: {} tokens", model.context_limit());

pub fn context_percentage(&self) -> Option<f32>

Get context usage percentage.

Returns the percentage of context used (0.0 - 100.0).

Example

println!("{:.1}% context used", model.context_percentage());