Struct LanguageModelSession 

pub struct LanguageModelSession { /* private fields */ }

A session for interacting with Apple’s Foundation Models

This provides access to on-device language models via the FoundationModels framework. Requires macOS 26+ or iOS 26+ with Apple Intelligence enabled.

Examples

Blocking response

let session = LanguageModelSession::new()?;
let response = session.response("What is Rust?")?;
println!("{}", response);

Streaming response

let session = LanguageModelSession::new()?;
session.stream_response("What is Rust?", |chunk| {
    print!("{}", chunk);
})?;

Implementations

impl LanguageModelSession

pub fn new() -> Result<Self>

Creates a new language model session

This checks that the Foundation Model is available on the system.

Errors

Returns Error::ModelNotAvailable if Apple Intelligence is not enabled or the system model is unavailable.

Examples found in repository

examples/response.rs (line 16)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    println!("=== Foundation Models - Blocking Response Example ===\n");

    // Create a new session
    println!("Creating session...");
    let session = LanguageModelSession::new()?;
    println!("Session created!\n");

    // Define the prompt
    let prompt = "What is Rust programming language? Please explain in 2-3 sentences.";
    println!("Prompt: \"{}\"\n", prompt);
    println!("Generating response...\n");

    // Get the complete response
    // This blocks until the entire response is generated
    let response = session.response(prompt)?;

    // Print the response
    println!("Response:\n{}\n", response);
    println!("=== Complete ===");

    Ok(())
}

examples/stream_response.rs (line 18)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    println!("=== Foundation Models - Streaming Response Example ===\n");

    // Create a new session
    println!("Creating session...");
    let session = LanguageModelSession::new()?;
    println!("Session created!\n");

    // Define the prompt
    let prompt = "Tell me a short story about a robot learning to paint.";
    println!("Prompt: \"{}\"\n", prompt);
    println!("Streaming response:\n");
    println!("---");

    // Stream the response chunk by chunk
    // The callback is called for each chunk as it's generated
    session.stream_response(prompt, |chunk| {
        // Print each chunk immediately
        print!("{}", chunk);

        // Flush stdout to ensure immediate display
        io::stdout().flush().unwrap();
    })?;

    // Print completion message
    println!("\n---");
    println!("\n=== Stream Complete ===");

    Ok(())
}

pub fn response(&self, prompt: &str) -> Result<String>

Generates a complete response to the given prompt

This method blocks until the entire response is generated and returned as a String. For a better user experience with incremental updates, use stream_response instead.

Arguments

• prompt - The input text to send to the model

Errors

• Error::ModelNotAvailable - If the Foundation Model is not available
• Error::InvalidInput - If the prompt is empty or invalid
• Error::GenerationError - If an error occurs during generation

Examples

let session = LanguageModelSession::new()?;
let response = session.response("Explain Rust ownership")?;
println!("Response: {}", response);

Examples found in repository

examples/response.rs (line 26)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    println!("=== Foundation Models - Blocking Response Example ===\n");

    // Create a new session
    println!("Creating session...");
    let session = LanguageModelSession::new()?;
    println!("Session created!\n");

    // Define the prompt
    let prompt = "What is Rust programming language? Please explain in 2-3 sentences.";
    println!("Prompt: \"{}\"\n", prompt);
    println!("Generating response...\n");

    // Get the complete response
    // This blocks until the entire response is generated
    let response = session.response(prompt)?;

    // Print the response
    println!("Response:\n{}\n", response);
    println!("=== Complete ===");

    Ok(())
}

pub fn stream_response<F>(&self, prompt: &str, on_chunk: F) -> Result<()>

where F: FnMut(&str),

Generates a streaming response to the given prompt

This method calls the provided callback for each chunk as it’s generated, providing immediate feedback to the user. The callback receives string slices containing incremental text deltas.

Arguments

• prompt - The input text to send to the model
• on_chunk - Callback function called for each generated chunk

Errors

• Error::ModelNotAvailable - If the Foundation Model is not available
• Error::InvalidInput - If the prompt is empty or invalid
• Error::GenerationError - If an error occurs during generation

Examples

let session = LanguageModelSession::new()?;

session.stream_response("Tell me a story", |chunk| {
    print!("{}", chunk);
    let _ = io::stdout().flush();
})?;

println!(); // newline after stream completes

Examples found in repository

examples/stream_response.rs (lines 29-35)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    println!("=== Foundation Models - Streaming Response Example ===\n");

    // Create a new session
    println!("Creating session...");
    let session = LanguageModelSession::new()?;
    println!("Session created!\n");

    // Define the prompt
    let prompt = "Tell me a short story about a robot learning to paint.";
    println!("Prompt: \"{}\"\n", prompt);
    println!("Streaming response:\n");
    println!("---");

    // Stream the response chunk by chunk
    // The callback is called for each chunk as it's generated
    session.stream_response(prompt, |chunk| {
        // Print each chunk immediately
        print!("{}", chunk);

        // Flush stdout to ensure immediate display
        io::stdout().flush().unwrap();
    })?;

    // Print completion message
    println!("\n---");
    println!("\n=== Stream Complete ===");

    Ok(())
}

pub fn cancel_stream(&self)

Cancels the current streaming response

This method immediately cancels any ongoing streaming operation started with stream_response. The streaming callback will stop receiving tokens and the stream will complete with the tokens received so far.

Notes

• This is a global operation that cancels the current stream
• Safe to call even if no stream is active
• After cancellation, the stream_response method will return normally

Examples

let session = LanguageModelSession::new()?;
let session_clone = session.clone();

// Start streaming in a thread
thread::spawn(move || {
    session_clone.stream_response("Long prompt...", |chunk| {
        print!("{}", chunk);
    }).ok();
});

// Cancel after a delay
thread::sleep(Duration::from_secs(2));
session.cancel_stream();

Trait Implementations

impl Clone for LanguageModelSession

fn clone(&self) -> LanguageModelSession

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.