Struct LanguageModelSession 

pub struct LanguageModelSession { /* private fields */ }

A stateful conversation with the on-device language model.

Sessions retain their conversation history; subsequent calls to respond build on the previous turns.

Examples

use foundation_models::LanguageModelSession;

let session = LanguageModelSession::new();
let answer = session.respond("Name three Norse gods.")?;
println!("{answer}");

Implementations

impl LanguageModelSession

pub fn new() -> Self

Create a session with the model’s default behaviour.

Panics

Panics if FoundationModels is not available on this OS. Check crate::SystemLanguageModel::is_available first if you need to handle that gracefully.

pub fn with_instructions(instructions: &str) -> Self

Create a session with custom system instructions (“system prompt”).

Panics

Panics if FoundationModels is not available, or if instructions contains an interior NUL byte.

pub fn try_new(instructions: Option<&str>) -> Option<Self>

Fallible constructor. Returns None when FoundationModels is not available (OS too old, model not enabled, etc.) or when instructions contains an interior NUL byte.

pub fn respond(&self, prompt: &str) -> Result<String, FMError>

Send a prompt and block until the full response is available.

Errors

Returns an FMError if the model rejects the prompt, the context window is exceeded, the session is cancelled, or the prompt contains an interior NUL byte.

pub fn prewarm(&self)

Pre-warm the model. Apple loads the weights + initialises the inference engine so the next respond call is faster. Returns immediately; the warm-up runs in the background.

pub fn is_responding(&self) -> bool

True if this session is currently producing a response (i.e. an earlier respond / stream is still in flight on Apple’s queue).

pub fn respond_with( &self, prompt: &str, options: GenerationOptions, ) -> Result<String, FMError>

Like respond, but with explicit generation options.

Errors

See respond.

pub fn stream<F>(&self, prompt: &str, on_chunk: F) -> Result<(), FMError>

where F: FnMut(StreamEvent<'_>) + Send + 'static,

Stream the response as the model generates it. The callback is invoked with each delta and a final invocation with done == true.

Errors

Returns an FMError mirroring respond. The callback may also receive a chunk and an error if the stream fails midway.

pub fn stream_with<F>( &self, prompt: &str, options: GenerationOptions, on_chunk: F, ) -> Result<(), FMError>

where F: FnMut(StreamEvent<'_>) + Send + 'static,

Like stream, but with explicit generation options.

Errors

See stream.

Trait Implementations

impl Debug for LanguageModelSession

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for LanguageModelSession

fn default() -> Self

Returns the “default value” for a type.

impl Drop for LanguageModelSession

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.

impl Send for LanguageModelSession

impl Sync for LanguageModelSession