Crate fm_rs 

Rust bindings for Apple’s FoundationModels.framework

This crate provides safe, idiomatic Rust bindings to Apple’s FoundationModels framework, which enables on-device AI capabilities powered by Apple Intelligence.

Platform Requirements

• Minimum OS: macOS 26.0+, iOS 26.0+, iPadOS 26.0+, visionOS 26.0+, tvOS 26.0+, watchOS 26.0+
• Apple Intelligence: Must be enabled on the device
• Device: Must support Apple Intelligence

Quick Start

use fm_rs::{SystemLanguageModel, Session, GenerationOptions};

// Create the default system language model
let model = SystemLanguageModel::new()?;

// Check availability
if !model.is_available() {
    println!("FoundationModels is not available on this device");
    return Ok(());
}

// Create a session with instructions
let session = Session::with_instructions(&model, "You are a helpful assistant.")?;

// Send a prompt
let options = GenerationOptions::builder()
    .temperature(0.7)
    .build();
let response = session.respond("What is the capital of France?", &options)?;

println!("Response: {}", response.content());

Streaming Responses

use fm_rs::{SystemLanguageModel, Session, GenerationOptions};

let model = SystemLanguageModel::new()?;
let session = Session::new(&model)?;

session.stream_response("Tell me a story", &GenerationOptions::default(), |chunk| {
    print!("{}", chunk);
})?;

Tool Calling

use fm_rs::{SystemLanguageModel, Session, Tool, ToolOutput, GenerationOptions};
use serde_json::{json, Value};
use std::sync::Arc;

struct WeatherTool;

impl Tool for WeatherTool {
    fn name(&self) -> &str { "get_weather" }
    fn description(&self) -> &str { "Gets the current weather for a location" }
    fn arguments_schema(&self) -> Value {
        json!({
            "type": "object",
            "properties": {
                "location": { "type": "string" }
            },
            "required": ["location"]
        })
    }
    fn call(&self, args: Value) -> fm_rs::Result<ToolOutput> {
        let location = args["location"].as_str().unwrap_or("Unknown");
        Ok(ToolOutput::new(format!("Sunny, 72°F in {location}")))
    }
}

let model = SystemLanguageModel::new()?;
let tools: Vec<Arc<dyn Tool>> = vec![Arc::new(WeatherTool)];
let session = Session::with_tools(&model, &tools)?;

let response = session.respond("What's the weather in Paris?", &GenerationOptions::default())?;

Resource Management

Session Lifecycle

• Ownership: Session owns its underlying Swift LanguageModelSession. When a Session is dropped, the Swift session is freed.
• Thread Safety: Both SystemLanguageModel and Session are Send + Sync. They can be shared across threads, though concurrent calls to the same session may block waiting for the model.
• Drop Behavior: Dropping a Session with active tool callbacks will wait (up to 1 second) for in-flight callbacks to complete before freeing resources.

Memory Considerations

• Sessions maintain conversation history in memory. Long conversations can consume significant memory. Use Session::transcript_json() to persist and Session::from_transcript() to restore conversations.
• Monitor context usage with context_usage_from_transcript() and implement compaction strategies using compact_transcript() when approaching limits.
• The default context window is approximately DEFAULT_CONTEXT_TOKENS tokens, though this may vary by device and model version.

Blocking Operations

• Session::respond() blocks until generation completes. For long generations, consider using Session::respond_with_timeout() or Session::stream_response().
• Tool callbacks are invoked synchronously during generation. Long-running tools will block the generation pipeline.

No Persistence Across Restarts

Sessions do not persist across process restarts. To resume a conversation:

1. Save the transcript with Session::transcript_json() before shutdown
2. Restore with Session::from_transcript() on next launch

Error Recovery

• If a session enters an error state, create a new session rather than retrying.
• Tool errors are reported via Error::ToolCall with context about which tool failed and the arguments that were passed.

Structs

CompactionConfig

Configuration for transcript compaction.

ContextLimit

Configuration for estimating context usage.

ContextUsage

Estimated context usage for a session.

GenerationOptions

Options that control how the model generates its response.

GenerationOptionsBuilder

Builder for configuring GenerationOptions.

Response

Response returned by the model.

Session

A session that interacts with a language model.

SystemLanguageModel

The system language model provided by Apple Intelligence.

ToolCallError

Error that occurred during tool invocation.

ToolOutput

Output returned by a tool invocation.

Enums

Error

Error types for FoundationModels operations.

ModelAvailability

Represents the availability status of a FoundationModel.

Sampling

Sampling strategy for token generation.

Constants

DEFAULT_CONTEXT_TOKENS

Default context window size for Apple’s on-device Foundation Models.

Traits

Generable

Trait for types that can provide a JSON Schema for structured generation.

Tool

A tool that can be invoked by the model.

Functions

compact_transcript

Compacts a transcript into a summary using the on-device model.

context_usage_from_transcript

Estimates token usage for the session transcript JSON.

estimate_tokens

Estimates tokens based on a characters-per-token heuristic.

fm_rust_string_free

Frees a string allocated by Rust (via CString::into_raw).

transcript_to_text

Extracts readable text from transcript JSON.

Type Aliases

Result

Result type for FoundationModels operations.

Derive Macros

Generable

Re-export the derive macro when the derive feature is enabled. Derives the Generable trait for a struct or enum.