rusty_foundationmodels 0.2.0

Safe Rust bindings for Apple's FoundationModels on-device AI (Apple Intelligence)
Documentation

rusty_foundationmodels

[!WARNING] This repository has been archived and is no longer maintained.

Development has moved to rs_ai, which contains the latest FoundationModels (and other localai like Phi Silica on Windows and Gemini Nano on Android) support, APIs, fixes, and ongoing work.

Please update your dependencies and use rs_ai for all new projects.

Safe Rust bindings for Apple's FoundationModels on-device AI framework (Apple Intelligence).

Run a local ≈3B-parameter language model entirely on-device — no network requests, no API keys, no data leaving the device.

Requirements

Requirement Value
macOS 26 (Tahoe) or later
Hardware Apple Silicon (M1 or later)
Setting Apple Intelligence enabled in System Settings
Build tool Xcode with the macOS 26 SDK

The crate still compiles and links on unsupported hardware or older macOS; all APIs return Err(Error::Unavailable) at runtime so you can handle the fallback gracefully.

Usage

Add to Cargo.toml:

[dependencies]
rusty_foundationmodels = "0.1"

Single-shot response

use rusty_foundationmodels::{is_available, respond};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    if !is_available() {
        eprintln!("Apple Intelligence not available on this device");
        return Ok(());
    }

    let answer = respond("What is the capital of France?").await?;
    println!("{answer}");
    Ok(())
}

Multi-turn conversation

use rusty_foundationmodels::Session;

let mut session = Session::with_instructions("You are a concise Rust expert.")?;
let r1 = session.respond("What is ownership?").await?;
let r2 = session.respond("Give me a one-line example.").await?;
println!("{r1}\n{r2}");

Dynamic profile switching

Swap system instructions mid-session without losing state:

use rusty_foundationmodels::Session;

let session = Session::with_instructions("You are a poet.")?;
let poem = session.respond("Write a haiku about Rust.").await?;

session.update_profile("You are now a stern code reviewer.")?;
let review = session.respond("Review this: `fn main() {}`").await?;

Multimodal prompts (image + text)

Pass an image alongside a text prompt for visual reasoning:

use rusty_foundationmodels::{Session, Attachment};

let session = Session::new()?;
let screenshot = std::fs::read("diagram.png")?;
let attachment = Attachment::image(screenshot, "image/png");
let analysis = session.respond_with_attachment(
    "Explain this architecture diagram.",
    &attachment,
    &Default::default(),
).await?;

Streaming structured output

Stream JSON fragments as the model fills a typed schema:

use rusty_foundationmodels::{Session, Schema, SchemaProperty, SchemaPropertyType};
use futures_util::StreamExt as _;

let session = Session::new()?;
let schema = Schema::new("CodeReview")
    .property(SchemaProperty::new("score", SchemaPropertyType::Integer))
    .property(SchemaProperty::new("summary", SchemaPropertyType::String));

let mut stream = session.stream_structured(
    "Review this code.", &schema, &Default::default(),
)?;
while let Some(chunk) = stream.next().await {
    // each chunk is a partial JSON blob
    println!("{}", chunk?);
}

Streaming

use rusty_foundationmodels::Session;
use futures_util::StreamExt as _;

let session = Session::new()?;
let mut stream = session.stream("Tell me a short story.")?;

while let Some(chunk) = stream.next().await {
    print!("{}", chunk?);
}
println!();

Structured generation

Describe the shape of the output with a Schema and the model returns a value that deserialises into your type:

use rusty_foundationmodels::{Session, Schema, SchemaProperty, SchemaPropertyType};
use serde::Deserialize;

#[derive(Deserialize)]
struct Capital {
    city: String,
    country: String,
    population: i64,
}

let session = Session::new()?;
let schema = Schema::new("Capital", "A country capital")
    .property(SchemaProperty::new("city", SchemaPropertyType::String).description("City name"))
    .property(SchemaProperty::new("country", SchemaPropertyType::String).description("Country name"))
    .property(SchemaProperty::new("population", SchemaPropertyType::Integer).description("Approximate population"));

let capital: Capital = session.respond_as("What is the capital of France?", &schema).await?;
println!("{} is in {} with ~{} people", capital.city, capital.country, capital.population);

Tool calling

Define Rust functions that the model can invoke during generation:

use rusty_foundationmodels::{Session, Schema, SchemaProperty, SchemaPropertyType, ToolDefinition};
use serde_json::Value;

let tools = vec![
    ToolDefinition::new(
        "get_weather",
        "Return current weather for a city",
        Schema::new("WeatherArgs", "Arguments for get_weather")
            .property(SchemaProperty::new("city", SchemaPropertyType::String).description("City name")),
    )
    .handler(|args: Value| {
        let city = args["city"].as_str().unwrap_or("unknown");
        Ok(format!("{city}: 22°C, sunny"))
    }),
];

let mut session = Session::with_tools(tools, None)?;
let response = session.respond("What is the weather in Tokyo?").await?;
println!("{response}");

Generation options

use rusty_foundationmodels::{respond_with_options, GenerationOptions};

let opts = GenerationOptions {
    temperature: Some(0.2),
    max_tokens: Some(256),
};
let response = respond_with_options("Summarize Rust in one sentence.", &opts).await?;

Availability checking

use rusty_foundationmodels::{availability, UnavailabilityReason};

match availability() {
    Ok(()) => println!("Apple Intelligence is ready"),
    Err(UnavailabilityReason::DeviceNotEligible) => {
        eprintln!("Requires Apple Silicon M1 or later")
    }
    Err(UnavailabilityReason::NotEnabled) => {
        eprintln!("Enable Apple Intelligence in System Settings → Apple Intelligence & Siri")
    }
    Err(UnavailabilityReason::ModelNotReady) => {
        eprintln!("Model is still downloading, try again in a few minutes")
    }
    Err(UnavailabilityReason::Unknown) => eprintln!("Unknown availability state"),
}

Running the integration tests

The integration tests require Apple Intelligence to be active. Once you have macOS 26 on Apple Silicon with Apple Intelligence enabled:

cargo test -- --include-ignored

How it works

The crate ships a Swift source file (bridge.swift) that is compiled by build.rs at build time using xcrun swiftc. The resulting static library exposes a small set of C-ABI functions that Rust calls through an unsafe extern "C" block. Swift's async/await tasks are bridged to Rust futures using futures_channel oneshot and mpsc channels.

If xcrun is not found or compilation fails (e.g. on non-macOS hosts or without the macOS 26 SDK), the build still succeeds — the Swift bridge is simply skipped and all public APIs return Err(Error::Unavailable) at runtime.

License

Licensed under the Mozilla Public License 2.0.