Crate foundation_models 

foundation-models

Safe, idiomatic Rust bindings for Apple’s FoundationModels framework — the on-device large language model that ships with Apple Intelligence on macOS 26.0+.

Status: experimental. API surface will change as more of FoundationModels is wrapped (tools, structured generation, transcripts).

Features

• On-device LLM — runs entirely locally on Apple Silicon
• Streaming generation — token-by-token deltas via callback
• Custom instructions — system-prompt style guidance
• Generation options — temperature, max tokens, sampling modes
• Zero dependencies — no objc2, no core-foundation, no procedural macros
• Async optional — opt-in async feature for runtime-agnostic awaiting

Requirements

• macOS 26.0 or newer (build host and runtime)
• Xcode 26 SDK (the crate’s build.rs detects this via xcrun --sdk macosx --show-sdk-version)
• Apple Intelligence enabled in System Settings
• Apple Silicon (Intel Macs are not eligible)

Installation

[dependencies]
foundation-models = { version = "0.1", features = ["macos_26_0"] }

Quick start

use foundation_models::prelude::*;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    if !SystemLanguageModel::is_available() {
        eprintln!("Unavailable: {:?}", SystemLanguageModel::availability());
        return Ok(());
    }

    let session = LanguageModelSession::with_instructions(
        "You answer in a single concise sentence."
    );

    let reply = session.respond("Why is the sky blue?")?;
    println!("{reply}");

    Ok(())
}

Streaming

use foundation_models::prelude::*;
use std::io::Write;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let session = LanguageModelSession::new();

    session.stream("Write a haiku about Rust.", |event| match event {
        StreamEvent::Chunk(s) => {
            print!("{s}");
            std::io::stdout().flush().ok();
        }
        StreamEvent::Done => println!(),
        StreamEvent::Error(e) => eprintln!("\nerror: {e}"),
        _ => {}
    })?;

    Ok(())
}

Generation options

use foundation_models::prelude::*;

let opts = GenerationOptions::new()
    .with_temperature(0.7)
    .with_maximum_response_tokens(500)
    .with_sampling(SamplingMode::TopP(0.9));

let session = LanguageModelSession::new();
let reply = session.respond_with("Suggest a recipe.", opts)?;
println!("{reply}");

Architecture

This crate uses the same Swift-bridge pattern as screencapturekit-rs:

┌────────────────────────────────────────────────────────────┐
│ Safe Rust API (LanguageModelSession, GenerationOptions)    │
├────────────────────────────────────────────────────────────┤
│ extern "C" FFI declarations (src/ffi/mod.rs)               │
├────────────────────────────────────────────────────────────┤
│ Swift @_cdecl bridge (swift-bridge/Sources/...)            │
├────────────────────────────────────────────────────────────┤
│ Apple FoundationModels.framework (Swift, async throws)     │
└────────────────────────────────────────────────────────────┘

The Swift layer hides the async throws surface behind callback-based C functions, so the Rust side stays dependency-free.

Roadmap

• Tool protocol bridging (function calling)
• Structured generation via Generable (currently Swift-macro-only)
• Transcript inspection (per-turn token counts, attachments)
• Adapter support
• Vision-modality input once Apple ships it

License

Licensed under either of Apache-2.0 or MIT at your option.

---

API Documentation

Safe, idiomatic Rust bindings for Apple’s FoundationModels framework — the on-device large language model that ships with Apple Intelligence.

Generate text, hold multi-turn conversations, and stream tokens from the system language model on macOS 26.0+.

Quick start

use foundation_models::{LanguageModelSession, SystemLanguageModel};

if !SystemLanguageModel::is_available() {
    eprintln!("Model unavailable: {:?}", SystemLanguageModel::availability());
    return Ok(());
}

let session = LanguageModelSession::new();
let reply = session.respond("Name three Norse gods.")?;
println!("{reply}");

Streaming

use foundation_models::{LanguageModelSession, StreamEvent};
use std::io::Write;

let session = LanguageModelSession::new();
session.stream("Tell me a haiku about Rust.", |event| match event {
    StreamEvent::Chunk(s) => {
        print!("{s}");
        std::io::stdout().flush().ok();
    }
    StreamEvent::Done => println!(),
    StreamEvent::Error(e) => eprintln!("\nstream error: {e}"),
    _ => {}
})?;

Re-exports

pub use error::FMError;

pub use error::Unavailability;

pub use generation::GenerationOptions;

pub use generation::SamplingMode;

pub use model::Availability;

pub use model::SystemLanguageModel;

pub use session::LanguageModelSession;

pub use session::StreamEvent;

Modules

error

Errors produced by the FoundationModels bridge.

ffi

Raw FFI declarations matching the Swift @_cdecl exports in swift-bridge/Sources/FoundationModelsBridge.

generation

Knobs that control how the model produces text.

model

SystemLanguageModel — entry point for querying device capability.

prelude

Common imports for users of this crate.

session

LanguageModelSession — a stateful conversation with the on-device model.