chat_applefm/
lib.rs

1//! Apple on-device foundation model provider for chat-rs.
2//!
3//! Talks to the ~3B-parameter model that ships with Apple Intelligence
4//! (macOS 26+) through Apple's FoundationModels framework. There is no
5//! HTTP and no weights file: the OS owns the model; this crate owns the
6//! translation.
7//!
8//! ## How it connects
9//!
10//! Rust cannot call Swift directly, so the crate embeds a small Swift
11//! package (`bridge/`) exposing a few plain C functions. Requests and
12//! responses cross that boundary as JSON strings — the same mental model
13//! as an HTTP provider's wire format, minus the network. The bridge is
14//! compiled automatically by `build.rs`; users only run `cargo build`.
15//!
16//! On non-macOS targets the crate still compiles (with a stub bridge)
17//! and reports the model as unavailable.
18//!
19//! **Binary crates on macOS need one build.rs line** so the Swift
20//! concurrency runtime resolves at load time:
21//!
22//! ```no_run
23//! // build.rs of your binary crate
24//! println!("cargo:rustc-link-arg=-Wl,-rpath,/usr/lib/swift");
25//! ```
26//!
27//! ## Current scope
28//!
29//! Text completion (with optional LoRA fine-tune) and availability
30//! probing. Tools, structured output, and streaming are rejected at
31//! request time for now and arrive in later slices.
32//!
33//! ```no_run
34//! # async fn run() -> Result<(), Box<dyn std::error::Error>> {
35//! use chat_applefm::AppleFMBuilder;
36//!
37//! let probe = chat_applefm::availability();
38//! if !probe.available {
39//!     println!("unavailable: {}", probe.reason.as_deref().unwrap_or("?"));
40//!     return Ok(());
41//! }
42//!
43//! let client = AppleFMBuilder::new()
44//!     .with_lora("adapters/transcripts.fmadapter") // optional fine-tune
45//!     .build()?; // validates config (e.g. the adapter path) upfront
46//! // System prompts go through Messages like any other provider; the
47//! // provider maps them onto the session's instructions.
48//! # Ok(()) }
49//! ```
50//!
51//! ## LoRA fine-tunes
52//!
53//! [`AppleFMBuilder::with_lora`] loads a `.fmadapter` package — a LoRA
54//! trained against the on-device base model with Apple's adapter
55//! training toolkit. Adapters are version-locked to the base model:
56//! after a macOS update rolls the model, retrain and reship.
57//!
58//! See `providers/AGENTS.md` for the overall provider architecture.
59
60#![allow(clippy::result_large_err)]
61
62mod api;
63mod builder;
64mod client;
65mod ffi;
66
67pub use builder::AppleFMBuilder;
68pub use client::{AppleFMClient, Sampling};
69
70use serde::Deserialize;
71
72/// Result of probing whether the on-device Apple model is usable here.
73#[derive(Debug, Clone, Deserialize)]
74pub struct Availability {
75    /// Whether a session can be created on this machine right now.
76    pub available: bool,
77    /// Human-readable explanation when `available` is `false`
78    /// (ineligible hardware, Apple Intelligence disabled, assets still
79    /// downloading, OS too old, or a bridge-less build).
80    #[serde(default)]
81    pub reason: Option<String>,
82}
83
84/// Ask the OS whether the Apple Intelligence on-device model can be used.
85///
86/// Cheap to call; performs no generation. This is the recommended first
87/// call before constructing a client, and what router strategies should
88/// consult when deciding whether this provider is eligible at all.
89pub fn availability() -> Availability {
90    let json = ffi::availability_json();
91    serde_json::from_str(&json).unwrap_or_else(|_| Availability {
92        available: false,
93        reason: Some(format!("malformed bridge reply: {json}")),
94    })
95}
chat_applefm/lib.rs

chat_applefm/
lib.rs
