car-inference 0.63.0

//! Apple FoundationModels backend — on-device inference through the
//! macOS 26+ system LLM. Bridges to a small Swift shim
//! (`swift/CarFoundationModels.swift`) compiled by `build.rs`.
//!
//! Four paths are exposed:
//! - [`generate`]: blocking text generation (single round-trip).
//! - [`stream`]: callback-based incremental streaming.
//! - [`generate_with_tools`]: blocking generation with a JSON tool
//!   catalog; captured tool calls come back as standard
//!   [`crate::tasks::generate::ToolCall`]s.
//! - [`generate_structured`]: schema-guided generation — a JSON Schema
//!   is enforced by Foundation Models' constrained decoding
//!   (`DynamicGenerationSchema` → `respond(to:schema:)`).
//!
//! # Tool calling — capture-and-return bridge
//!
//! CAR's contract is "models propose; the runtime validates and
//! executes": backends return the tool *call*, never its result.
//! Foundation Models inverts that — the framework invokes the Swift
//! `Tool.call` itself mid-turn. The shim bridges the two by
//! registering capture-only tools built from each JSON-Schema tool
//! definition via `DynamicGenerationSchema` (so the model sees real
//! per-tool schemas): the first invocation records `(name, arguments)`
//! and throws a sentinel to end the turn, and the captured call is
//! returned to Rust as a standard `ToolCall`. The engine executes it
//! and drives the follow-up turn exactly as it does for remote
//! backends.
//!
//! **How many calls come back is host-dependent, and was measured.**
//! Whether the collector ends up holding one call or all of them turns
//! on whether the framework still invokes the rest of a batch after
//! the first tool throws — which Apple documents neither way. On
//! macOS 26 the answer is one, a valid sequential tool-use trace. On
//! macOS 27 it is *all of them*: two tools proposed, both recorded,
//! through this same unchanged shim. So the catalog entry claims
//! `tool_use` and deliberately NOT `multi_tool_call`, and
//! [`supports_parallel_tool_calls`] upgrades the live row in
//! `registry.rs` on hosts where it holds. A host that cannot do it
//! never advertises it.
//!
//! # Boundaries (honest, not aspirational)
//!
//! - **Image input** works from macOS 27, where the system model declares
//!   the `vision` capability. [`generate_with_images`] decodes
//!   `ContentBlock::ImageBase64` payloads through ImageIO to `CGImage`,
//!   because the framework offers no raw-bytes initialiser, and attaches
//!   them ahead of the prompt. Below 27 — and on any device whose model
//!   does not declare `vision` — it returns
//!   [`InferenceError::UnsupportedMode`], which is a *routing* signal, so
//!   the request falls through to a richer model rather than failing.
//!   [`supports_vision`] reads the model's own capability set rather than
//!   guessing from the OS version, and `registry.rs` gates the row's
//!   `vision` claim on the same probe, so the claim can never outrun the
//!   path that serves it.
//! - **Audio and video input** remain rejected upstream with
//!   [`InferenceError::UnsupportedMode`]; only images crossed over.
//! - **Schema fidelity**: the JSON-Schema→`DynamicGenerationSchema`
//!   conversion natively covers `object` (declared, or typeless with
//!   `properties`), `string` (+ string enums and `const`), `integer`,
//!   `number`, `boolean`, `array`, and unions (`anyOf`, `oneOf`, type
//!   arrays). Before crossing the FFI the Rust side rewrites the schema
//!   so that conversion keeps JSON Schema's meaning
//!   (`normalize_schema`): constraints beside a union are merged into
//!   every branch, and a property that allows `null` — which has no
//!   generation type — is made optional, with the `null` written back
//!   into the output when the property was required
//!   (`fill_required_nulls`). What still cannot be represented —
//!   `allOf`/`not`/`$ref`, typeless nodes, unrecognized types,
//!   non-string enums, `oneOf` branches that can overlap (generated as
//!   `anyOf`), `null` outside an object property — degrades to a
//!   **permissive string field** (never to an empty object, which would
//!   force `{}` under constrained decoding) or loses that one
//!   constraint, and every such degradation is reported via
//!   `tracing::warn!` before the call ([`schema_degradations`]).
//! - **Pre-call assistant text is discarded on captured-call turns**:
//!   ending the turn on the capture sentinel means `respond()` throws
//!   before yielding content, so `generate_with_tools` returns
//!   `text == ""` whenever a tool call was captured. Any prose the
//!   model produced before deciding to call the tool is lost — same
//!   information the engine acts on (the call), but consumers must not
//!   expect Anthropic-style "text + tool_use in one turn". **macOS 27
//!   does not lift this**, unlike the parallel-call boundary above:
//!   capturing requires aborting the turn, and an aborted turn yields
//!   no content, regardless of how many calls it proposed first.
//! - **Runtime verification** requires Apple Intelligence to be
//!   provisioned; unit tests exercise the wiring up to the
//!   `is_available()` gate only.
//!
//! Several of those boundaries were things macOS 27 was expected to
//! change. It changed exactly one — parallel capture — and it did so
//! without any code change here, which is why that is now a measured
//! host property rather than a redesign. The empty-text boundary and
//! the multimodal rejection did not move. Before acting on any of
//! this, read `docs/proposals/macos-27-foundation-models.md`: it
//! records what was verified against the real 27 SDK, which boundaries
//! are consequences of the capture-and-return bridge rather than of
//! model capability, and the CI trap that leaves macOS 27 code paths
//! untested while looking green.
//!
//! Cfg-gated to `aarch64-apple-darwin`; everything below that line is
//! invisible on Linux/Intel-Mac builds. On those platforms the upstream
//! schema check (`is_foundation_models()`) still works (so registries
//! can describe the model) but dispatch errors out before calling here.

use std::ffi::{c_char, c_int, c_void, CStr, CString};
use std::ptr;
use std::sync::{Mutex, OnceLock};
use std::time::{Duration, Instant};

pub use super::fm_schema::schema_degradations;
use super::fm_schema::{fill_structured_nulls, fill_tool_call_nulls, normalize_schema};
use crate::InferenceError;

/// How long [`is_available`] caches the framework probe before re-checking.
/// Apple Intelligence can be toggled on/off in System Settings and the
/// model can finish provisioning after process start, so a permanent
/// cache would strand long-running daemons. Five seconds is enough to
/// cover a tight router loop without staling against settings changes.
const AVAILABILITY_CACHE_TTL: Duration = Duration::from_secs(5);

// ---------------------------------------------------------------------
// extern "C" surface emitted by the Swift shim.
//
// Gated by `car_fm_swift_built`, set by `build.rs` only when `swiftc`
// successfully compiled the shim into a static library. On hosts
// without full Xcode (Command Line Tools only) the cfg is absent and
// we provide stubs that report unavailable — keeps the crate buildable
// and lets the runtime path return a clean UnsupportedMode instead of
// failing at link time.
// ---------------------------------------------------------------------

#[cfg(car_fm_swift_built)]
extern "C" {
    fn car_fm_is_available() -> c_int;
    fn car_fm_context_size() -> c_int;
    fn car_fm_supports_parallel_tool_calls() -> c_int;
    fn car_fm_supports_vision() -> c_int;
    fn car_fm_pcc_available() -> c_int;
    fn car_fm_pcc_context_size() -> c_int;
    fn car_fm_pcc_generate(
        prompt: *const c_char,
        instructions: *const c_char,
        reasoning_level: *const c_char,
        max_tokens: i32,
        temperature: f64,
        out_text: *mut *mut c_char,
        out_err: *mut *mut c_char,
    ) -> c_int;
    fn car_fm_generate_with_images(
        prompt: *const c_char,
        instructions: *const c_char,
        images_json: *const c_char,
        max_tokens: i32,
        temperature: f64,
        out_text: *mut *mut c_char,
        out_err: *mut *mut c_char,
    ) -> c_int;
    fn car_fm_count_tokens(
        instructions: *const c_char,
        prompt: *const c_char,
        completion: *const c_char,
        out_prompt_tokens: *mut i32,
        out_completion_tokens: *mut i32,
    ) -> c_int;
    fn car_fm_free_string(ptr: *mut c_char);
    fn car_fm_generate(
        prompt: *const c_char,
        instructions: *const c_char,
        max_tokens: i32,
        temperature: f64,
        out_text: *mut *mut c_char,
        out_err: *mut *mut c_char,
    ) -> c_int;
    fn car_fm_generate_stream(
        prompt: *const c_char,
        instructions: *const c_char,
        max_tokens: i32,
        temperature: f64,
        callback: extern "C" fn(token: *const c_char, state: *mut c_void) -> c_int,
        state: *mut c_void,
        out_err: *mut *mut c_char,
    ) -> c_int;
    fn car_fm_generate_with_tools(
        prompt: *const c_char,
        instructions: *const c_char,
        tools_json: *const c_char,
        tool_choice: *const c_char,
        max_tokens: i32,
        temperature: f64,
        out_text: *mut *mut c_char,
        out_tool_calls_json: *mut *mut c_char,
        out_err: *mut *mut c_char,
    ) -> c_int;
    fn car_fm_generate_structured(
        prompt: *const c_char,
        instructions: *const c_char,
        schema_json: *const c_char,
        max_tokens: i32,
        temperature: f64,
        out_json: *mut *mut c_char,
        out_err: *mut *mut c_char,
    ) -> c_int;
}

#[cfg(not(car_fm_swift_built))]
mod swift_stubs {
    use super::{c_char, c_int, c_void};
    /// Always returns 0 (unavailable). The runtime check in
    /// [`super::is_available`] catches this and prevents any of the
    /// other extern paths from being called.
    pub(super) unsafe fn car_fm_is_available() -> c_int {
        0
    }
    /// Always returns 0 ("window unknown"), so callers fall back to the
    /// catalog's declared `context_length`.
    pub(super) unsafe fn car_fm_context_size() -> c_int {
        0
    }
    /// Always returns 0 — no shim, so no tool path at all.
    pub(super) unsafe fn car_fm_supports_parallel_tool_calls() -> c_int {
        0
    }
    /// Always returns 0 — no shim, so no vision either.
    pub(super) unsafe fn car_fm_supports_vision() -> c_int {
        0
    }
    /// Always returns 0 — no shim, so no PCC either.
    pub(super) unsafe fn car_fm_pcc_available() -> c_int {
        0
    }
    /// Always returns 0 ("window unknown").
    pub(super) unsafe fn car_fm_pcc_context_size() -> c_int {
        0
    }
    /// Unreachable: gated upstream by [`super::pcc_available`].
    pub(super) unsafe fn car_fm_pcc_generate(
        _prompt: *const c_char,
        _instructions: *const c_char,
        _reasoning_level: *const c_char,
        _max_tokens: i32,
        _temperature: f64,
        _out_text: *mut *mut c_char,
        _out_err: *mut *mut c_char,
    ) -> c_int {
        unreachable!("car_fm_pcc_generate called without the Swift bridge")
    }
    /// Unreachable: gated upstream by [`super::supports_vision`].
    pub(super) unsafe fn car_fm_generate_with_images(
        _prompt: *const c_char,
        _instructions: *const c_char,
        _images_json: *const c_char,
        _max_tokens: i32,
        _temperature: f64,
        _out_text: *mut *mut c_char,
        _out_err: *mut *mut c_char,
    ) -> c_int {
        unreachable!("car_fm_generate_with_images called without the Swift bridge")
    }
    /// Always fails — no shim, so the caller reports unknown usage.
    pub(super) unsafe fn car_fm_count_tokens(
        _instructions: *const c_char,
        _prompt: *const c_char,
        _completion: *const c_char,
        _out_prompt_tokens: *mut i32,
        _out_completion_tokens: *mut i32,
    ) -> c_int {
        1
    }
    /// No allocation crossed the boundary, so nothing to free.
    pub(super) unsafe fn car_fm_free_string(_ptr: *mut c_char) {}
    /// Unreachable: gated upstream by [`super::is_available`].
    pub(super) unsafe fn car_fm_generate(
        _prompt: *const c_char,
        _instructions: *const c_char,
        _max_tokens: i32,
        _temperature: f64,
        _out_text: *mut *mut c_char,
        _out_err: *mut *mut c_char,
    ) -> c_int {
        unreachable!("car_fm_generate called without the Swift bridge")
    }
    /// Unreachable: gated upstream by [`super::is_available`].
    pub(super) unsafe fn car_fm_generate_stream(
        _prompt: *const c_char,
        _instructions: *const c_char,
        _max_tokens: i32,
        _temperature: f64,
        _callback: extern "C" fn(token: *const c_char, state: *mut c_void) -> c_int,
        _state: *mut c_void,
        _out_err: *mut *mut c_char,
    ) -> c_int {
        unreachable!("car_fm_generate_stream called without the Swift bridge")
    }
    /// Unreachable: gated upstream by [`super::is_available`].
    pub(super) unsafe fn car_fm_generate_with_tools(
        _prompt: *const c_char,
        _instructions: *const c_char,
        _tools_json: *const c_char,
        _tool_choice: *const c_char,
        _max_tokens: i32,
        _temperature: f64,
        _out_text: *mut *mut c_char,
        _out_tool_calls_json: *mut *mut c_char,
        _out_err: *mut *mut c_char,
    ) -> c_int {
        unreachable!("car_fm_generate_with_tools called without the Swift bridge")
    }
    /// Unreachable: gated upstream by [`super::is_available`].
    pub(super) unsafe fn car_fm_generate_structured(
        _prompt: *const c_char,
        _instructions: *const c_char,
        _schema_json: *const c_char,
        _max_tokens: i32,
        _temperature: f64,
        _out_json: *mut *mut c_char,
        _out_err: *mut *mut c_char,
    ) -> c_int {
        unreachable!("car_fm_generate_structured called without the Swift bridge")
    }
}

#[cfg(not(car_fm_swift_built))]
use swift_stubs::{
    car_fm_context_size, car_fm_count_tokens, car_fm_free_string, car_fm_generate,
    car_fm_generate_stream, car_fm_generate_structured, car_fm_generate_with_images,
    car_fm_generate_with_tools, car_fm_is_available, car_fm_pcc_available, car_fm_pcc_context_size,
    car_fm_pcc_generate, car_fm_supports_parallel_tool_calls, car_fm_supports_vision,
};

// ---------------------------------------------------------------------
// Public API.
// ---------------------------------------------------------------------

/// Returns true when the FoundationModels framework is available **and**
/// the on-device model is provisioned. Cached for [`AVAILABILITY_CACHE_TTL`]
/// so a tight router loop doesn't repeatedly cross the FFI boundary,
/// while still allowing recovery from "framework unavailable at startup,
/// available later" — which is the common case for long-running
/// daemons whose host machine finishes Apple Intelligence provisioning
/// minutes after launch.
pub fn is_available() -> bool {
    // (Instant, value) pair under a Mutex. The probe itself is cheap; the
    // Mutex overhead is negligible vs the FFI call we're avoiding.
    static CACHE: Mutex<Option<(Instant, bool)>> = Mutex::new(None);

    let now = Instant::now();
    let mut guard = match CACHE.lock() {
        Ok(g) => g,
        Err(poisoned) => poisoned.into_inner(),
    };
    if let Some((stamped, value)) = *guard {
        if now.duration_since(stamped) < AVAILABILITY_CACHE_TTL {
            return value;
        }
    }
    let value = unsafe { car_fm_is_available() != 0 };
    // Stamped when the answer was produced. `now` predates the probe, so the
    // entry aged by however long the probe took before it was ever consulted.
    // Cheap here (one FFI call) where it is not in `mlx_vlm_cli`, which shares
    // this shape — but the correct instant is the same one in both.
    *guard = Some((Instant::now(), value));
    value
}

/// The on-device model's real context window, when the OS will say.
///
/// Returns `None` on anything below macOS/iOS 26.4, where
/// `SystemLanguageModel.contextSize` does not exist; when the shim was
/// *compiled* against an SDK older than that (see the two gates on
/// `car_fm_context_size` in `swift/CarFoundationModels.swift`); and on
/// hosts where the Swift shim was not built at all. Callers keep their
/// declared `context_length` in each case.
///
/// Why this exists: the window is an Apple-owned number that has no
/// reason to stay fixed across OS releases, and a catalog literal that
/// over-claims it fails *after* context assembly has already been paid
/// for — `build_context_for_model` sizes the assembly budget from it.
/// Asking the framework removes that whole class of bug. Measured 4096
/// on both macOS 26 and macOS 27.
///
/// Cached for the process lifetime rather than on a TTL like
/// [`is_available`]. The two differ on purpose: availability genuinely
/// changes under a running daemon (Apple Intelligence can finish
/// provisioning minutes after launch), whereas a model's context window
/// is fixed for as long as the process is running — the OS cannot move it
/// without a reboot. `refresh_availability()` is reachable from more than
/// just the constructor, so a plain call per refresh would cross the FFI
/// boundary for a value that cannot have changed.
pub fn context_size() -> Option<u32> {
    static CACHE: OnceLock<Option<u32>> = OnceLock::new();
    *CACHE.get_or_init(|| {
        let raw = unsafe { car_fm_context_size() };
        u32::try_from(raw).ok().filter(|&v| v > 0)
    })
}

/// Whether one turn can return more than one captured tool call.
///
/// True from macOS/iOS 27 on, false below, cached for the process lifetime
/// like [`context_size`] — the host OS cannot change under a running process.
///
/// The capture bridge throws a sentinel from each tool to keep execution on
/// CAR's side of the contract. Whether the collector holds one call or all of
/// them depends on whether the framework still invokes the rest of a batch
/// after the first throws, which Apple documents neither way. On macOS 27 it
/// does: two tools proposed, both recorded, through the *unchanged* shim.
/// That is why this is a measurement rather than a flag, and why no Swift
/// redesign was needed to lift the boundary.
///
/// Read by `registry.rs` to add `ModelCapability::MultiToolCall` to the
/// `apple/foundation:default` row on hosts where it holds. The catalog
/// literal stays conservative, so a host that cannot do it never advertises
/// it.
///
/// This does NOT lift the empty-text boundary: the turn is still aborted, so
/// a captured turn returns `""`. Preserving assistant prose would need the
/// runtime to execute the call and the session to continue — a different
/// design, not a capability bit.
pub fn supports_parallel_tool_calls() -> bool {
    static CACHE: OnceLock<bool> = OnceLock::new();
    *CACHE.get_or_init(|| unsafe { car_fm_supports_parallel_tool_calls() != 0 })
}

/// Token counts for one turn, from the framework's own tokenizer.
///
/// `(prompt_tokens, completion_tokens)`, or `None` below macOS/iOS 26.4,
/// where `SystemLanguageModel.tokenCount(for:)` does not exist, and wherever
/// the shim was not built.
///
/// Why this exists: before it, the FoundationModels backend reported no usage
/// at all. A `car-bench` run over 19 tasks came back `total_input_tokens: 0,
/// total_output_tokens: 0` — so every receipt and every cost attribution for
/// `apple/foundation:default` was reading zeros, and the same gap blocks a
/// Private Cloud Compute row, whose `QuotaUsage` reports a coarse status and
/// no counts either.
///
/// **What it counts, precisely:** the text CAR sent and the text it got back.
/// It is not the framework's internal accounting for the turn, which Apple
/// does not expose, so it excludes whatever chat scaffolding sits around them.
/// A close measured floor, not an exact billing figure — the right trade for
/// a local model whose cost is zero, where the number's job is context
/// budgeting and non-empty receipts.
///
/// Deliberately not cached, unlike [`context_size`]: this is per-turn data.
pub fn count_tokens(
    instructions: Option<&str>,
    prompt: &str,
    completion: &str,
) -> Option<(u64, u64)> {
    let instructions_c = CString::new(instructions.unwrap_or("")).ok()?;
    let prompt_c = CString::new(prompt).ok()?;
    let completion_c = CString::new(completion).ok()?;
    let mut prompt_tokens: i32 = -1;
    let mut completion_tokens: i32 = -1;
    let rc = unsafe {
        car_fm_count_tokens(
            instructions_c.as_ptr(),
            prompt_c.as_ptr(),
            completion_c.as_ptr(),
            &mut prompt_tokens,
            &mut completion_tokens,
        )
    };
    if rc != 0 || prompt_tokens < 0 || completion_tokens < 0 {
        return None;
    }
    Some((prompt_tokens as u64, completion_tokens as u64))
}

/// The completion to pass [`count_tokens`] for a tool-enabled turn: the
/// returned text followed by the captured calls (name and JSON arguments).
///
/// A turn that captures a call is aborted by the bridge and returns empty
/// text, so counting the text alone reported every captured tool call as zero
/// output tokens while the model had generated a tool name and a full argument
/// object (Parslee-ai/car#1765). The calls go through the same framework
/// tokenizer as text, which is how this backend counts all of its output.
pub fn completion_text(text: &str, tool_calls: &[crate::ToolCall]) -> String {
    let calls = crate::tool_calls_text(tool_calls);
    match (text.is_empty(), calls.is_empty()) {
        (_, true) => text.to_string(),
        (true, false) => calls,
        (false, false) => format!("{text}\n{calls}"),
    }
}

/// Whether this device's system model accepts images as input.
///
/// Reads `SystemLanguageModel.capabilities`, the model's own declaration,
/// rather than inferring from the OS version. False below macOS/iOS 27, where
/// the public API is text-only, and wherever the shim was not built.
///
/// Asking the model beats hardcoding a version: if Apple ships vision to a
/// tier that lacks it today, or withholds it on a device that cannot run it,
/// this follows without a code change. Cached for the process lifetime — the
/// device's model does not change under a running process.
///
/// `registry.rs` reads this to add `ModelCapability::Vision` to the
/// `apple/foundation:default` row. The catalog literal omits it, so a host
/// that cannot do it never advertises it — and, critically, the row is only
/// upgraded where [`generate_with_images`] can actually serve the request.
pub fn supports_vision() -> bool {
    static CACHE: OnceLock<bool> = OnceLock::new();
    *CACHE.get_or_init(|| unsafe { car_fm_supports_vision() != 0 })
}

/// Blocking generation with image input.
///
/// `images` are base64 payloads exactly as `ContentBlock::ImageBase64`
/// carries them; the shim decodes each through ImageIO to a `CGImage`,
/// because the framework offers no raw-bytes initialiser.
///
/// Returns [`InferenceError::UnsupportedMode`] when the host cannot do
/// vision, so the router falls through to a richer model rather than failing
/// the request — the same posture the text-only rejection had before this
/// path existed.
pub fn generate_with_images(
    prompt: &str,
    instructions: Option<&str>,
    images: &[String],
    max_tokens: u32,
    temperature: f32,
) -> Result<String, InferenceError> {
    if !is_available() {
        return Err(unavailable_error());
    }
    if !supports_vision() {
        return Err(InferenceError::UnsupportedMode {
            mode: "multimodal-content",
            backend: "foundation-models",
            reason: "this device's system model does not accept image input \
                     (requires macOS 27+); route image content to a VL model",
        });
    }
    let images_json = serde_json::to_string(images).map_err(|e| {
        InferenceError::InferenceFailed(format!("image list failed to serialize: {e}"))
    })?;
    let prompt_c = CString::new(prompt)
        .map_err(|e| InferenceError::InferenceFailed(format!("prompt has interior NUL: {e}")))?;
    let instructions_c = CString::new(instructions.unwrap_or("")).map_err(|e| {
        InferenceError::InferenceFailed(format!("instructions have interior NUL: {e}"))
    })?;
    let images_c = CString::new(images_json).map_err(|e| {
        InferenceError::InferenceFailed(format!("image JSON has interior NUL: {e}"))
    })?;

    let mut out_text: *mut c_char = ptr::null_mut();
    let mut out_err: *mut c_char = ptr::null_mut();
    let rc = unsafe {
        car_fm_generate_with_images(
            prompt_c.as_ptr(),
            instructions_c.as_ptr(),
            images_c.as_ptr(),
            max_tokens as i32,
            temperature as f64,
            &mut out_text,
            &mut out_err,
        )
    };
    if rc != 0 {
        let message = consume_swift_string(out_err);
        // rc 4 is "this host cannot do vision" — a routing signal, not a
        // failure, so it keeps the UnsupportedMode shape the router acts on
        // and the request falls through to a richer model.
        if rc == 4 {
            return Err(InferenceError::UnsupportedMode {
                mode: "multimodal-content",
                backend: "foundation-models",
                reason: "this device's system model does not accept image input",
            });
        }
        return Err(map_shim_error(message));
    }
    Ok(consume_swift_string(out_text))
}

/// Map a shim error string onto the right [`InferenceError`].
///
/// macOS 27 replaced the opaque error surface with a typed
/// `LanguageModelError`, and the shim now forwards it as JSON. The mapping
/// that matters is **retryability**: `rateLimited` and `timeout` are
/// transient, and before this every FoundationModels failure arrived as a
/// flat string and became a non-retryable `InferenceFailed` — so a rate limit
/// ended the request instead of backing off.
///
/// Non-JSON input (any OS below 27, or a non-`LanguageModelError`) falls
/// through to `InferenceFailed` with the original text, which is exactly the
/// previous behaviour, except the untyped context-overflow text, which maps to
/// the context-window refusal (car#2665). That fallback is why this needed no
/// ABI change.
fn map_shim_error(raw: String) -> InferenceError {
    let unsupported = |reason: &'static str| InferenceError::UnsupportedMode {
        mode: "foundation-models",
        backend: "foundation-models",
        reason,
    };
    let Ok(parsed) = serde_json::from_str::<serde_json::Value>(&raw) else {
        // A tool-bearing session reports an overflow through an internal
        // framework type the shim cannot match by type, so only its text
        // identifies it (measured on macOS 27.0.1, car#2665).
        // Anchored: the framework's text leads the string, so an unrelated
        // error that merely quotes the phrase stays a failure.
        if raw.starts_with(UNTYPED_OVERFLOW.0) {
            if let Some((used, window)) = counts_after(&raw, UNTYPED_OVERFLOW) {
                return context_overflow(Some(used), Some(window), &raw);
            }
        }
        return InferenceError::InferenceFailed(raw);
    };
    let Some(kind) = parsed.get("car_fm_error").and_then(|k| k.as_str()) else {
        return InferenceError::InferenceFailed(raw);
    };
    let message = parsed
        .get("message")
        .and_then(|m| m.as_str())
        .unwrap_or(&raw)
        .to_string();

    match kind {
        // Retryable. `status: None` because this is on-device — there is no
        // HTTP status, and inventing one would misreport the source.
        "rateLimited" | "timeout" => InferenceError::Transient {
            status: None,
            message: format!("FoundationModels {kind}: {message}"),
        },
        // A request too large for the window, which is a ruling on the
        // request's size and not a model failure (car#2665). The shim sends
        // the numbers for `LanguageModelError`; the macOS 26
        // `GenerationError` carries them only in its text.
        "contextSizeExceeded" => {
            let (used, window) = structured_overflow_numbers(&parsed)
                .or_else(|| overflow_numbers(&message))
                .unzip();
            context_overflow(used, window, &message)
        }
        // Content-policy outcomes. Deliberately NOT transient: retrying the
        // same prompt reproduces them, and dressing a refusal as a blip would
        // burn the caller's retry budget on a deterministic answer.
        //
        // Typed as a content refusal, which is what it is: the OS's guardrail
        // ruled on the request, the model was not at fault, and the same
        // prompt gets the same ruling. Untyped, every decline booked as a
        // model failure — 58 in one day of voice turns.
        "guardrailViolation" | "refusal" => InferenceError::ContentRefused {
            usage: None,
            provider: "apple".to_string(),
            kind: Some(kind.to_string()),
            code: None,
            message,
        },
        // The OS model cannot do what was asked — a capability, transcript
        // content, generation guide or locale it does not support. A ruling on
        // the request's shape, like `UnsupportedMode` from any other backend.
        "unsupportedCapability" => unsupported("the requested capability"),
        "unsupportedTranscriptContent" => unsupported("this transcript content"),
        "unsupportedGenerationGuide" => unsupported("this generation guide"),
        "unsupportedLanguageOrLocale" => unsupported("this language or locale"),
        _ => InferenceError::InferenceFailed(format!("FoundationModels {kind}: {message}")),
    }
}

/// A context-window overflow, as the routing signal it is (car#2665).
///
/// The window is fixed and the request did not fit, so this is a ruling on the
/// request's size, typed like any backend's unsupported mode: it moves an
/// unpinned request on to the next candidate and stays off this model's
/// circuit breaker and health record. Before, it was an `InferenceFailed`, and
/// every `car do` turn that tried this model counted as the model failing,
/// because that turn's tool catalog alone exceeds the window. The numbers
/// that make it actionable go in the log, since the variant's reason is static.
fn context_overflow(used: Option<i64>, window: Option<i64>, detail: &str) -> InferenceError {
    match (used, window) {
        (Some(used), Some(window)) => tracing::warn!(
            used,
            window,
            "FoundationModels context exceeded: {used} tokens against a {window}-token window. \
             Instructions, tools, prompt and output all count against it."
        ),
        _ => tracing::warn!("FoundationModels context exceeded: {detail}"),
    }
    InferenceError::UnsupportedMode {
        mode: "context-window",
        backend: "foundation-models",
        reason: "the request does not fit the on-device model's context window \
                 (instructions, tools, prompt and output all count against it) — \
                 route to a model with a larger window or shorten the history",
    }
}

/// The untyped error a tool-bearing session raises on overflow, which the
/// shim cannot match by type: "Provided 20,590 tokens, but the maximum allowed
/// is 8,192." (measured on macOS 27.0.1).
const UNTYPED_OVERFLOW: (&str, &str) = ("Provided ", " tokens, but the maximum allowed is ");

/// The text of `GenerationError.exceededContextWindowSize`, which the shim
/// maps to `contextSizeExceeded` by type but whose counts live only here:
/// "Content contains 52944 tokens, which exceeds the maximum allowed context
/// size of 8192." (measured on macOS 27.0.1).
const GENERATION_OVERFLOW: (&str, &str) = (
    "Content contains ",
    " tokens, which exceeds the maximum allowed context size of ",
);

/// The `(used, window)` token counts a structured `contextSizeExceeded`
/// carries: `token_count` is what the request used, `context_size` the window.
fn structured_overflow_numbers(parsed: &serde_json::Value) -> Option<(i64, i64)> {
    let used = parsed.get("token_count").and_then(|v| v.as_i64())?;
    let window = parsed.get("context_size").and_then(|v| v.as_i64())?;
    Some((used, window))
}

/// The `(used, window)` token counts in either measured overflow text, for
/// the log line of a `contextSizeExceeded` that arrived without them.
fn overflow_numbers(text: &str) -> Option<(i64, i64)> {
    counts_after(text, UNTYPED_OVERFLOW).or_else(|| counts_after(text, GENERATION_OVERFLOW))
}

/// The two counts in `<lead><used><middle><window>`, commas allowed.
fn counts_after(text: &str, (lead, middle): (&str, &str)) -> Option<(i64, i64)> {
    let count = |s: &str| -> Option<i64> {
        let digits: String = s
            .chars()
            .take_while(|c| c.is_ascii_digit() || *c == ',')
            .filter(char::is_ascii_digit)
            .collect();
        digits.parse().ok()
    };
    let rest = &text[text.find(lead)? + lead.len()..];
    let gap = rest.find(middle)?;
    Some((count(&rest[..gap])?, count(&rest[gap + middle.len()..])?))
}

/// Whether Private Cloud Compute is usable *right now*, verified by use.
///
/// **`isAvailable` is not sufficient**, and that is why this probe generates.
/// On a machine with Apple Intelligence enabled but no Apple Account signed
/// in, PCC reports `availability == .available`, `isAvailable == true`, a
/// healthy `quotaUsage` and a 32768-token `contextSize` — then every
/// `respond()` fails with a bare `LanguageModelError` code -1 wrapping
/// `ModelManagerServices.ModelManagerError 1046`. Not a typed case, so the
/// error surface does not name the cause either.
///
/// PCC is authenticated by the user's Apple Account, so `isAvailable`
/// reflects *device eligibility*, not authentication. A catalog row gated on
/// it would advertise a route that fails at generation on any signed-out
/// machine — the same class of bug as a row claiming a window twice the real
/// one.
///
/// Costs one tiny generation per [`PCC_RECHECK`] at most. That is the price
/// of not lying about the route. The answer is not kept for the process's
/// life: an Apple Account signed in or out later must show without a restart,
/// so it is probed again once it is older than [`PCC_RECHECK`], and a PCC
/// generation that fails forgets it (car#1770).
pub fn pcc_available() -> bool {
    PCC_PROBE.available(
        || unsafe { car_fm_pcc_available() != 0 },
        std::time::Instant::now(),
    )
}

/// How long a PCC probe's answer stands.
const PCC_RECHECK: std::time::Duration = std::time::Duration::from_secs(60);

static PCC_PROBE: PccProbe = PccProbe::new();

/// The last PCC probe and when it ran. Held while a probe runs, so callers
/// arriving meanwhile wait for its answer instead of each starting one.
struct PccProbe(std::sync::Mutex<Option<(bool, std::time::Instant)>>);

impl PccProbe {
    const fn new() -> Self {
        Self(std::sync::Mutex::new(None))
    }

    fn available(&self, probe: impl FnOnce() -> bool, now: std::time::Instant) -> bool {
        let mut last = self
            .0
            .lock()
            .unwrap_or_else(|poisoned| poisoned.into_inner());
        if let Some((answer, at)) = *last {
            if now.saturating_duration_since(at) < PCC_RECHECK {
                return answer;
            }
        }
        let answer = probe();
        *last = Some((answer, now));
        answer
    }

    fn forget(&self) {
        *self
            .0
            .lock()
            .unwrap_or_else(|poisoned| poisoned.into_inner()) = None;
    }
}

/// PCC's context window, or `None` when unavailable. Measured 32768.
pub fn pcc_context_size() -> Option<u32> {
    static CACHE: OnceLock<Option<u32>> = OnceLock::new();
    *CACHE.get_or_init(|| {
        let raw = unsafe { car_fm_pcc_context_size() };
        u32::try_from(raw).ok().filter(|&v| v > 0)
    })
}

/// Blocking generation against Private Cloud Compute.
///
/// `reasoning_level` accepts `low`/`medium`/`high` (mapped onto Apple's
/// light/moderate/deep) or any other string, forwarded verbatim as a custom
/// level. PCC is the tier that declares the `reasoning` capability; the
/// on-device model does not, which is why this knob lives only here.
pub fn pcc_generate(
    prompt: &str,
    instructions: Option<&str>,
    reasoning_level: Option<&str>,
    max_tokens: u32,
    temperature: f32,
) -> Result<String, InferenceError> {
    if !pcc_available() {
        return Err(InferenceError::UnsupportedMode {
            mode: "private-cloud-compute",
            backend: "foundation-models",
            reason: "Private Cloud Compute is not usable on this device — it needs \
                     macOS 27+, an eligible device, and a signed-in Apple Account",
        });
    }
    let prompt_c = CString::new(prompt)
        .map_err(|e| InferenceError::InferenceFailed(format!("prompt has interior NUL: {e}")))?;
    let instructions_c = CString::new(instructions.unwrap_or(""))
        .map_err(|e| InferenceError::InferenceFailed(format!("instructions have NUL: {e}")))?;
    let reasoning_c = CString::new(reasoning_level.unwrap_or(""))
        .map_err(|e| InferenceError::InferenceFailed(format!("reasoning level has NUL: {e}")))?;

    let mut out_text: *mut c_char = ptr::null_mut();
    let mut out_err: *mut c_char = ptr::null_mut();
    let rc = unsafe {
        car_fm_pcc_generate(
            prompt_c.as_ptr(),
            instructions_c.as_ptr(),
            reasoning_c.as_ptr(),
            max_tokens.min(i32::MAX as u32) as i32,
            temperature as f64,
            &mut out_text,
            &mut out_err,
        )
    };
    if rc != 0 {
        // The account may have changed since the probe: ask again next time.
        PCC_PROBE.forget();
        let message = consume_swift_string(out_err);
        if rc == 4 {
            return Err(InferenceError::UnsupportedMode {
                mode: "private-cloud-compute",
                backend: "foundation-models",
                reason: "Private Cloud Compute is not available on this device",
            });
        }
        return Err(map_shim_error(message));
    }
    Ok(consume_swift_string(out_text))
}

/// Blocking single-shot generation. The caller is responsible for
/// running this on a blocking-friendly executor (the Swift bridge uses
/// a `DispatchSemaphore` to wait on the underlying async task).
pub fn generate(
    prompt: &str,
    instructions: Option<&str>,
    max_tokens: u32,
    temperature: f32,
) -> Result<String, InferenceError> {
    if !is_available() {
        return Err(unavailable_error());
    }

    let prompt_c = CString::new(prompt)
        .map_err(|e| InferenceError::InferenceFailed(format!("prompt has interior NUL: {e}")))?;
    let instr_c = match instructions {
        Some(s) if !s.is_empty() => Some(CString::new(s).map_err(|e| {
            InferenceError::InferenceFailed(format!("instructions have interior NUL: {e}"))
        })?),
        _ => None,
    };

    let mut out_text: *mut c_char = ptr::null_mut();
    let mut out_err: *mut c_char = ptr::null_mut();

    let rc = unsafe {
        car_fm_generate(
            prompt_c.as_ptr(),
            instr_c.as_ref().map_or(ptr::null(), |s| s.as_ptr()),
            max_tokens.min(i32::MAX as u32) as i32,
            temperature as f64,
            &mut out_text as *mut *mut c_char,
            &mut out_err as *mut *mut c_char,
        )
    };

    if rc != 0 {
        return Err(map_shim_error(consume_swift_string(out_err)));
    }
    Ok(consume_swift_string(out_text))
}

/// Warn once per degradation the Swift converter will apply to
/// `schema`. `what` names the schema's role in the log line (e.g.
/// `tool 'get_weather' parameters`, `response_format JsonSchema`).
fn warn_schema_degradations(what: &str, schema: &serde_json::Value) {
    for finding in schema_degradations(schema) {
        tracing::warn!(
            "FoundationModels constrained decoding: {what}: {finding} — the generated value is \
             preserved but this part of the schema contract is not enforced"
        );
    }
}

/// Blocking generation with a tool catalog. `tools` is the same
/// JSON-Schema tool array `GenerateRequest.tools` carries
/// (`[{name, description, parameters}]`). Returns the final text plus
/// any captured tool calls in the standard `ToolCall` shape the
/// remote backends emit — the runtime executes them, this backend
/// never does (see the module docs for the capture-and-return bridge).
///
/// The number of calls returned per turn is host-dependent: one on
/// macOS 26, every proposed call on macOS 27. See
/// [`supports_parallel_tool_calls`], which is what the router reads.
///
/// When a call was captured, `text` is **always empty**, on every host
/// — ending the turn on the capture sentinel discards any prose the
/// model produced before deciding to call the tool. That boundary is
/// independent of the parallel-call one and macOS 27 does NOT lift it:
/// capturing requires aborting the turn, and an aborted turn yields no
/// content. Lifting it would mean letting the framework run the call
/// and continuing the session, which is a different design, not a
/// capability bit (see the module-level Boundaries).
/// `tool_choice` accepts CAR's existing spellings — `auto`, `required`/`any`,
/// `none`, or a tool name — and is honoured from macOS 27 on, where
/// `GenerationOptions.toolCallingMode` exists. Below 27 it is **ignored, not
/// faked**: there is no way to enforce it, and pretending otherwise would let
/// a caller believe a forcing request was applied when it was not. A named
/// tool maps to "required" because the framework has no forcing-by-name
/// primitive; the runtime validates which tool actually came back regardless.
pub fn generate_with_tools(
    prompt: &str,
    instructions: Option<&str>,
    tools: &[serde_json::Value],
    tool_choice: Option<&str>,
    max_tokens: u32,
    temperature: f32,
) -> Result<(String, Vec<crate::tasks::generate::ToolCall>), InferenceError> {
    if !is_available() {
        return Err(unavailable_error());
    }

    let prompt_c = CString::new(prompt)
        .map_err(|e| InferenceError::InferenceFailed(format!("prompt has interior NUL: {e}")))?;
    let instr_c = match instructions {
        Some(s) if !s.is_empty() => Some(CString::new(s).map_err(|e| {
            InferenceError::InferenceFailed(format!("instructions have interior NUL: {e}"))
        })?),
        _ => None,
    };
    let mut sent_tools = tools.to_vec();
    for tool in &mut sent_tools {
        let name = tool
            .get("name")
            .and_then(|n| n.as_str())
            .unwrap_or("<unnamed>")
            .to_string();
        if let Some(params) = tool.get_mut("parameters") {
            warn_schema_degradations(&format!("tool '{name}' parameters"), params);
            *params = normalize_schema(params, &mut Vec::new());
        }
    }
    let tools_json = serde_json::to_string(&sent_tools)
        .map_err(|e| InferenceError::InferenceFailed(format!("tools serialization: {e}")))?;
    let choice_c = tool_choice
        .map(str::trim)
        .filter(|c| !c.is_empty())
        .and_then(|c| CString::new(c).ok());
    let tools_c = CString::new(tools_json)
        .map_err(|e| InferenceError::InferenceFailed(format!("tools have interior NUL: {e}")))?;

    let mut out_text: *mut c_char = ptr::null_mut();
    let mut out_calls: *mut c_char = ptr::null_mut();
    let mut out_err: *mut c_char = ptr::null_mut();

    let rc = unsafe {
        car_fm_generate_with_tools(
            prompt_c.as_ptr(),
            instr_c.as_ref().map_or(ptr::null(), |s| s.as_ptr()),
            tools_c.as_ptr(),
            choice_c.as_ref().map_or(ptr::null(), |s| s.as_ptr()),
            max_tokens.min(i32::MAX as u32) as i32,
            temperature as f64,
            &mut out_text as *mut *mut c_char,
            &mut out_calls as *mut *mut c_char,
            &mut out_err as *mut *mut c_char,
        )
    };

    if rc != 0 {
        return Err(map_shim_error(consume_swift_string(out_err)));
    }
    let text = consume_swift_string(out_text);
    let calls_json = consume_swift_string(out_calls);
    let mut tool_calls = parse_bridge_tool_calls(&calls_json)?;
    fill_tool_call_nulls(&mut tool_calls, tools);
    Ok((text, tool_calls))
}

/// Parse the shim's `[{"name": ..., "arguments": {...}}]` wire shape
/// into the standard [`ToolCall`] list. `id` is `None` — Foundation
/// Models has no provider call IDs; consumers synthesize positional
/// ones exactly as they do for other id-less backends.
fn parse_bridge_tool_calls(
    calls_json: &str,
) -> Result<Vec<crate::tasks::generate::ToolCall>, InferenceError> {
    if calls_json.trim().is_empty() {
        return Ok(vec![]);
    }
    #[derive(serde::Deserialize)]
    struct BridgeCall {
        name: String,
        #[serde(default)]
        arguments: std::collections::HashMap<String, serde_json::Value>,
    }
    let calls: Vec<BridgeCall> = serde_json::from_str(calls_json).map_err(|e| {
        InferenceError::InferenceFailed(format!(
            "FoundationModels bridge returned malformed tool-call JSON: {e}"
        ))
    })?;
    Ok(calls
        .into_iter()
        .map(|c| crate::tasks::generate::ToolCall {
            id: None,
            name: c.name,
            arguments: c.arguments,
        })
        .collect())
}

/// Blocking schema-guided generation (`ResponseFormat::JsonSchema`).
/// The JSON Schema is converted to a `DynamicGenerationSchema` and
/// enforced by Foundation Models' constrained decoding; the returned
/// string is the generated JSON document.
pub fn generate_structured(
    prompt: &str,
    instructions: Option<&str>,
    schema: &serde_json::Value,
    max_tokens: u32,
    temperature: f32,
) -> Result<String, InferenceError> {
    if !is_available() {
        return Err(unavailable_error());
    }

    let prompt_c = CString::new(prompt)
        .map_err(|e| InferenceError::InferenceFailed(format!("prompt has interior NUL: {e}")))?;
    let instr_c = match instructions {
        Some(s) if !s.is_empty() => Some(CString::new(s).map_err(|e| {
            InferenceError::InferenceFailed(format!("instructions have interior NUL: {e}"))
        })?),
        _ => None,
    };
    warn_schema_degradations("response_format JsonSchema", schema);
    let schema_json = serde_json::to_string(&normalize_schema(schema, &mut Vec::new()))
        .map_err(|e| InferenceError::InferenceFailed(format!("schema serialization: {e}")))?;
    let schema_c = CString::new(schema_json)
        .map_err(|e| InferenceError::InferenceFailed(format!("schema has interior NUL: {e}")))?;

    let mut out_json: *mut c_char = ptr::null_mut();
    let mut out_err: *mut c_char = ptr::null_mut();

    let rc = unsafe {
        car_fm_generate_structured(
            prompt_c.as_ptr(),
            instr_c.as_ref().map_or(ptr::null(), |s| s.as_ptr()),
            schema_c.as_ptr(),
            max_tokens.min(i32::MAX as u32) as i32,
            temperature as f64,
            &mut out_json as *mut *mut c_char,
            &mut out_err as *mut *mut c_char,
        )
    };

    if rc != 0 {
        return Err(map_shim_error(consume_swift_string(out_err)));
    }
    Ok(fill_structured_nulls(
        consume_swift_string(out_json),
        schema,
    ))
}

/// Token shape used by the streaming callback. The Swift side already
/// performs prefix-diffing on the cumulative snapshots Foundation
/// Models emits, so each delta is the newly-appended slice — no
/// further work needed on the consumer.
pub struct StreamCallback<'a> {
    on_delta: Box<dyn FnMut(&str) -> bool + Send + 'a>,
}

impl<'a> StreamCallback<'a> {
    /// `on_delta` is invoked for each incremental text fragment.
    /// Returning `false` cancels the stream.
    pub fn new<F>(on_delta: F) -> Self
    where
        F: FnMut(&str) -> bool + Send + 'a,
    {
        Self {
            on_delta: Box::new(on_delta),
        }
    }
}

extern "C" fn stream_trampoline(token: *const c_char, state: *mut c_void) -> c_int {
    // SAFETY: this function runs on a thread Swift owns; an unwinding
    // panic crossing back into Swift is undefined behavior. Wrap the
    // body in catch_unwind and translate panics into "cancel" so the
    // model turn aborts cleanly instead of taking the process down.
    let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
        if state.is_null() {
            return 1;
        }
        let cb = unsafe { &mut *(state as *mut StreamCallback) };
        let s = if token.is_null() {
            ""
        } else {
            match unsafe { CStr::from_ptr(token) }.to_str() {
                Ok(s) => s,
                Err(_) => return 1,
            }
        };
        if (cb.on_delta)(s) {
            0 // continue
        } else {
            1 // cancel
        }
    }));
    result.unwrap_or(1)
}

/// Blocking streaming generation. Each delta is forwarded to the
/// callback as a string slice owned by Swift — copy if it needs to
/// outlive the call.
pub fn stream(
    prompt: &str,
    instructions: Option<&str>,
    max_tokens: u32,
    temperature: f32,
    mut callback: StreamCallback<'_>,
) -> Result<(), InferenceError> {
    if !is_available() {
        return Err(unavailable_error());
    }

    let prompt_c = CString::new(prompt)
        .map_err(|e| InferenceError::InferenceFailed(format!("prompt has interior NUL: {e}")))?;
    let instr_c = match instructions {
        Some(s) if !s.is_empty() => Some(CString::new(s).map_err(|e| {
            InferenceError::InferenceFailed(format!("instructions have interior NUL: {e}"))
        })?),
        _ => None,
    };

    let mut out_err: *mut c_char = ptr::null_mut();
    let state: *mut c_void = &mut callback as *mut StreamCallback as *mut c_void;

    let rc = unsafe {
        car_fm_generate_stream(
            prompt_c.as_ptr(),
            instr_c.as_ref().map_or(ptr::null(), |s| s.as_ptr()),
            max_tokens.min(i32::MAX as u32) as i32,
            temperature as f64,
            stream_trampoline,
            state,
            &mut out_err as *mut *mut c_char,
        )
    };

    if rc != 0 {
        return Err(map_shim_error(consume_swift_string(out_err)));
    }
    Ok(())
}

// ---------------------------------------------------------------------
// Helpers.
// ---------------------------------------------------------------------

fn consume_swift_string(ptr: *mut c_char) -> String {
    if ptr.is_null() {
        return String::new();
    }
    let s = unsafe { CStr::from_ptr(ptr) }
        .to_string_lossy()
        .into_owned();
    unsafe { car_fm_free_string(ptr) };
    s
}

fn unavailable_error() -> InferenceError {
    InferenceError::UnsupportedMode {
        mode: "apple-foundation-models",
        backend: "foundation-models",
        reason: "FoundationModels framework reports unavailable on this host. Requires macOS 26+ \
             on Apple Silicon with Apple Intelligence enabled. Falling through to the next \
             router candidate.",
    }
}

#[cfg(test)]
mod tests {
    use super::{
        completion_text, count_tokens, map_shim_error, overflow_numbers, parse_bridge_tool_calls,
        pcc_available, pcc_context_size, pcc_generate, structured_overflow_numbers, PccProbe,
        PCC_RECHECK,
    };
    use crate::InferenceError;

    #[test]
    fn parses_bridge_tool_call_wire_shape() {
        let calls = parse_bridge_tool_calls(
            r#"[{"name":"get_weather","arguments":{"city":"Austin","days":3}}]"#,
        )
        .unwrap();
        assert_eq!(calls.len(), 1);
        assert_eq!(calls[0].name, "get_weather");
        assert_eq!(calls[0].id, None);
        assert_eq!(
            calls[0].arguments.get("city"),
            Some(&serde_json::json!("Austin"))
        );
        assert_eq!(calls[0].arguments.get("days"), Some(&serde_json::json!(3)));
    }

    #[test]
    fn empty_or_missing_calls_parse_to_empty() {
        assert!(parse_bridge_tool_calls("").unwrap().is_empty());
        assert!(parse_bridge_tool_calls("[]").unwrap().is_empty());
    }

    #[test]
    fn malformed_calls_json_is_an_error_not_a_silent_drop() {
        assert!(parse_bridge_tool_calls("{not json").is_err());
    }

    /// A PCC probe's answer stands for [`PCC_RECHECK`], then is asked again,
    /// and a forgotten one is asked again at once: an account signed in or
    /// out after startup shows without a restart (car#1770).
    #[test]
    fn a_pcc_probe_answer_is_rechecked_and_can_be_forgotten() {
        use std::sync::atomic::{AtomicU32, Ordering};
        let cache = PccProbe::new();
        let probes = AtomicU32::new(0);
        let probe = |answer: bool| {
            let probes = &probes;
            move || {
                probes.fetch_add(1, Ordering::SeqCst);
                answer
            }
        };
        let start = std::time::Instant::now();
        assert!(!cache.available(probe(false), start));
        assert!(!cache.available(probe(true), start + PCC_RECHECK / 2));
        assert_eq!(
            probes.load(Ordering::SeqCst),
            1,
            "within the window, not asked again"
        );
        assert!(cache.available(probe(true), start + PCC_RECHECK));
        assert_eq!(probes.load(Ordering::SeqCst), 2, "signed in later, seen");
        cache.forget();
        assert!(!cache.available(probe(false), start + PCC_RECHECK));
        assert_eq!(
            probes.load(Ordering::SeqCst),
            3,
            "forgotten, asked again at once"
        );
    }

    /// PCC's usability probe must agree with what `pcc_generate` will do.
    ///
    /// The point of the probe is that `isAvailable` lies on a signed-out
    /// machine: PCC reports available, healthy quota and a 32768-token
    /// window, then fails every generation with an untyped error. So the
    /// contract under test is "refusing to generate implies the probe said
    /// unavailable" — never the reverse, which would advertise a dead route.
    #[test]
    fn pcc_refuses_to_generate_exactly_when_it_reports_unavailable() {
        if pcc_available() {
            // Usable here: nothing to assert without spending a real cloud
            // round trip, and the probe already made one to say so.
            return;
        }
        let err = pcc_generate("hi", None, None, 8, 0.0)
            .expect_err("an unavailable PCC must not generate");
        assert!(
            matches!(err, InferenceError::UnsupportedMode { mode, .. } if mode
                == "private-cloud-compute"),
            "refusal must be a routing signal so the router falls through, got {err:?}"
        );
        // The message has to name the Apple Account, because that is the one
        // thing the framework's own error never says — it surfaces as a bare
        // code -1 wrapping ModelManagerError 1046.
        assert!(
            err.to_string().contains("Apple Account"),
            "refusal must name the actual cause: {err}"
        );
    }

    /// A context window is reported even where generation is refused, and
    /// that asymmetry is the evidence the probe exists for.
    #[test]
    #[ignore = "requires macOS 26 with FoundationModels Private Cloud Compute"]
    fn pcc_context_size_is_independent_of_usability() {
        let window = pcc_context_size().expect("PCC reports a context window");
        assert!(
            window >= 4096,
            "PCC's window should exceed the on-device model's, got {window}"
        );
    }

    /// A rate limit and a timeout must come back RETRYABLE.
    ///
    /// This is the behaviour change the structured errors bought. Before,
    /// every FoundationModels failure arrived as a flat string and became a
    /// non-retryable `InferenceFailed`, so a rate limit ended the request
    /// instead of backing off.
    #[test]
    fn transient_shim_errors_map_to_transient() {
        for kind in ["rateLimited", "timeout"] {
            let raw = format!("{{\"car_fm_error\":\"{kind}\",\"message\":\"slow down\"}}");
            match map_shim_error(raw) {
                InferenceError::Transient { status, message } => {
                    assert!(status.is_none(), "on-device failures have no HTTP status");
                    assert!(
                        message.contains(kind),
                        "message should name the kind: {message}"
                    );
                }
                other => panic!("{kind} must be Transient, got {other:?}"),
            }
        }
    }

    /// A refusal must NOT be retryable.
    ///
    /// Retrying the same prompt reproduces it, so dressing a content-policy
    /// outcome as a blip would burn the caller's retry budget on a
    /// deterministic answer.
    #[test]
    fn content_policy_outcomes_are_not_transient() {
        for kind in ["guardrailViolation", "refusal"] {
            let raw = format!("{{\"car_fm_error\":\"{kind}\",\"message\":\"no\"}}");
            let error = map_shim_error(raw);
            assert!(
                matches!(&error, InferenceError::ContentRefused { kind: Some(k), .. } if k == kind),
                "{kind} must be a typed content refusal, got {error:?}"
            );
            assert_eq!(
                error.to_string(),
                format!("apple refused this request on content grounds (type={kind}): no")
            );
        }
    }

    /// What the OS model cannot do is a ruling on the request, not a model
    /// failure: typed like any backend's unsupported mode.
    #[test]
    fn unsupported_kinds_are_unsupported_mode() {
        for kind in [
            "unsupportedCapability",
            "unsupportedTranscriptContent",
            "unsupportedGenerationGuide",
            "unsupportedLanguageOrLocale",
        ] {
            let raw = format!("{{\"car_fm_error\":\"{kind}\",\"message\":\"no\"}}");
            assert!(
                matches!(map_shim_error(raw), InferenceError::UnsupportedMode { .. }),
                "{kind}"
            );
        }
    }

    /// car#2665: a context overflow is a routing signal, not a model failure,
    /// in every form the framework reports it: the structured shim kind and
    /// the two raw strings measured on macOS 27.0.1. None of them may count
    /// against the circuit breaker or the model's health record.
    #[test]
    fn context_overflow_is_a_routing_signal_not_a_model_failure() {
        let forms = [
            "{\"car_fm_error\":\"contextSizeExceeded\",\"message\":\"too big\",\
             \"context_size\":4096,\"token_count\":5200}"
                .to_string(),
            "{\"car_fm_error\":\"contextSizeExceeded\",\"message\":\"Content contains 52944 \
             tokens, which exceeds the maximum allowed context size of 8192.\"}"
                .to_string(),
            "Provided 20,590 tokens, but the maximum allowed is 8,192.::Provided 20,590 tokens, \
             but the maximum allowed is 8,192.: The operation couldn\u{2019}t be completed. \
             (TokenGenerationInference.DecoderModelError error 3.)::inferenceFailed::The \
             operation couldn\u{2019}t be completed. (TokenGenerationInference.DecoderModelError \
             error 3.)"
                .to_string(),
        ];
        for raw in forms {
            let err = map_shim_error(raw.clone());
            assert!(
                matches!(
                    err,
                    InferenceError::UnsupportedMode {
                        mode: "context-window",
                        backend: "foundation-models",
                        ..
                    }
                ),
                "{raw} -> {err:?}"
            );
            assert!(
                !crate::error_counts_against_circuit_breaker(&err),
                "an overflow must not trip the breaker: {raw}"
            );
            assert_eq!(
                crate::failure_attribution(&err),
                Some(crate::outcome::UnattributedReason::Capability),
                "{raw}"
            );
        }
        // The numbers that make it actionable are recovered for the log.
        assert_eq!(
            overflow_numbers("Provided 20,590 tokens, but the maximum allowed is 8,192."),
            Some((20_590, 8_192))
        );
        assert_eq!(
            overflow_numbers(
                "Content contains 52944 tokens, which exceeds the maximum allowed context size \
                 of 8192."
            ),
            Some((52_944, 8_192))
        );
        assert_eq!(overflow_numbers("some opaque framework failure"), None);
        // `token_count` is the request, `context_size` the window.
        let structured: serde_json::Value = serde_json::from_str(
            "{\"car_fm_error\":\"contextSizeExceeded\",\"message\":\"too big\",\
             \"context_size\":4096,\"token_count\":5200}",
        )
        .unwrap();
        assert_eq!(structured_overflow_numbers(&structured), Some((5200, 4096)));
        // The untyped match is anchored: an error that only quotes the phrase
        // is still a failure.
        let quoted = "tool error: Provided 5 tokens, but the maximum allowed is 9".to_string();
        assert!(
            matches!(map_shim_error(quoted.clone()), InferenceError::InferenceFailed(m) if m == quoted),
            "an unanchored mention must stay InferenceFailed"
        );
    }

    /// Anything that is not the structured shape keeps the old behaviour
    /// verbatim — that fallback is why this needed no ABI change.
    #[test]
    fn unstructured_errors_pass_through_unchanged() {
        let raw = "some opaque framework failure".to_string();
        match map_shim_error(raw.clone()) {
            InferenceError::InferenceFailed(message) => assert_eq!(message, raw),
            other => panic!("plain strings must stay InferenceFailed, got {other:?}"),
        }
    }

    /// An empty completion must count as a real `0`, never as "unknown".
    ///
    /// This is the distinction the whole return type exists for. A turn can
    /// legitimately produce no completion text, and if that collapsed into
    /// `None` its receipt would go back to reporting nothing — which is the
    /// bug this counting was added to fix, reintroduced through the error path
    /// instead of the happy one. (A captured tool-call turn is not such a
    /// turn: its calls are counted, see `completion_text`.)
    ///
    /// Ignored by default: where the host cannot count at all (below macOS
    /// 26.4, or no shim), `None` is the correct answer for every input and the
    /// distinction does not exist to test.
    #[test]
    #[ignore = "requires macOS 26.4 FoundationModels token counting and the shim"]
    fn empty_completion_counts_as_zero_not_unknown() {
        let Some((prompt_tokens, completion_tokens)) = count_tokens(None, "hi", "") else {
            panic!("missing what this test requires: macOS 26.4 FoundationModels token counting and the shim");
        };
        assert_eq!(
            completion_tokens, 0,
            "an empty completion is zero tokens, not an unavailable count"
        );
        assert!(
            prompt_tokens > 0,
            "a non-empty prompt must count above zero, got {prompt_tokens}"
        );
    }

    /// car#1765: a captured tool-call turn returns empty text, so the
    /// completion handed to the tokenizer must carry the call itself — its
    /// name and every argument — or the turn counts as zero output tokens.
    #[test]
    fn tool_call_turn_completion_carries_the_call() {
        let calls = parse_bridge_tool_calls(
            r#"[{"name":"get_weather","arguments":{"city":"Austin","days":3}}]"#,
        )
        .unwrap();
        let completion = completion_text("", &calls);
        for generated in ["get_weather", "city", "Austin", "days", "3"] {
            assert!(
                completion.contains(generated),
                "{generated:?} was generated and must be counted: {completion:?}"
            );
        }
        let with_prose = completion_text("Checking.", &calls);
        assert!(
            with_prose.starts_with("Checking.") && with_prose.contains("get_weather"),
            "text and calls are both output: {with_prose:?}"
        );
        assert_eq!(completion_text("plain", &[]), "plain");
    }

    /// car#1765, end to end through the framework tokenizer: the turn the
    /// issue describes — a captured call, no text — counts above zero output
    /// tokens. Needs the tokenizer, which exists only from macOS 26.4.
    #[test]
    #[ignore = "requires macOS 26.4+ and the FoundationModels Swift shim"]
    fn captured_tool_call_counts_above_zero_output_tokens() {
        let calls = parse_bridge_tool_calls(
            r#"[{"name":"get_weather","arguments":{"city":"Austin","days":3}}]"#,
        )
        .unwrap();
        let (_, completion_tokens) =
            count_tokens(None, "weather in Austin?", &completion_text("", &calls))
                .expect("this host can count tokens");
        assert!(
            completion_tokens > 0,
            "a captured tool call is generated output, got {completion_tokens}"
        );
    }

    /// Instructions are counted as part of the input, not dropped.
    ///
    /// They occupy the same 4096-token window as the prompt (TN3193), so a
    /// count that ignored them would under-report exactly where the window
    /// matters most — a long system prompt.
    #[test]
    #[ignore = "requires macOS 26.4 FoundationModels token counting and the shim"]
    fn instructions_count_toward_the_input() {
        let Some((bare, _)) = count_tokens(None, "hi", "") else {
            panic!("missing what this test requires: macOS 26.4 FoundationModels token counting and the shim");
        };
        let Some((with_instructions, _)) =
            count_tokens(Some("You are a concise and careful assistant."), "hi", "")
        else {
            return;
        };
        assert!(
            with_instructions > bare,
            "instructions must add to the input count: {with_instructions} vs {bare}"
        );
    }
}