Enum ModelSource 

pub enum ModelSource {
    Local {
        hf_repo: String,
        hf_filename: String,
        tokenizer_repo: String,
    },
    RemoteApi {
        endpoint: String,
        api_key_env: String,
        api_key_envs: Vec<String>,
        api_version: Option<String>,
        protocol: ApiProtocol,
    },
    CodexCli {
        model: String,
    },
    Ollama {
        model_tag: String,
        host: String,
    },
    Mlx {
        hf_repo: String,
        hf_weight_file: Option<String>,
    },
    WhisperCpp {
        model: String,
    },
    WindowsSpeech {},
    VllmMlx {
        endpoint: String,
        model_name: String,
    },
    ManagedVllmMlx {
        hf_repo: String,
        hf_weight_file: Option<String>,
    },
    AppleFoundationModels {
        use_case: Option<String>,
    },
    Proprietary {
        provider: String,
        endpoint: String,
        auth: ProprietaryAuth,
        protocol: ProprietaryProtocol,
    },
    Delegated {
        hint: Option<String>,
    },
}

How to access the model.

Variants

Local

Local GGUF file via Candle backend.

Fields

hf_repo: String

hf_filename: String

tokenizer_repo: String

RemoteApi

Remote API endpoint (OpenAI-compatible, Anthropic, etc.)

Fields

endpoint: String

api_key_env: String

Environment variable name containing the API key (never the key itself). The env var value may contain comma-separated keys for load balancing.

api_key_envs: Vec<String>

Additional environment variable names for load balancing across multiple keys. Each env var may also contain comma-separated keys.

api_version: Option<String>

protocol: ApiProtocol

CodexCli

One text-generation turn through the locally installed OpenAI Codex CLI.

Codex remains the sole holder of its ChatGPT-subscription credential: CAR neither reads that credential nor accepts an API key for this source. An optional reasoning suffix is part of the model string (for example, gpt-5.6-sol:high) and maps to Codex’s model_reasoning_effort config.

Fields

model: String

Ollama

Ollama local server.

Fields

model_tag: String

host: String

Mlx

Local MLX model via mlx-rs backend (Apple Silicon, safetensors format). Models from mlx-community on HuggingFace.

Fields

hf_repo: String

HuggingFace repo (e.g., “mlx-community/Qwen3-4B-4bit”).

hf_weight_file: Option<String>

Optional specific weight filename. If None, auto-discovers safetensors files.

WhisperCpp

Local whisper.cpp speech-to-text model — a ggml .bin from the ggerganov/whisper.cpp HF repo, run in-process via the shared car-whisper crate. Cross-platform (Windows/Linux/macOS): this is the on-device STT path where MLX isn’t available. Cached at ~/.tokhn/whisper/ggml-<model>.bin.

Fields

model: String

whisper.cpp model id — the suffix of ggml-<model>.bin (e.g. "large-v3-turbo-q5_0").

WindowsSpeech

Windows OS text-to-speech via Windows.Media.SpeechSynthesis (WinRT), run in-process. The catalog-side analog of the car-voice TtsProvider::WindowsSpeech live path and the parity counterpart of Apple’s OS synthesizer — free, on-device, no model download, no MLX. Windows-only; availability is false on every other target (like AppleFoundationModels).

VllmMlx

Local vLLM-MLX server (Apple Silicon, OpenAI-compatible API). Routes through RemoteBackend with OpenAI protocol handler.

Fields

endpoint: String

Server endpoint (e.g., “http://localhost:8000”).

model_name: String

The model name as known to vLLM-MLX (e.g., “mlx-community/Qwen3-4B-4bit”).

ManagedVllmMlx

CAR-owned supervised vLLM-MLX process backed by a managed HuggingFace artifact. Unlike VllmMlx, CAR downloads, admits, spawns, reaps, and accounts this allocation. Dispatch rewrites a clone to VllmMlx only after the child reports healthy.

Fields

hf_repo: String

hf_weight_file: Option<String>

AppleFoundationModels

Apple’s on-device system model via the FoundationModels framework (macOS 26+, Apple Silicon). Inference happens in-process through a Swift shim — there is no HTTP, no API key, and no model file: the OS owns the weights. Availability is checked at runtime via @available(macOS 26.0, *); on older macOS or non-Apple-Silicon hosts the backend reports UnsupportedMode and the router falls through to the next candidate.

Fields

use_case: Option<String>

Optional Apple use-case hint passed through to LanguageModelSession. Apple’s framework tunes its prompt and safety scaffolding per use case (e.g. “general”, “summarize”). None uses the default.

Proprietary

Proprietary provider with custom auth and protocol.

For vendor-specific APIs that aren’t generic OpenAI-compatible endpoints. Parslee is the first proprietary provider — custom auth (OAuth2), custom response format, multi-provider routing built into the API.

Fields

provider: String

Provider identifier (e.g., “parslee”).

endpoint: String

Base URL for the API.

auth: ProprietaryAuth

Auth configuration.

protocol: ProprietaryProtocol

Custom protocol details.

Delegated

Inference is delegated to a host-registered runner. CAR does not own the wire format — the runner (typically a JS / Python host) translates the GenerateRequest to its provider’s API, streams chunks back through the runner’s event callback, and returns the final aggregated result.

Closes Parslee-ai/car-releases#24. Use this when the host already has an SDK relationship with a provider (Anthropic, OpenAI, GitHub Models, Vercel AI SDK) and wants CAR to sit in the lifecycle / policy / replay path without learning every provider’s wire format.

Routing requires that a runner has been registered via crate::set_inference_runner (or its FFI equivalent — registerInferenceRunner on JS, register_inference_runner on Python, the InferenceRunner foreign trait on UniFFI, inference.register_runner on the WebSocket protocol). Without a runner, dispatch fails with InferenceFailed.

Fields

hint: Option<String>

Opaque hint passed through to the runner — typically the provider id ("anthropic", "openai", "vercel-ai-sdk") so a multi-provider runner can dispatch internally. CAR does not interpret this string.

Implementations

impl ModelSource

pub fn kind(&self) -> &'static str

The source’s serialized type tag (codex_cli, remote_api, …): where and how the model runs. Two rows of one model on different sources are different routes, with their own credentials and billing.

Trait Implementations

impl Clone for ModelSource

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ModelSource

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for ModelSource

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl JsonSchema for ModelSource

fn schema_name() -> String

The name of the generated JSON Schema.

fn schema_id() -> Cow<'static, str>

Returns a string that uniquely identifies the schema produced by this type.

fn json_schema(generator: &mut SchemaGenerator) -> Schema

Generates a JSON Schema for this type.

fn is_referenceable() -> bool

Whether JSON Schemas generated for this type should be re-used where possible using the $ref keyword.

impl Serialize for ModelSource

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.