Enum ModelSource 

pub enum ModelSource {
    Local {
        hf_repo: String,
        hf_filename: String,
        tokenizer_repo: String,
    },
    RemoteApi {
        endpoint: String,
        api_key_env: String,
        api_key_envs: Vec<String>,
        api_version: Option<String>,
        protocol: ApiProtocol,
    },
    Ollama {
        model_tag: String,
        host: String,
    },
    Mlx {
        hf_repo: String,
        hf_weight_file: Option<String>,
    },
    VllmMlx {
        endpoint: String,
        model_name: String,
    },
    AppleFoundationModels {
        use_case: Option<String>,
    },
    Proprietary {
        provider: String,
        endpoint: String,
        auth: ProprietaryAuth,
        protocol: ProprietaryProtocol,
    },
    Delegated {
        hint: Option<String>,
    },
}

How to access the model.

Variants

Local

Local GGUF file via Candle backend.

Fields

hf_repo: String

hf_filename: String

tokenizer_repo: String

RemoteApi

Remote API endpoint (OpenAI-compatible, Anthropic, etc.)

Fields

endpoint: String

api_key_env: String

Environment variable name containing the API key (never the key itself). The env var value may contain comma-separated keys for load balancing.

api_key_envs: Vec<String>

Additional environment variable names for load balancing across multiple keys. Each env var may also contain comma-separated keys.

api_version: Option<String>

protocol: ApiProtocol

Ollama

Ollama local server.

Fields

model_tag: String

host: String

Mlx

Local MLX model via mlx-rs backend (Apple Silicon, safetensors format). Models from mlx-community on HuggingFace.

Fields

hf_repo: String

HuggingFace repo (e.g., “mlx-community/Qwen3-4B-4bit”).

hf_weight_file: Option<String>

Optional specific weight filename. If None, auto-discovers safetensors files.

VllmMlx

Local vLLM-MLX server (Apple Silicon, OpenAI-compatible API). Routes through RemoteBackend with OpenAI protocol handler.

Fields

endpoint: String

Server endpoint (e.g., “http://localhost:8000”).

model_name: String

The model name as known to vLLM-MLX (e.g., “mlx-community/Qwen3-4B-4bit”).

AppleFoundationModels

Apple’s on-device system model via the FoundationModels framework (macOS 26+, Apple Silicon). Inference happens in-process through a Swift shim — there is no HTTP, no API key, and no model file: the OS owns the weights. Availability is checked at runtime via @available(macOS 26.0, *); on older macOS or non-Apple-Silicon hosts the backend reports UnsupportedMode and the router falls through to the next candidate.

Fields

use_case: Option<String>

Optional Apple use-case hint passed through to LanguageModelSession. Apple’s framework tunes its prompt and safety scaffolding per use case (e.g. “general”, “summarize”). None uses the default.

Proprietary

Proprietary provider with custom auth and protocol.

For vendor-specific APIs that aren’t generic OpenAI-compatible endpoints. Parslee is the first proprietary provider — custom auth (OAuth2), custom response format, multi-provider routing built into the API.

Fields

provider: String

Provider identifier (e.g., “parslee”).

endpoint: String

Base URL for the API.

auth: ProprietaryAuth

Auth configuration.

protocol: ProprietaryProtocol

Custom protocol details.

Delegated

Inference is delegated to a host-registered runner. CAR does not own the wire format — the runner (typically a JS / Python host) translates the GenerateRequest to its provider’s API, streams chunks back through the runner’s event callback, and returns the final aggregated result.

Closes Parslee-ai/car-releases#24. Use this when the host already has an SDK relationship with a provider (Anthropic, OpenAI, GitHub Models, Vercel AI SDK) and wants CAR to sit in the lifecycle / policy / replay path without learning every provider’s wire format.

Routing requires that a runner has been registered via crate::set_inference_runner (or its FFI equivalent — registerInferenceRunner on JS, register_inference_runner on Python, the InferenceRunner foreign trait on UniFFI, inference.register_runner on the WebSocket protocol). Without a runner, dispatch fails with InferenceFailed.

Fields

hint: Option<String>

Opaque hint passed through to the runner — typically the provider id ("anthropic", "openai", "vercel-ai-sdk") so a multi-provider runner can dispatch internally. CAR does not interpret this string.

Trait Implementations

impl Clone for ModelSource

fn clone(&self) -> ModelSource

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ModelSource

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for ModelSource

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for ModelSource

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.