Enum ExecutionProvider 

pub enum ExecutionProvider {
    Cpu,
    Auto,
    DirectMl,
    CoreMl,
}

Where inference runs.

Cpu is the default. Auto registers the platform accelerator (DirectML on Windows, CoreML on macOS) and falls back to CPU, with the reason recorded, when registration, model load, the probe inference or the vector parity check fails. DirectMl / CoreMl request that accelerator explicitly and fail loudly instead of falling back: a failed parity check is EmbedError::AcceleratorParity.

An accelerator is used only if it reproduces the CPU provider’s vectors. Whenever the session was built on anything other than CPU, Embedder::load builds a CPU session for the same model, embeds fixed short, medium and long prompts (alone and as one batch) through both sessions with the crate’s real inference path, and requires every pair to have cosine similarity of at least ACCELERATOR_PARITY_MIN_COSINE on the full native-dimension vector, with finite non-zero output. On success the CPU session is dropped before load returns, so steady-state memory is unchanged; the check costs one extra CPU session build and a few inferences at load. Cpu does none of this. The check cannot be turned off.

Measured: DirectML (ONNX Runtime 1.24.4) returns wrong EmbeddingGemma 2 Q4 vectors (cosine to CPU 0.58-0.72) that are finite, non-zero and faster than CPU, so a probe or speed check alone would accept them. CoreML is unmeasured; it must pass the same check.

Variants

Cpu

Auto

DirectMl

CoreMl

Trait Implementations

impl Clone for ExecutionProvider

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for ExecutionProvider

impl Debug for ExecutionProvider

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ExecutionProvider

fn default() -> Self

Returns the “default value” for a type.

impl Eq for ExecutionProvider

impl From<ExecutionProvider> for ExecutionProvider

fn from(value: ExecutionProvider) -> Self

Converts to this type from the input type.

impl PartialEq for ExecutionProvider

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for ExecutionProvider