Struct CoreMlUnits 

pub struct CoreMlUnits { /* private fields */ }

The compute units of each Core ML function (embed_32 .. embed_512).

Self::per_length_default is what a Core ML bundle uses unless EmbedderConfig::with_coreml_units says otherwise. Each function is loaded once on its own unit and kept resident.

Implementations

impl CoreMlUnits

pub fn per_length_default() -> Self

The measured default (Mac matrix, RightKit issue #81): cpu_and_ne for graphs of up to 128 tokens (32, 64, 128; p50 about 2.3-3.0 ms) and cpu_and_gpu for 256 and 512 (p50 about 5.1 ms and 9.8-18.7 ms). The first cpu_and_gpu call per length can take up to 51 s cold (shader build), which is why load pre-warms every function.

pub fn uniform(unit: CoreMlComputeUnit) -> Self

Every function on unit (forces all, cpu_and_ne, cpu_and_gpu or cpu_only).

pub fn with_length(self, length: usize, unit: CoreMlComputeUnit) -> Option<Self>

This map with the function of length tokens moved to unit. None when length is not one of 32, 64, 128, 256, 512.

pub fn unit_for(&self, length: usize) -> Option<CoreMlComputeUnit>

The unit of the function of length tokens; None for a length that is not a graph.

pub fn entries(&self) -> impl Iterator<Item = (usize, CoreMlComputeUnit)> + '_

(length, unit) for every function, shortest first.

pub fn is_cpu_only(&self) -> bool

True when every function runs on cpu_only.

Trait Implementations

impl Clone for CoreMlUnits

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CoreMlUnits

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CoreMlUnits

fn default() -> Self

Returns the “default value” for a type.

impl Display for CoreMlUnits

fn fmt(&self, f: &mut Formatter<'_>) -> Result

32=cpu_and_ne,64=cpu_and_ne,...: parses back to the same map.

impl Eq for CoreMlUnits

impl From<CoreMlComputeUnit> for CoreMlUnits

fn from(unit: CoreMlComputeUnit) -> Self

Every function on unit, like CoreMlUnits::uniform.

impl FromStr for CoreMlUnits

fn from_str(text: &str) -> Result<Self, Self::Err>

A caller-facing override string (a setting or environment variable):

• default or per_length: Self::per_length_default;
• a bare unit (all, cpu_and_ne, cpu_and_gpu, cpu_only): every function on it;
• length=unit pairs separated by commas (256=cpu_only,512=cpu_only): those lengths override the per-length default, the others keep it. A length may appear once.

type Err = CoreMlUnitsParseError

The associated error which can be returned from parsing.

impl PartialEq for CoreMlUnits

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for CoreMlUnits