Struct Options 

#[non_exhaustive]
pub struct Options {
    pub cache_capacity: usize,
    pub shapeless: bool,
}

Options of compile, compile_mut and compile_flat_with: Options::default().cache_capacity(8).

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

cache_capacity: usize

The most traced signatures a compiled function keeps (least recently used first out); 0 never caches. Default 64.

shapeless: bool

Trace once per input structure, dtypes and ranks, and replay for any shapes (Options::shapeless). Default off.

Implementations

impl Options

pub fn cache_capacity(self, n: usize) -> Self

Sets the cache capacity.

pub fn shapeless(self, on: bool) -> Self

Shapeless compilation (MLX’s shapeless=True): the function is traced once per input structure, dtypes and ranks, and replayed for any shapes of those ranks. Each replay infers the shapes again, and element-wise operations broadcast their operands as the uncompiled ones do, so the results are the shaped compile’s bit for bit.

use nuvec_compile::Options;
use nuvec_core::{Array, DType, ops};

let swiglu = Options::default()
    .shapeless(true)
    .compile(|(g, u): &(Array, Array)| g.multiply(g.sigmoid()).multiply(u));
for rows in [7, 512, 2048] {
    let g = ops::ones([rows, 64], DType::F32);
    swiglu(&(g.clone(), g))?.eval()?; // traced once, for every row count
}

• Operations whose parameters hold traced sizes fail the trace with an error naming them: reshapes, slices and slice updates, broadcasts other than an element-wise operation’s own, random draws, custom Metal kernels and attention (whose kernel choice depends on the lengths). Compile those without the option.
• As in MLX, Rust code that reads a shape while tracing (x.shape()[0], a branch on a size) bakes the first call’s value in, and constants built from a traced shape keep it: a replay on another shape then fails in the operation that uses them.
• compile_mut keys its state exactly and its inputs shapelessly.

pub fn compile<'f, I, O, F, R>(self, f: F) -> impl Fn(&I) -> Result<O> + 'f

where I: Tree, O: Tree, F: Fn(&I) -> R + 'f, R: IntoResult<Ok = O>,

compile with these options.

pub fn compile_mut<'f, S, I, O, F, R>( self, f: F, ) -> impl Fn(&mut S, &I) -> Result<O> + 'f

where S: Tree, I: Tree, O: Tree, F: Fn(&mut S, &I) -> R + 'f, R: IntoResult<Ok = O>,

compile_mut with these options.

Trait Implementations

impl Clone for Options

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Options

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Options

fn default() -> Self

Returns the “default value” for a type.