Enum Precision 

pub enum Precision {
    F16,
    F32,
    Int8,
}

The number format. The CPU computes in FP32 for both float settings.

Variants

F16

FP16 weights and GEMM inputs with FP32 accumulation, as Laya’s autocast runs on a GPU.

F32

FP32 throughout, closest to Laya on the CPU.

Int8

INT8 weights and activations in the encoder and decision head GEMMs on the CPU, with the scorer in FP32. Faster, and further from Laya than FP32: see spec/10-cpu.md for the gate a checkpoint has to pass. Not on GPUs yet.

Trait Implementations

impl Clone for Precision

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for Precision

impl Debug for Precision

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Precision

fn default() -> Self

Returns the “default value” for a type.

impl Eq for Precision

impl PartialEq for Precision

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for Precision