lattice-inference 0.4.2

Pure Rust transformer inference engine — safetensors loading, SIMD matmul, BGE/Qwen3 embeddings
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
//! QuaRot: Hadamard-rotated 4-bit quantization (Ashkboos et al., NeurIPS 2024).
//!
//! See [ADR-044](../../../../docs/adr/ADR-044-quarot-rotated-quantization.md).

pub mod convert;
pub mod forward_equivalence;
pub mod hadamard;
pub mod io;
pub mod lm_head;
pub mod lora;
pub mod pipeline;
pub mod plan;
pub mod rmsnorm_fusion;
pub mod rotation;

pub use io::{QuarotTensorReader, SourceDType};