Stability tier: Experimental
This is a pure ML inference kernel with high churn, 153 unsafe blocks, and 22
dead_code_allows. It is NOT intended for direct use by platform or feature crates.
Consumers should go through lattice-embed. The unsafe blocks are documented in
foundation/STABILITY.md §Tech Debt. Tracking issue: #1306.
See foundation/STABILITY.md for the full policy.
lattice-inference: pure Rust transformer inference for embedding models.
Supports two architectures:
- BERT/BGE (encoder-only): bidirectional attention, mean pooling
- Qwen3 (decoder-only): causal GQA with RoPE, SwiGLU, last-token pooling
Module Organization
- [
model] — Model configs and loaders (BERT, Qwen, Qwen3.5, BitNet) - [
tokenizer] — Tokenizers (WordPiece, SentencePiece, BPE) - [
weights] — Weight storage formats (f32, f16, Q8) - [
attention] — Attention mechanisms (standard, GQA, flash, GDN) - [
forward] — Compute backends (CPU, NEON, Metal GPU, batched prefill)