lattice-inference 0.2.2

Pure Rust transformer inference engine — safetensors loading, SIMD matmul, BGE/Qwen3 embeddings
Documentation
lattice-inference-0.2.2 has been yanked.

Stability tier: Experimental

This is a pure ML inference kernel with high churn, 153 unsafe blocks, and 22 dead_code_allows. It is NOT intended for direct use by platform or feature crates. Consumers should go through lattice-embed. The unsafe blocks are documented in foundation/STABILITY.md §Tech Debt. Tracking issue: #1306. See foundation/STABILITY.md for the full policy.

lattice-inference: pure Rust transformer inference for embedding models.

Supports two architectures:

  • BERT/BGE (encoder-only): bidirectional attention, mean pooling
  • Qwen3 (decoder-only): causal GQA with RoPE, SwiGLU, last-token pooling

Module Organization

  • [model] — Model configs and loaders (BERT, Qwen, Qwen3.5, BitNet)
  • [tokenizer] — Tokenizers (WordPiece, SentencePiece, BPE)
  • [weights] — Weight storage formats (f32, f16, Q8)
  • [attention] — Attention mechanisms (standard, GQA, flash, GDN)
  • [forward] — Compute backends (CPU, NEON, Metal GPU, batched prefill)