Expand description
Pure-Rust (candle) port of the Cactus needle-1 Simple Attention Network — the encoder-decoder we CAN faithfully reimplement (its architecture is documented; needle-2’s novel MHC/engram/MTP stack is not). This is the model the finetuning harness trains: load the published bf16 safetensors, run teacher-forced, and serve fully in-Rust (no cactus C++ wheel, no Python).
Architecture (config.json + neural/needle_torch.py reference): d_model 512, 8 heads / 4 KV (GQA),
head_dim 64, vocab 8192, RoPE θ=1e4, ZCRMSNorm (1+scale)·x/rms(x). Encoder ×12: pre-norm →
GQA self-attn (per-head q/k ZCRMSNorm + RoPE) → sigmoid-gated residual (NO FFN). Decoder ×8: gated
self-attn + gated cross-attn (NO FFN). Tied-ish lm_head (shipped separately).