Expand description
Host-side sampler chain (BASE-2, MEMRA-BUILD-MAP §BASE-2). Ports llama.cpp CPU sampler semantics (llama-sampler.cpp): repetition/freq/presence penalties -> temperature -> top-k -> top-p -> min-p -> categorical draw. Greedy (temp<=0) = argmax, the bit-exact reference.
Runs on the host over the full [n_vocab] f32 logit vector already brought back by the per-step D2H sync (decode.rs) — at B=2-4 this is single-µs, no GPU kernel needed (the GPU-fused sampler is a deferred PERF item, only needed once CUDA-graph removes the D2H barrier).
Structs§
- Sampler
- Stateful sampler: owns the RNG + the recent-token history (for penalties).
- Sampler
Config - Sampler configuration. Defaults = greedy (temp 0). Order of application matches llama.cpp.