lattice-inference 0.9.0

Pure Rust transformer inference engine — safetensors loading, SIMD matmul, BGE/Qwen3 embeddings
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

/// Why autoregressive generation terminated.
///
/// Returned as `GenerateOutput::stop_reason`. All real generation exit paths set `Some(…)`.
/// `None` is reserved for non-generation returns (empty prompt, stub code paths) that do not
/// correspond to any of these causes.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[non_exhaustive]
pub enum StopReason {
    /// EOS token, configured stop-token id, or stop-string match.
    Eos,
    /// Grammar-constrained generation reached a state with no valid continuation.
    Grammar,
    /// `max_new_tokens` budget (or answer-budget cap) exhausted before a stop condition.
    Length,
    /// KV cache / context capacity reached before a stop condition.
    KvFull,
    /// Streaming callback returned `false`, requesting cancellation.
    Interrupt,
}