Struct StopTokens 

pub struct StopTokens { /* private fields */ }

The set of token ids a decode loop must stop on, carried as one value so a caller cannot accidentally carry only half of it.

This type exists because Option<usize> was the shape of a real bug: every ferrox-server decode loop threaded a single eos_id from the loader to the sampler, so a Llama-3 or gemma checkpoint served over HTTP ran past <|eot_id|> / <end_of_turn> to max_tokens even after eog_token_ids landed for the CLI. Passing a StopTokens makes “I only have the metadata EOS” an explicit choice (StopTokens::from_eos, for the synthetic-weights and Kimi paths that have no GGUF metadata to read) rather than the default.

Implementations

impl StopTokens

pub fn from_gguf(file: &impl TensorSource) -> Self

Everything eog_token_ids finds in this checkpoint: the eos/eot/eom metadata ids plus every vocabulary entry whose text is on llama.cpp’s literal EOG list.

pub fn from_eos(eos: Option<usize>) -> Self

Just the one id. For callers with no GGUF metadata behind them — the synthetic random-weights demo model, and the Kimi checkpoint directory whose tokenizer is a separate file format.

pub fn from_special_tokens<'a>( specials: impl IntoIterator<Item = (&'a str, u32)>, ) -> Self

For checkpoints whose vocabulary is not GGUF metadata — Kimi K3 ships a tokenizer_config.json with a name→id special-token map. Folds in every entry whose text is on llama.cpp’s EOG list, so [EOT] stops a turn there exactly as it does in a GGUF.

pub fn with_id(self, id: Option<usize>) -> Self

Folds one more id in — used to keep a metadata eos_token_id that a vocabulary spells in a way the literal list does not know.

pub fn contains(&self, id: usize) -> bool

pub fn is_empty(&self) -> bool

pub fn len(&self) -> usize

Trait Implementations

impl Clone for StopTokens

fn clone(&self) -> StopTokens

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for StopTokens

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for StopTokens

fn default() -> StopTokens

Returns the “default value” for a type.