Function generate 

pub fn generate<M: Model>(
    model: &M,
    tokenizer: &dyn Tokenizer,
    prompt: &str,
    config: &GenerationConfig,
    on_token: impl FnMut(u32, &str),
) -> Result<String>

Greedily generates a completion for prompt.

Runs one “prefill” forward pass over the whole encoded prompt, then repeatedly samples the highest-scoring next token (crate::sampling::GreedySampler — see that module’s docs for why greedy is what this crate implements today), feeds it back through model one token at a time (each call appending to the same crate::kv_cache::KvCache, obtained via Model::new_cache), and stops after config.max_new_tokens tokens or upon sampling config.eos_token_id, whichever comes first.

Calls on_token(id, text) once per generated token, in order, before that token is folded into the running completion — so a caller can render tokens as they arrive instead of waiting for the whole completion (see this module’s docs). Returns the full completion text (every generated token, decoded together so multi-token Unicode characters join correctly — see kopitiam_tokenizer::Tokenizer::decode’s docs on why decoding token-by-token can split a character but decoding the whole sequence at the end never does).

Errors

Propagates any kopitiam_core::Error from tokenizing, from a forward pass (including kopitiam_core::Error::IndexOutOfBounds if generation would exceed the model’s context window — see crate::kv_cache::KvCache::append), or from decoding.