Struct DecodeStream 

pub struct DecodeStream { /* private fields */ }

DecodeStream will keep the state necessary to produce individual chunks of strings given an input stream of token_ids

Implementations

impl DecodeStream

pub fn new( tokenizer: Arc<dyn Tokenizer>, prompt_token_ids: &[TokenIdType], skip_special_tokens: bool, ) -> Self

pub fn step(&mut self, id: TokenIdType) -> Result<Option<String>>

Step appends a token_id to the internal state and tries to produce a text chunk. Returning None means the given id is not enough to produce a chunk.

pub fn step_batch(&mut self, token_ids: &[u32]) -> Result<Vec<String>>

Process multiple tokens at once

pub fn flush(&mut self) -> Result<Option<String>>

Force flush any remaining text

pub fn tokens(&self) -> &[u32]

Get all tokens processed so far