Skip to main content

Decoder

llm_tokenizer::traits

Trait Decoder 

Source 
pub trait Decoder: Send + Sync {
    // Required method
    fn decode(
        &self,
        token_ids: &[TokenIdType],
        skip_special_tokens: bool,
    ) -> Result<String>;
}
Expand description

Core decoding trait - can be implemented independently

Required Methods§

Source

fn decode( &self, token_ids: &[TokenIdType], skip_special_tokens: bool, ) -> Result<String>

Implementors§