Trait Decoder 

pub trait Decoder: Send + Sync {
    // Required method
    fn decode(
        &self,
        token_ids: &[TokenIdType],
        skip_special_tokens: bool,
    ) -> Result<String>;
}

Core decoding trait - can be implemented independently

Required Methods

fn decode( &self, token_ids: &[TokenIdType], skip_special_tokens: bool, ) -> Result<String>

Implementors

impl Decoder for CachedTokenizer

impl Decoder for HuggingFaceTokenizer

impl Decoder for MockTokenizer