tekken 0.2.0

Rust implementation of Mistral Tekken tokenizer with image and audio support
Documentation