snaptokens 0.2.6

Fast BPE tokenization from local Hugging Face tokenizer.json files
Documentation