Library containing text tokenization and related functionality, for preparing
inputs and decoding outputs for text models (eg. BERT).
The functionality is a subset of that found in [Hugging Face
Tokenizers](https://github.com/huggingface/tokenizers). It has less
functionality, but also fewer dependencies, and none that require C/C++.