1 2 3 4 5 6
#Tokenizer-API An API to interface a tokenizer with tantivy. The API will be kept stable in order to not break support for existing tokenizers.