Expand description
Tokenizer implementation for morphological analysis.
This module provides a builder pattern for creating tokenizers and the tokenizer itself.
Structsยง
- JsTokenizer
- Tokenizer for performing morphological analysis.
- JsTokenizer
Builder - Builder for creating a Tokenizer with custom configuration.