Crate lindera_analysis 

Text analysis chain for Lindera.

This crate layers Lucene-style text analysis on top of the pure morphological segmenter provided by the lindera crate:

• character_filter: transforms the input text before segmentation (with offset correction back to the original text)
• token_filter: transforms the tokens produced by the segmenter
• tokenizer: composes character filters, a Segmenter, and token filters into a single pipeline, configurable programmatically or via a YAML file

Modules

character_filter

token_filter

tokenizer

worker