tantivy-tokenizer-tiny-segmenter-0.3.0
- tantivy-tokenizer-tiny-segmenter 0.3.0
- Docs.rs crate page
- MIT
- Links
- Repository
- Crates.io
- Source
- Owners
- osyoyu
- Dependencies
- tantivy ^0.10.2 normal
  tinysegmenter ^0.1.1 normal
- Versions
Platform
Feature flags

Rust

☰

Module tokenizer

Structs

tantivy_tokenizer_tiny_segmenter

[−][src]Module tantivy_tokenizer_tiny_segmenter::tokenizer

Structs

TinySegmenterTokenizer

Tokenizer for Japanese text, based on TinySegmenter.