tocken-0.1.0
- tocken 0.1.0
- Permalink
- Docs.rs crate page
- Apache-2.0
- Links
- Repository
- crates.io
- Source
- Owners
- kemingy
- Dependencies
- argh ^0.1.13 normal
  
  env_logger ^0.11.6 normal
  
  log ^0.4.22 normal
  
  regex ^1.11.1 normal
  
  serde ^1.0.216 normal
  
  serde_json ^1.0.133 normal
  
  tantivy-stemmers ^0.4.0 normal
  
  unicode-normalization ^0.1.24 normal
  
  unicode-segmentation ^1.12.0 normal
- Versions
- 100% of the crate is documented
Platform
Feature flags

docs.rs

Rust

Crate tocken

tocken0.1.0

All Items

Crate Items

Modules

Crate tocken

Expand description

Tokenizer for text keyword match.

Modules§

stopwords: Stopwords
tokenizer: Naive tokenizer for Unicode.