Docs.rs

rust_tokenizers-6.2.1
- rust_tokenizers 6.2.1
- Docs.rs crate page
- Apache-2.0
- Links
- Repository
- crates.io
- Source
- Owners
- guillaume-be
- Dependencies
- csv ^1.1.3 normal
  
  hashbrown ^0.9.1 normal
  
  itertools ^0.9.0 normal
  
  lazy_static ^1.4.0 normal
  
  protobuf =2.18.0 normal
  
  rayon ^1.4.1 normal
  
  regex ^1.4.1 normal
  
  serde ^1.0.117 normal
  
  serde_json ^1.0.59 normal
  
  thiserror ^1.0.21 normal
  
  unicode-normalization ^0.1.13 normal
  
  unicode-normalization-alignments ^0.1.12 normal
  
  anyhow ^1.0.33 dev
  
  dirs ^3.0.1 dev
  
  reqwest ^0.10.8 dev
  
  tempfile ^3.1.0 dev
  
  protobuf-codegen-pure ^2.18.0 build
- Versions
- 90.35% of the crate is documented
Go to latest version
Platform
Feature flags

docs.rs

Rust

Redirecting to ../../../rust_tokenizers/tokenizer/struct.ProphetNetTokenizer.html...