rust_tokenizers-6.2.3
- rust_tokenizers 6.2.3
- Docs.rs crate page
- Apache-2.0
- Links
- Repository
- Crates.io
- Source
- Owners
- guillaume-be
- Dependencies
- csv ^1.1.6 normal
  hashbrown ^0.9.1 normal
  itertools ^0.10.0 normal
  lazy_static ^1.4.0 normal
  protobuf =2.22.1 normal
  rayon ^1.5.0 normal
  regex ^1.4.5 normal
  serde ^1.0.125 normal
  serde_json ^1.0.64 normal
  thiserror ^1.0.24 normal
  unicode-normalization ^0.1.17 normal
  unicode-normalization-alignments ^0.1.12 normal
  anyhow ^1.0.38 dev
  dirs ^3.0.1 dev
  reqwest ^0.11.2 dev
  tempfile ^3.2.0 dev
  protobuf-codegen-pure ^2.22.1 build
- Versions
- 90.95% of the crate is documented
Go to latest version
Platform
Feature flags

Rust

☰

Module error

Enums

Module rust_tokenizers::error[−][src]

Expand description

Tokenizer error variants

Enums