rust_tokenizers-8.1.1
- rust_tokenizers 8.1.1
- Permalink
- Docs.rs crate page
- Apache-2.0
- Links
- Repository
- Crates.io
- Source
- Owners
- guillaume-be
- Dependencies
- csv ^1 normal
  hashbrown ^0.14 normal
  itertools ^0.11 normal
  lazy_static ^1 normal
  protobuf ^2 normal
  rayon ^1 normal
  regex ^1 normal
  serde ^1 normal
  serde_json ^1 normal
  thiserror ^1 normal
  unicode-normalization ^0.1 normal
  unicode-normalization-alignments ^0.1.12 normal
  anyhow ^1 dev
  cached-path ^0.6 dev
  dirs ^5 dev
  tempfile ^3 dev
  protobuf-codegen-pure ^2 build
- Versions
- 41.67% of the crate is documented
Platform
Feature flags

Rust

rust_tokenizers8.1.1

OffsetSize

In crate rust_tokenizers

Type Alias rust_tokenizers::OffsetSize

pub type OffsetSize = u32;

Expand description

Crate-wide primitive used to store offset positions