hermes-tokenizer-1.8.97
- hermes-tokenizer 1.8.97
- Docs.rs crate page
- MIT
- 26 July 2026
- Links
- Repository
- crates.io
- Source
- Owners
- ppodolsky
- Dependencies
- aho-corasick ^1.1 normal
  anyhow ^1.0 normal
  icu_normalizer ^2.2 normal
  icu_properties ^2.2 normal
  memchr ^2.8 normal
  parking_lot ^0.12 normal
  rayon ^1.11 normal
  rustc-hash ^2.1 normal
  serde ^1.0 normal
  serde_json ^1.0 normal
  ahash ^0.8 dev
  tokenizers ^0.23 dev
  libc ^0.2 normal
- Versions
- 100% of the crate is documented
Go to latest version
Platform
- x86_64-unknown-linux-gnu
Feature flags

docs.rs

Rust

Skip to main content

All

hermes_tokenizer1.8.97

Crate Items

Structs
Constants

List of all items

Structs

Tokenizer

Constants

UPSTREAM_GIGATOKEN_REVISION