unitoken-0.1.5
- unitoken 0.1.5
- Permalink
- Docs.rs crate page
- MIT
- 12 July 2026
- Links
- Repository
- crates.io
- Source
- Owners
- clouds56
- Dependencies
- ahash ^0.8.12 normal
  chrono ^0.4.42 normal optional
  clap ^4.5.53 normal optional
  fancy-regex ^0.16.2 normal
  hashbrown ^0.16.1 normal
  hotpath ^0.8.0 normal
  indicatif ^0.18.3 normal optional
  lazy_static ^1.5.0 normal
  memchr ^2.7.6 normal
  metrics ^0.24.3 normal
  moka ^0.12.11 normal
  ndarray ^0.17.1 normal optional
  npyz ^0.8.4 normal
  numpy ^0.27.1 normal optional
  ordermap ^1.0.0 normal
  pyo3 ^0.27 normal optional
  rayon ^1.11.0 normal
  rgb ^0.8.52 normal optional
  serde ^1.0.228 normal
  serde_json ^1.0.145 normal
  textplots ^0.8.7 normal optional
  thiserror ^2.0.17 normal
  tracing ^0.1.43 normal
  tracing-subscriber ^0.3.22 normal optional
  criterion ^0.5.1 dev
  pyo3-introspection ^0.27 dev
- Versions
- 36.11% of the crate is documented
Platform
- x86_64-unknown-linux-gnu
Feature flags

docs.rs

Rust

Skip to main content

Crate unitoken

unitoken0.1.5

All Items

Crate Items

Modules
Enums
Type Aliases

Crate unitoken

Modules§

bpe
counter
pretokenizer
spec
traits

Enums§

MyError

Type Aliases§

MyResult