rustbpe-0.1.0
- rustbpe 0.1.0
- Permalink
- Docs.rs crate page
- MIT
- Links
- crates.io
- Source
- Owners
- karpathy
- Dependencies
- ahash ^0.8.12 normal
  compact_str ^0.9.0 normal
  dary_heap ^0.3 normal
  fancy-regex ^0.16.1 normal
  indexmap ^2.2 normal
  log ^0.4.28 normal
  pyo3 ^0.23 normal
  pyo3-log ^0.12.4 normal
  rayon ^1.11.0 normal
  criterion ^0.5 dev
- Versions
- 91.67% of the crate is documented
Platform
- x86_64-unknown-linux-gnu
Feature flags

docs.rs

Rust

Skip to main content

Crate rustbpe

rustbpe0.1.0

All Items

Crate Items

Structs

Crate rustbpe

Structs§

Tokenizer: A Byte Pair Encoding tokenizer that matches the GPT-4 style implementation