bpe-0.2.1
- bpe 0.2.1
- Permalink
- Docs.rs crate page
- MIT
- 06 June 2026
- Links
- Repository
- crates.io
- Source
- Owners
- hendrikvanantwerpen
- gorzell
- aneubeck
- Dependencies
- aneubeck-daachorse ^1.1.1 normal
  base64 ^0.22 normal optional
  fnv ^1.0 normal
  itertools ^0.14 normal
  rand ^0.9 normal optional
  serde ^1 normal
  tiktoken-rs ^0.6 dev
- Versions
- 63.04% of the crate is documented
Platform
- x86_64-unknown-linux-gnu
Feature flags

docs.rs

Rust

Skip to main content

All

bpe0.2.1

Crate Items

Structs
Functions

List of all items

Structs

appendable_encoder::AppendableEncoder
byte_pair_encoding::BytePairEncoding
interval_encoding::IntervalEncoding
prependable_encoder::PrependableEncoder

Functions

byte_pair_encoding::create_test_bytes
byte_pair_encoding::create_test_string
byte_pair_encoding::create_test_string_with_predicate
byte_pair_encoding::find_hash_factor_for_dictionary
byte_pair_encoding::find_hash_factor_for_tiktoken
byte_pair_encoding::read_tiktoken
byte_pair_encoding::select_test_string