bpe-openai-0.2.1
- bpe-openai 0.2.1
- Docs.rs crate page
- MIT
- Links
- Repository
- crates.io
- Source
- Owners
- hendrikvanantwerpen
- gorzell
- aneubeck
- Dependencies
- bpe ^0.2.0 normal
  
  either ^1.13 normal
  
  regex-automata ^0.4 normal
  
  rmp-serde ^1 normal
  
  bpe ^0.2.0 dev
  
  tiktoken-rs ^0.6 dev
  
  base64 ^0.22.1 build
  
  bpe ^0.2.0 build
  
  flate2 ^1.0 build
  
  rmp-serde ^1 build
  
  serde ^1 build
- Versions
- 73.33% of the crate is documented
Go to latest version
Platform
Feature flags

docs.rs

Rust

bpe_openai0.2.1

Crate Items

Structs
Functions

List of all items

Structs

Pretokenizer
Tokenizer
appendable_encoder::AppendableEncoder
byte_pair_encoding::BytePairEncoding
interval_encoding::IntervalEncoding
prependable_encoder::PrependableEncoder

Functions

cl100k_base
o200k_base