pretokie-0.0.4
Platform
Feature flags

docs.rs

Rust

pretokie 0.0.4

Fast, zero-allocation pretokenizers for BPE tokenizers

Crate
Source
Builds
Feature flags

..
config.rs
iter.rs
mod.rs

1
2

pub mod config;
pub mod iter;