Docs.rs

pretokie-0.0.1
Platform
Feature flags

docs.rs

Rust

pretokie 0.0.1

Fast, zero-allocation pretokenizers for BPE tokenizers

Crate
Source
Builds
Feature flags

Documentation

..
impls
lib.rs
util.rs