Docs.rs
rust_tokenizers-8.0.0
Platform
i686-pc-windows-msvc
i686-unknown-linux-gnu
x86_64-apple-darwin
x86_64-pc-windows-msvc
x86_64-unknown-linux-gnu
Feature flags
docs.rs
About docs.rs
Privacy policy
Rust
Rust website
The Book
Standard Library API Reference
Rust by Example
The Cargo Guide
Clippy Documentation
rust_tokenizers 8.0.0
High performance tokenizers for Rust
Crate
Source
Builds
Feature flags
Documentation
..
tokenizer
vocab
adapters.rs
error.rs
lib.rs