Docs.rs
aleph-alpha-tokenizer-0.3.1
Platform
i686-pc-windows-msvc
i686-unknown-linux-gnu
x86_64-apple-darwin
x86_64-pc-windows-msvc
x86_64-unknown-linux-gnu
Feature flags
docs.rs
About docs.rs
Privacy policy
Rust
Rust website
The Book
Standard Library API Reference
Rust by Example
The Cargo Guide
Clippy Documentation
aleph-alpha-tokenizer 0.3.1
A fast implementation of a wordpiece-inspired tokenizer
Crate
Source
Builds
Feature flags
Documentation
src
.cargo_vcs_info.json
Cargo.lock
Cargo.toml
Cargo.toml.orig
LICENSE-APACHE
LICENSE-MIT
README.md