Docs.rs
bitcannon-0.1.0-rc.3
Platform
x86_64-unknown-linux-gnu
Feature flags
docs.rs
About docs.rs
Badges
Builds
Metadata
Shorthand URLs
Download
Rustdoc JSON
Build queue
Privacy policy
Rust
Rust website
The Book
Standard Library API Reference
Rust by Example
The Cargo Guide
Clippy Documentation
bitcannon 0.1.0-rc.3
SIMD Unicode atom classification + bitstream pre-tokenization: one classify pass into byte-exact token spans (cl100k, o200k, GPT-2/ByteLevel, DeepSeek, Whitespace, Bert, ...).
Crate
Source
Builds
Feature flags
Documentation
Coverage
65.48%
55
out of
84
items documented
0
out of
26
items with examples
Size
Source code size: 307.7 kB
This is the summed size of all the files inside the crates.io package for this release.
Documentation size: 1.8 MB
This is the summed size of all files generated by rustdoc for all configured targets
Ø build duration
this release: 6s
Average build duration of
successful
builds.
all releases: 5s
Average build duration of
successful
builds in releases after 2024-10-23.
Links
Homepage
huggingface/tokenizers
11193
1234
113
crates.io
Dependencies
onig ^6.5.1
dev
Versions
0.1.0-rc.3
(2026-10-19)
0.1.0-rc.2
(2026-09-21)
0.1.0-rc.1
(2026-09-21)
0.1.0-dev.0
(2026-09-21)
Owners
