Docs.rs
text-tokenizer-0.5.10
text-tokenizer 0.5.10
Permalink
Docs.rs crate page
MIT
Links
Homepage
Repository
crates.io
Source
Owners
merl-twin
Dependencies
lazy_static ^1.5
normal
text-parsing ^0.6
normal
unicode-properties ^0.1
normal
unicode-segmentation ^1.2
normal
Versions
0%
of the crate is documented
Platform
i686-pc-windows-msvc
i686-unknown-linux-gnu
x86_64-apple-darwin
x86_64-pc-windows-msvc
x86_64-unknown-linux-gnu
Feature flags
Rust
About docs.rs
Privacy policy
Rust website
The Book
Standard Library API Reference
Rust by Example
The Cargo Guide
Clippy Documentation
text_
tokenizer
0.5.10
Crate Items
Structs
Enums
Traits
List of all items
Structs
EMOJIMAP
Text
TextStr
TextToken
TextTokens
TokenizerParams
Tokens
UnicodeSentenceBreaker
Enums
Bound
Error
Formatter
Number
Numerical
OriginalError
Separator
Special
Struct
Token
Token2
TokenizerOptions
Unicode
Word
Traits
IntoTokenizer
SentenceBreaker