Docs.rs
tokstream-core-0.1.2
tokstream-core 0.1.2
Permalink
Docs.rs crate page
MIT
Links
Repository
crates.io
Source
Owners
M4n5ter
Dependencies
anyhow ^1.0.100
normal
tokenizers ^0.22.2
normal
tokenizers ^0.22.2
normal
Versions
0%
of the crate is documented
Platform
x86_64-unknown-linux-gnu
Feature flags
docs.rs
About docs.rs
Badges
Builds
Metadata
Shorthand URLs
Download
Rustdoc JSON
Build queue
Privacy policy
Rust
Rust website
The Book
Standard Library API Reference
Rust by Example
The Cargo Guide
Clippy Documentation
Skip to main content
Crate tokstream_core
tokstream_
core
0.1.2
All Items
Crate Items
Structs
Enums
Functions
Crate
tokstream_
core
Copy item path
Source
Structs
§
Candidate
Filter
Engine
Options
Filter
Config
Simple
Rng
Text
Stream
Token
Piece
Tokenizer
Engine
Enums
§
Random
Mode
Text
Repeat
Functions
§
tokenizer_
from_
json_
bytes