Expand description
RLX text: tokenizer + chat template + sampling helpers.
Promoted from rlx-models/crates/rlx-cli so that downstream LM apps
(server, web playground, training tools) have one published crate to
depend on without taking the CLI helper layer.
Re-exports§
pub use chat::ChatMessage;pub use chat::ChatTemplate;pub use chat::ChatTemplateSource;pub use chat::auto_chat_template;pub use sampling::argmax;pub use sampling::sample_next;pub use tokenizer::TokenizerHandle;pub use tokenizer::decode_ids;pub use tokenizer::load_tokenizer;
Modules§
- chat
- Chat-template engine for RLX runners.
- sampling
- Sampling helpers for LM runners.
- tokenizer
- Thin wrapper around the
tokenizerscrate.
Structs§
- Sample
Opts - Sampling parameters. Greedy when
temperature == 0.