Skip to main content

Crate rlx_text

Crate rlx_text 

Source
Expand description

RLX text: tokenizer + chat template + sampling helpers.

Promoted from rlx-models/crates/rlx-cli so that downstream LM apps (server, web playground, training tools) have one published crate to depend on without taking the CLI helper layer.

Re-exports§

pub use chat::ChatMessage;
pub use chat::ChatTemplate;
pub use chat::ChatTemplateSource;
pub use chat::auto_chat_template;
pub use sampling::argmax;
pub use sampling::sample_next;
pub use tokenizer::TokenizerHandle;
pub use tokenizer::decode_ids;
pub use tokenizer::load_tokenizer;

Modules§

chat
Chat-template engine for RLX runners.
sampling
Sampling helpers for LM runners.
tokenizer
Thin wrapper around the tokenizers crate.

Structs§

SampleOpts
Sampling parameters. Greedy when temperature == 0.