Crate rlx_text 

RLX text: tokenizer + chat template + sampling helpers.

Promoted from rlx-models/crates/rlx-cli so that downstream LM apps (server, web playground, training tools) have one published crate to depend on without taking the CLI helper layer.

Re-exports

pub use chat::ChatMessage;

pub use chat::ChatTemplate;

pub use chat::ChatTemplateSource;

pub use chat::auto_chat_template;

pub use detokenize::StreamingDetokenizer;

pub use detokenize::incremental_emit;

pub use sampling::argmax;

pub use sampling::sample_next;

pub use tokenizer::TokenizerHandle;

pub use tokenizer::decode_ids;

pub use tokenizer::load_tokenizer;

pub use tool_parse::ToolCall;

pub use tool_parse::ToolFormat;

Modules

chat

Chat-template engine for RLX runners.

detokenize

Incremental detokenization for streaming generation.

sampling

Sampling helpers for LM runners.

tokenizer

Thin wrapper around the tokenizers crate.

tool_parse

Parsing model-emitted tool / function calls into structured form.

Structs

SampleOpts

Sampling parameters. Greedy when temperature == 0 and no advanced sampler is enabled. All “advanced” knobs default to off / no-op so legacy callers see classic top-k/top-p/temperature behaviour.