rlx-text 0.2.4

RLX text — tokenizer wrappers, chat templates, sampling. Public surface for downstream LM apps.
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# rlx-text

Public surface for downstream LM apps:

- **Chat templates** — Jinja-rendered chat templates loaded inline or from
  GGUF metadata (`tokenizer.chat_template`, `tokenizer.ggml.bos_token_id`, …).
- **Tokenizer wrapper** — thin façade over the `tokenizers` crate so model
  crates don't all rebuild the same encode/decode glue.
- **Sampling** — re-exports `rlx_runtime::SampleOpts` plus helpers
  (`argmax`, top-k, top-p, repetition penalty).

Replaces the chat/sampling/tokenizer pieces that used to live in
`rlx-models/rlx-cli`, breaking the "runners depend on the CLI helper crate"
coupling.

## License

GPL-3.0-only.