Crate rten_generate

Utilities to simplify running auto-regressive RTen models such as transformer decoders.

For working examples, see the examples in the rten-examples crate which import rten_generate.

Re-exports

• pub use generator::Generator;
• pub use generator::GeneratorConfig;
• pub use generator::GeneratorError;
• pub use generator::GeneratorUtils;
• pub use generator::ModelInputsConfig;

Modules

• filter
  Filters for processing model outputs prior to sampling.
• generator
  Tools to run the generation loop for an auto-regressive model.
• metrics
  Record timing metrics during generation.
• model
  Abstraction over rten::Model for querying and executing ML models.
• sampler
  Samplers which select a token from model outputs.
• text_decoder
  Iterator adapters to decode token IDs into text using rten-text.