rten-generate 0.19.0

Utilities to simplify running auto-regressive models with RTen

Coverage
92.31%
72 out of 78 items documented0 out of 51 items with examples
Size
Source code size: 93.25 kB This is the summed size of all the files inside the crates.io package for this release.
Documentation size: 12.84 MB This is the summed size of all files generated by rustdoc for all configured targets
Ø build duration
this release: 56s Average build duration of successful builds.
all releases: 40s Average build duration of successful builds in releases after 2024-10-23.
Links
Homepage
robertknight/rten
339 27 44
crates.io
Dependencies
Versions
Owners

rten-generate is a layer on top of RTen which handles the generation loop for auto-regressive transformer models (aka. "transformer decoders" or "generative AI"). This includes managing the KV cache, sampling and post-processing logits etc.