stream-chunkrec 0.1.0

Recombine LLM streaming token deltas into stable text. Buffers partial words, handles UTF-8 fragments across chunks. Zero deps.
Documentation
  • Coverage
  • 100%
    6 out of 6 items documented1 out of 6 items with examples
  • Size
  • Source code size: 19.11 kB This is the summed size of all the files inside the crates.io package for this release.
  • Documentation size: 305.09 kB This is the summed size of all files generated by rustdoc for all configured targets
  • Ø build duration
  • this release: 33s Average build duration of successful builds.
  • all releases: 33s Average build duration of successful builds in releases after 2024-10-23.
  • Links
  • Homepage
  • MukundaKatta/stream-chunkrec
    0 0 0
  • crates.io
  • Dependencies
  • Versions
  • Owners
  • MukundaKatta

stream-chunkrec

crates.io

UTF-8-safe streaming recombiner. Buffers the trailing partial codepoint between pushes so multi-byte chars never turn into replacement chars.

use stream_chunkrec::Recombiner;
let mut r = Recombiner::new();
assert_eq!(r.push(&[0x63, 0x61, 0x66, 0xC3]), "caf");  // "café" split
assert_eq!(r.push(&[0xA9]), "é");

Zero deps. MIT or Apache-2.0.