pub struct Utf8StreamDecoder { /* private fields */ }Expand description
Incrementally decodes a byte stream into UTF-8 text.
Chunks can split a multibyte code point (CJK, emoji, accented characters,
smart quotes) across boundaries. Decoding each chunk independently with
String::from_utf8_lossy corrupts such code points into U+FFFD
replacement characters. This decoder buffers any trailing incomplete
sequence until the rest of its bytes arrive, while still replacing genuinely
invalid bytes with U+FFFD (matching from_utf8_lossy semantics). Call
Utf8StreamDecoder::finish at end of stream so a truncated final sequence
becomes U+FFFD instead of vanishing.
Implementations§
Source§impl Utf8StreamDecoder
impl Utf8StreamDecoder
pub fn new() -> Self
Sourcepub fn push(&mut self, bytes: &[u8]) -> String
pub fn push(&mut self, bytes: &[u8]) -> String
Appends bytes and returns the decodable UTF-8 prefix. A trailing
incomplete multibyte sequence is retained for the next call.
Sourcepub fn push_cow<'a>(&mut self, bytes: &'a [u8]) -> Cow<'a, str>
pub fn push_cow<'a>(&mut self, bytes: &'a [u8]) -> Cow<'a, str>
Like Utf8StreamDecoder::push, but borrows bytes without copying when
nothing is pending and the chunk is valid UTF-8 (the common case for
process output), so hot read loops avoid a per-chunk allocation.
Sourcepub fn push_bytes(&mut self, bytes: &[u8], out: &mut Vec<u8>)
pub fn push_bytes(&mut self, bytes: &[u8], out: &mut Vec<u8>)
Appends bytes and writes the decodable UTF-8 prefix directly into
out. A trailing incomplete multibyte sequence is retained for the
next call.
This avoids the intermediate String allocation that push creates
when the caller only needs bytes (e.g. feeding an SSE byte buffer).