pub struct Decoder { /* private fields */ }Expand description
An incremental decoder for one Encoding.
A Decoder is created via Encoding::new_decoder and is fed the byte stream in one or
more calls to decode_to_string (or its lossy counterpart,
decode_to_string_lossy). Chunks may be split at any
byte boundary — including in the middle of a multi-byte sequence or a UTF-16 surrogate pair
— and the decoder carries the necessary state between calls to produce output identical to
decoding the concatenation of all chunks in one call.
The final chunk (which may be empty) must be passed with last = true so the decoder can
detect a sequence left truncated at the true end of the stream.
§Examples
Feeding a UTF-8 string one byte at a time still reassembles multi-byte characters correctly:
use oxixml_encoding::Encoding;
let text = "a→b🎉c"; // mixes ASCII, a 3-byte BMP arrow, and a 4-byte astral emoji.
let mut decoder = Encoding::Utf8.new_decoder();
let mut out = String::new();
let bytes = text.as_bytes();
for i in 0..bytes.len() {
let last = i + 1 == bytes.len();
decoder.decode_to_string(&bytes[i..=i], &mut out, last).expect("valid utf-8");
}
assert_eq!(out, text);Implementations§
Source§impl Decoder
impl Decoder
Sourcepub const fn has_pending_state(&self) -> bool
pub const fn has_pending_state(&self) -> bool
Whether this decoder is currently holding onto part of an incomplete multi-byte sequence
(or an unpaired UTF-16 high surrogate) carried over from a previous
decode_to_string call.
Useful for a caller that wants to detect a stream that ends mid-sequence without
making a final last = true call (for example, when a network connection drops).
Sourcepub fn decode_to_string(
&mut self,
input: &[u8],
dst: &mut String,
last: bool,
) -> Result<usize, DecodeError>
pub fn decode_to_string( &mut self, input: &[u8], dst: &mut String, last: bool, ) -> Result<usize, DecodeError>
Decode as much of input as possible, appending the result to dst.
input need not end on a character boundary: an incomplete trailing sequence is saved
internally and completed by the start of the next call. Pass last = true for the final
chunk of the stream (an empty final chunk is fine) so that a sequence left dangling at
true end-of-input is reported as DecodeErrorKind::TruncatedSequence rather than
silently held onto forever.
Returns the number of bytes of input incorporated into the decode (either emitted as
characters or buffered as a pending partial sequence). On success this is always
input.len(). On error, see DecodeError for how offset/length are interpreted;
any valid characters that appeared before the error have already been appended to dst,
and the decoder’s carried state is reset to empty so the Decoder remains usable for a
fresh call (e.g. after the caller skips the offending bytes).
§Errors
Returns DecodeError if input (combined with any carried-over state) contains a
byte sequence that is not valid in this decoder’s encoding, or ends in a sequence that
last = true asserts will never be completed.
Sourcepub fn decode_to_string_lossy(
&mut self,
input: &[u8],
dst: &mut String,
last: bool,
) -> usize
pub fn decode_to_string_lossy( &mut self, input: &[u8], dst: &mut String, last: bool, ) -> usize
Like decode_to_string, but never fails: any malformed or
disallowed byte sequence is replaced by one U+FFFD REPLACEMENT CHARACTER per the
Unicode “maximal subpart” recovery algorithm, and decoding continues from the
byte immediately after the replaced sequence.
Always consumes (and returns) the full length of input.