Skip to main content

Decoder

Struct Decoder 

Source
pub struct Decoder { /* private fields */ }
Expand description

An incremental decoder for one Encoding.

A Decoder is created via Encoding::new_decoder and is fed the byte stream in one or more calls to decode_to_string (or its lossy counterpart, decode_to_string_lossy). Chunks may be split at any byte boundary — including in the middle of a multi-byte sequence or a UTF-16 surrogate pair — and the decoder carries the necessary state between calls to produce output identical to decoding the concatenation of all chunks in one call.

The final chunk (which may be empty) must be passed with last = true so the decoder can detect a sequence left truncated at the true end of the stream.

§Examples

Feeding a UTF-8 string one byte at a time still reassembles multi-byte characters correctly:

use oxixml_encoding::Encoding;

let text = "a→b🎉c"; // mixes ASCII, a 3-byte BMP arrow, and a 4-byte astral emoji.
let mut decoder = Encoding::Utf8.new_decoder();
let mut out = String::new();
let bytes = text.as_bytes();
for i in 0..bytes.len() {
    let last = i + 1 == bytes.len();
    decoder.decode_to_string(&bytes[i..=i], &mut out, last).expect("valid utf-8");
}
assert_eq!(out, text);

Implementations§

Source§

impl Decoder

Source

pub const fn encoding(&self) -> Encoding

The encoding this decoder was created for.

Source

pub const fn has_pending_state(&self) -> bool

Whether this decoder is currently holding onto part of an incomplete multi-byte sequence (or an unpaired UTF-16 high surrogate) carried over from a previous decode_to_string call.

Useful for a caller that wants to detect a stream that ends mid-sequence without making a final last = true call (for example, when a network connection drops).

Source

pub fn decode_to_string( &mut self, input: &[u8], dst: &mut String, last: bool, ) -> Result<usize, DecodeError>

Decode as much of input as possible, appending the result to dst.

input need not end on a character boundary: an incomplete trailing sequence is saved internally and completed by the start of the next call. Pass last = true for the final chunk of the stream (an empty final chunk is fine) so that a sequence left dangling at true end-of-input is reported as DecodeErrorKind::TruncatedSequence rather than silently held onto forever.

Returns the number of bytes of input incorporated into the decode (either emitted as characters or buffered as a pending partial sequence). On success this is always input.len(). On error, see DecodeError for how offset/length are interpreted; any valid characters that appeared before the error have already been appended to dst, and the decoder’s carried state is reset to empty so the Decoder remains usable for a fresh call (e.g. after the caller skips the offending bytes).

§Errors

Returns DecodeError if input (combined with any carried-over state) contains a byte sequence that is not valid in this decoder’s encoding, or ends in a sequence that last = true asserts will never be completed.

Source

pub fn decode_to_string_lossy( &mut self, input: &[u8], dst: &mut String, last: bool, ) -> usize

Like decode_to_string, but never fails: any malformed or disallowed byte sequence is replaced by one U+FFFD REPLACEMENT CHARACTER per the Unicode “maximal subpart” recovery algorithm, and decoding continues from the byte immediately after the replaced sequence.

Always consumes (and returns) the full length of input.

Trait Implementations§

Source§

impl Clone for Decoder

Source§

fn clone(&self) -> Decoder

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for Decoder

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.