Trait constriction::stream::Decode

pub trait Decode<const PRECISION: usize>: Code {
    type FrontendError: Debug;
    type BackendError: Debug;

    // Required method
    fn decode_symbol<D>(
        &mut self,
        model: D
    ) -> Result<D::Symbol, CoderError<Self::FrontendError, Self::BackendError>>
       where D: DecoderModel<PRECISION>,
             D::Probability: Into<Self::Word>,
             Self::Word: AsPrimitive<D::Probability>;

    // Provided methods
    fn decode_symbols<'s, I, M>(
        &'s mut self,
        models: I
    ) -> DecodeSymbols<'s, Self, I::IntoIter, PRECISION> 
       where I: IntoIterator<Item = M> + 's,
             M: DecoderModel<PRECISION>,
             M::Probability: Into<Self::Word>,
             Self::Word: AsPrimitive<M::Probability> { ... }
    fn try_decode_symbols<'s, I, M, E>(
        &'s mut self,
        models: I
    ) -> TryDecodeSymbols<'s, Self, I::IntoIter, PRECISION> 
       where I: IntoIterator<Item = Result<M, E>> + 's,
             M: DecoderModel<PRECISION>,
             M::Probability: Into<Self::Word>,
             Self::Word: AsPrimitive<M::Probability> { ... }
    fn decode_iid_symbols<M>(
        &mut self,
        amt: usize,
        model: M
    ) -> DecodeIidSymbols<'_, Self, M, PRECISION> 
       where M: DecoderModel<PRECISION> + Copy,
             M::Probability: Into<Self::Word>,
             Self::Word: AsPrimitive<M::Probability> { ... }
    fn maybe_exhausted(&self) -> bool { ... }
}

A trait for stream decoders (i.e., decompressors)

This trait defines methods for decoding a single symbol or a sequence of symbols.

Naming Convention

This trait is deliberately called Decode and not Decoder. See corresponding comment for the Code trait for the reasoning.

Required Associated Types

type FrontendError: Debug

The error type for logical decoding errors.

This may be Infallible for surjective coders, such as AnsCoder.

Frontend errors are errors that relate to the logical decoding process, such as when a user tries to decode a bitstring that is invalid under the used encoder. For decoding, this may include the case where there’s no compressed data left if the decoder does not allow decoding in such a situation.

type BackendError: Debug

The error type for reading in encoded data.

This does not include the case of running out of data, which is either represented by FrontendError, or which may even be allowed by the decoder.

If you stick with the default backend(s) and don’t do anything fancy, then the BackendError will most likely be Infallible, which means that the compiler can optimize away error checks. You can explicitly express that you expect an Infallible error type by calling .unwrap_infallible() on a Result<T, Infallible> or on a Result<T, CoderError<Infallible, Infallible>> (you’ll have to bring the trait UnwrapInfallible into scope).

If you use a custom backend, then the BackendError will typically be the ReadError type of the of the backend. For example, it could signal an I/O error (other than “end of file”) if you’re decoding directly from a file or socket rather than from an in-memory buffer.

Required Methods

fn decode_symbol<D>( &mut self, model: D ) -> Result<D::Symbol, CoderError<Self::FrontendError, Self::BackendError>>

where D: DecoderModel<PRECISION>, D::Probability: Into<Self::Word>, Self::Word: AsPrimitive<D::Probability>,

Decodes a single symbol using the given entropy model.

This is the most basic decoding method. If you want to decode more than a single symbol then you may want to call decode_symbols, try_decode_symbols, or decode_iid_symbols instead.

Note that:

• the model can be passed either by value or by reference since, if M implements DecoderModel<PRECISION>, then &M does so too; and
• the PRECISION will typically be inferred from the entropy model, and it may not be larger than Word::BITS; this is enforced by run-time assertions (that get optimized away unless they fail) but it will be enforced at compile time in future versions of constriction as soon as the type system allows this.

Errors

Returns Err(CoderError::Frontend(e)) if there was a logic error e during encoding (such as trying to decode an invalid compressed bitstring if your coder is not surjective, or running out of compressed data if your coder does not allow decoding in such a situation). Returns Err(CoderError::Backend(e)) if reading compressed data lead to an I/O error e (other than “end of file”). Otherwise, returns Ok(()).

Example

use constriction::{
    stream::{model::DefaultLeakyQuantizer, stack::DefaultAnsCoder, Decode},
    UnwrapInfallible,
};

// Get some mock compressed data.
let compressed = vec![0x1E34_22B0];
// (calling this "compressed" is a bit of a misnomer since the compressed representation
// of just two symbols is quite long due to the constant overhead.)

// Create an ANS Coder from the compressed data and an entropy model.
let mut ans_coder = DefaultAnsCoder::from_compressed(compressed).unwrap();
let quantizer = DefaultLeakyQuantizer::new(-100i32..=100);
let entropy_model = quantizer.quantize(probability::distribution::Gaussian::new(0.0, 10.0));

// Decode a single symbol, passing the entropy model by reference:
assert_eq!(ans_coder.decode_symbol(&entropy_model).unwrap_infallible(), -8);

// Decode another symbol using the same entropy model, this time passing it by value:
assert_eq!(ans_coder.decode_symbol(entropy_model).unwrap_infallible(), 12);

// Verify that we've consumed all data on the coder.
assert!(ans_coder.is_empty());

Provided Methods

fn decode_symbols<'s, I, M>( &'s mut self, models: I ) -> DecodeSymbols<'s, Self, I::IntoIter, PRECISION>

where I: IntoIterator<Item = M> + 's, M: DecoderModel<PRECISION>, M::Probability: Into<Self::Word>, Self::Word: AsPrimitive<M::Probability>,

Decodes a sequence of symbols, using an individual entropy model for each symbol.

This method is lazy: it doesn’t actually decode anything until you iterate over the returned iterator.

The return type implements Iterator<Item = Result<M::Symbol, ...>>, and it implements ExactSizeIterator if the provided iterator models implements ExactSizeIterator. The provided iterator models may yield entropy models either by value or by reference since, if M implements DecoderModel<PRECISION>, then &M does so too.

This method does not short-circuit. If an error e occurs then the returned iterator will yield Err(e) but you can, in principle, continue to iterate over it. In practice, however, continuing to iterate after an error will likely yield either further errors or garbled symbols. Therefore, you’ll usually want to short-circuit the returned iterator yourself, e.g., by applying the ? operator on each yielded item or by collecting the items from the returned iterator into a Result<Container<_>, _> as in the second example below.

Examples

Iterate over the decoded symbols in a for loop

use constriction::{
    stream::{model::DefaultLeakyQuantizer, stack::DefaultAnsCoder, Decode},
    UnwrapInfallible,
};

// Create a decoder from some mock compressed data and create some mock entropy models.
let compressed = vec![0x2C63_D22E, 0x0000_0377];
let mut decoder = DefaultAnsCoder::from_compressed(compressed).unwrap();
let quantizer = DefaultLeakyQuantizer::new(-100i32..=100);
let entropy_models = (0..5).map(
    |i| quantizer.quantize(probability::distribution::Gaussian::new((i * 10) as f64, 10.0))
);

// Decode the symbols and iterate over them in a `for` loop:
for symbol in decoder.decode_symbols(entropy_models) {
    dbg!(symbol.unwrap_infallible()); // Prints the symbols `-3`, `12`, `19`, `28`, and `41`.
}

collect the decoded symbols into a Result<Vec<M::Symbol>, _>

A useful trick when you have an iterator over Results is that you can collect it into Result<SomeContainer<_>, _>:

// Assume same setup as in previous example.
let symbols = decoder.decode_symbols(entropy_models).collect::<Result<Vec<_>, _>>();
assert_eq!(symbols.unwrap_infallible(), [-3, 12, 19, 28, 41]);

This results in only a single memory allocation of the exact correct size, and it automatically short-circuits upon error (which is moot in this example because, for this particular entropy coder, both FrontendError and BackendError are Infallible, i.e., decoding cannot fail).

See Also

• decode_symbol if you want to decode only a single symbol;
• try_decode_symbols if generating the entropy models may fail; and
• decode_iid_symbols if all symbols use the same entropy model.

fn try_decode_symbols<'s, I, M, E>( &'s mut self, models: I ) -> TryDecodeSymbols<'s, Self, I::IntoIter, PRECISION>

where I: IntoIterator<Item = Result<M, E>> + 's, M: DecoderModel<PRECISION>, M::Probability: Into<Self::Word>, Self::Word: AsPrimitive<M::Probability>,

Decodes a sequence of symbols from a fallible iterator over entropy models.

This method is equivalent to decode_symbols, except that it takes a fallible iterator (i.e., an iterator that yields Results).

Just like decode_symbols,

• this method is lazy, i.e., it doesn’t decode until you iterate over the returned iterator;
• the returned iterator implements ExactSizeIterator if models implements ExactSizeIterator; and
• you can, in principle, continue to decode after an error, but you’ll likely rather want to short-circuit the returned iterator. This applies to both decoding errors and to errors from the supplied iterator models.

The returned iterator yields

• Ok(symbol) on success;
• Err(TryCodingError::InvalidEntropyModel(e)) if the iterator models yielded Err(e); and
• Err(TryCodingError::CodingError(e)) if decoding resulted in Err(e).

This method may be useful for parameterized entropy models whose parameters have to satisfy certain constraints (e.g., they have to be positive), but they come from an untrusted source they may violate the constraints.

Example

use constriction::{
    stream::{model::DefaultLeakyQuantizer, stack::DefaultAnsCoder, Decode, TryCodingError},
    CoderError,
};
use core::convert::Infallible;

/// Helper function to decode symbols with Gaussian entropy models with untrusted parameters.
fn try_decode_gaussian(compressed: Vec<u32>, means: &[f64], stds: &[f64])
    -> Result<Vec<i32>, TryCodingError<CoderError<Infallible, Infallible>, &'static str>>
{
    let mut decoder = DefaultAnsCoder::from_compressed(compressed).unwrap();
    let quantizer = DefaultLeakyQuantizer::new(-100i32..=100);
    let entropy_models = means.iter().zip(stds).map(|(&mean, &std)| {
        if std > 0.0 && mean.is_finite() {
            Ok(quantizer.quantize(probability::distribution::Gaussian::new(mean, std)))
        } else {
            Err("argument error")
        }
    });
    decoder.try_decode_symbols(entropy_models).collect::<Result<Vec<_>, _>>()
}

let compressed = vec![0x2C63_D22E, 0x0000_0377];
let means = [0.0, 10.0, 20.0, 30.0, 40.0];
let valid_stds = [10.0, 10.0, 10.0, 10.0, 10.0];
let invalid_stds = [10.0, 10.0, -1.0, 10.0, 10.0]; // Invalid: negative standard deviation

assert_eq!(
    try_decode_gaussian(compressed.clone(), &means, &valid_stds),
    Ok(vec![-3, 12, 19, 28, 41])
);
assert_eq!(
    try_decode_gaussian(compressed, &means, &invalid_stds),
    Err(TryCodingError::InvalidEntropyModel("argument error"))
);

fn decode_iid_symbols<M>( &mut self, amt: usize, model: M ) -> DecodeIidSymbols<'_, Self, M, PRECISION>

where M: DecoderModel<PRECISION> + Copy, M::Probability: Into<Self::Word>, Self::Word: AsPrimitive<M::Probability>,

Decodes amt symbols using the same entropy model for all symbols.

The return type implements ExactSizeIterator<Item = Result<M::Symbol, ...>>.

Just like decode_symbols,

• this method is lazy, i.e., it doesn’t decode until you iterate over the returned iterator; and
• you can, in principle, continue to decode after an error, but you’ll likely rather want to short-circuit the returned iterator.

While this method takes model formally by value, you’ll typically want to pass the DecoderModel by reference (which is possible since any reference to a DecoderModel implements DecoderModel too). The bound on M: Copy prevents accidental misuse in this regard. We provide the ability to pass the DecoderModel by value as an opportunity for microoptimzations when dealing with models that can be cheaply copied (see, e.g., LookupDecoderModel::as_view).

If you want to decode each symbol with its individual entropy model, then consider calling decode_symbols instead. If you just want to decode a single symbol, then call decode_symbol instead.

fn maybe_exhausted(&self) -> bool

Checks if there might be no compressed data left for decoding.

If this method returns false then there must be additional data left to decode. If this method returns true then it is unknown whether or not the decoder is exhausted.

The default implementation always returns true, which is always correct albeit not particularly useful. Consider overwriting the default implementation and call ReadWords::maybe_exhausted on your backend if appropriate.

This method is useful for checking for consistency. If you have decoded all data that you expect to be on a decoder but this method still returns false then either your code has a bug or the compressed data was corrupted. If this method returns true then that’s no guarantee that everything is correct, but at least there’s no evidence of the contrary.

Calling this method can be awkward for entropy coders that implement Decode<PRECISION> for more than one value of PRECISION. The method Code::decoder_maybe_exhausted is provided as a more convenient forwarding method.

Object Safety

This trait is not object safe.

Implementors

impl<Word, State, Backend, const PRECISION: usize> Decode<PRECISION> for RangeDecoder<Word, State, Backend>

where Word: BitArray + Into<State>, State: BitArray + AsPrimitive<Word>, Backend: ReadWords<Word, Queue>,

type FrontendError = DecoderFrontendError

type BackendError = <Backend as ReadWords<Word, Queue>>::ReadError

impl<Word, State, Backend, const PRECISION: usize> Decode<PRECISION> for AnsCoder<Word, State, Backend>

where Word: BitArray + Into<State>, State: BitArray + AsPrimitive<Word>, Backend: ReadWords<Word, Stack>,

type FrontendError = Infallible

type BackendError = <Backend as ReadWords<Word, Stack>>::ReadError

impl<Word, State, CompressedBackend, RemaindersBackend, const PRECISION: usize> Decode<PRECISION> for ChainCoder<Word, State, CompressedBackend, RemaindersBackend, PRECISION>

where Word: BitArray + Into<State>, State: BitArray + AsPrimitive<Word>, CompressedBackend: ReadWords<Word, Stack>, RemaindersBackend: WriteWords<Word>,

type FrontendError = DecoderFrontendError

type BackendError = BackendError<<CompressedBackend as ReadWords<Word, Stack>>::ReadError, <RemaindersBackend as WriteWords<Word>>::WriteError>