Trait constriction::stream::Decode
source · pub trait Decode<const PRECISION: usize>: Code {
type FrontendError: Debug;
type BackendError: Debug;
// Required method
fn decode_symbol<D>(
&mut self,
model: D
) -> Result<D::Symbol, CoderError<Self::FrontendError, Self::BackendError>>
where D: DecoderModel<PRECISION>,
D::Probability: Into<Self::Word>,
Self::Word: AsPrimitive<D::Probability>;
// Provided methods
fn decode_symbols<'s, I, M>(
&'s mut self,
models: I
) -> DecodeSymbols<'s, Self, I::IntoIter, PRECISION> ⓘ
where I: IntoIterator<Item = M> + 's,
M: DecoderModel<PRECISION>,
M::Probability: Into<Self::Word>,
Self::Word: AsPrimitive<M::Probability> { ... }
fn try_decode_symbols<'s, I, M, E>(
&'s mut self,
models: I
) -> TryDecodeSymbols<'s, Self, I::IntoIter, PRECISION> ⓘ
where I: IntoIterator<Item = Result<M, E>> + 's,
M: DecoderModel<PRECISION>,
M::Probability: Into<Self::Word>,
Self::Word: AsPrimitive<M::Probability> { ... }
fn decode_iid_symbols<M>(
&mut self,
amt: usize,
model: M
) -> DecodeIidSymbols<'_, Self, M, PRECISION> ⓘ
where M: DecoderModel<PRECISION> + Copy,
M::Probability: Into<Self::Word>,
Self::Word: AsPrimitive<M::Probability> { ... }
fn maybe_exhausted(&self) -> bool { ... }
}
Expand description
A trait for stream decoders (i.e., decompressors)
This trait defines methods for decoding a single symbol or a sequence of symbols.
Naming Convention
This trait is deliberately called Decode
and not Decoder
. See corresponding comment
for the Code
trait for the reasoning.
Required Associated Types§
sourcetype FrontendError: Debug
type FrontendError: Debug
The error type for logical decoding errors.
This may be Infallible
for surjective coders, such as
AnsCoder
.
Frontend errors are errors that relate to the logical decoding process, such as when a user tries to decode a bitstring that is invalid under the used encoder. For decoding, this may include the case where there’s no compressed data left if the decoder does not allow decoding in such a situation.
sourcetype BackendError: Debug
type BackendError: Debug
The error type for reading in encoded data.
This does not include the case of running out of data, which is either represented
by FrontendError
, or which may even be allowed by the
decoder.
If you stick with the default backend(s) and don’t do anything fancy, then the
BackendError
will most likely be Infallible
, which means that the compiler can
optimize away error checks. You can explicitly express that you expect an
Infallible
error type by calling .unwrap_infallible()
on a Result<T, Infallible>
or on a Result<T, CoderError<Infallible, Infallible>>
(you’ll have to
bring the trait UnwrapInfallible
into scope).
If you use a custom backend, then the BackendError
will typically be the
ReadError
type of the of the backend. For example, it could signal an I/O error
(other than “end of file”) if you’re decoding directly from a file or socket rather
than from an in-memory buffer.
Required Methods§
sourcefn decode_symbol<D>(
&mut self,
model: D
) -> Result<D::Symbol, CoderError<Self::FrontendError, Self::BackendError>>where
D: DecoderModel<PRECISION>,
D::Probability: Into<Self::Word>,
Self::Word: AsPrimitive<D::Probability>,
fn decode_symbol<D>(
&mut self,
model: D
) -> Result<D::Symbol, CoderError<Self::FrontendError, Self::BackendError>>where
D: DecoderModel<PRECISION>,
D::Probability: Into<Self::Word>,
Self::Word: AsPrimitive<D::Probability>,
Decodes a single symbol using the given entropy model.
This is the most basic decoding method. If you want to decode more than a single
symbol then you may want to call decode_symbols
, try_decode_symbols
, or
decode_iid_symbols
instead.
Note that:
- the
model
can be passed either by value or by reference since, ifM
implementsDecoderModel<PRECISION>
, then&M
does so too; and - the
PRECISION
will typically be inferred from the entropy model, and it may not be larger thanWord::BITS
; this is enforced by run-time assertions (that get optimized away unless they fail) but it will be enforced at compile time in future versions ofconstriction
as soon as the type system allows this.
Errors
Returns Err(CoderError::Frontend(e))
if there was a logic error e
during
encoding (such as trying to decode an invalid compressed bitstring if your coder is
not surjective, or running out of compressed data if your coder does not allow
decoding in such a situation). Returns Err(CoderError::Backend(e))
if reading
compressed data lead to an I/O error e
(other than “end of file”). Otherwise,
returns Ok(())
.
Example
use constriction::{
stream::{model::DefaultLeakyQuantizer, stack::DefaultAnsCoder, Decode},
UnwrapInfallible,
};
// Get some mock compressed data.
let compressed = vec![0x1E34_22B0];
// (calling this "compressed" is a bit of a misnomer since the compressed representation
// of just two symbols is quite long due to the constant overhead.)
// Create an ANS Coder from the compressed data and an entropy model.
let mut ans_coder = DefaultAnsCoder::from_compressed(compressed).unwrap();
let quantizer = DefaultLeakyQuantizer::new(-100i32..=100);
let entropy_model = quantizer.quantize(probability::distribution::Gaussian::new(0.0, 10.0));
// Decode a single symbol, passing the entropy model by reference:
assert_eq!(ans_coder.decode_symbol(&entropy_model).unwrap_infallible(), -8);
// Decode another symbol using the same entropy model, this time passing it by value:
assert_eq!(ans_coder.decode_symbol(entropy_model).unwrap_infallible(), 12);
// Verify that we've consumed all data on the coder.
assert!(ans_coder.is_empty());
Provided Methods§
sourcefn decode_symbols<'s, I, M>(
&'s mut self,
models: I
) -> DecodeSymbols<'s, Self, I::IntoIter, PRECISION> ⓘwhere
I: IntoIterator<Item = M> + 's,
M: DecoderModel<PRECISION>,
M::Probability: Into<Self::Word>,
Self::Word: AsPrimitive<M::Probability>,
fn decode_symbols<'s, I, M>(
&'s mut self,
models: I
) -> DecodeSymbols<'s, Self, I::IntoIter, PRECISION> ⓘwhere
I: IntoIterator<Item = M> + 's,
M: DecoderModel<PRECISION>,
M::Probability: Into<Self::Word>,
Self::Word: AsPrimitive<M::Probability>,
Decodes a sequence of symbols, using an individual entropy model for each symbol.
This method is lazy: it doesn’t actually decode anything until you iterate over the returned iterator.
The return type implements Iterator<Item = Result<M::Symbol, ...>>
, and it
implements ExactSizeIterator
if the provided iterator models
implements
ExactSizeIterator
. The provided iterator models
may yield entropy models either
by value or by reference since, if M
implements DecoderModel<PRECISION>
, then
&M
does so too.
This method does not short-circuit. If an error e
occurs then the returned
iterator will yield Err(e)
but you can, in principle, continue to iterate over it.
In practice, however, continuing to iterate after an error will likely yield either
further errors or garbled symbols. Therefore, you’ll usually want to short-circuit
the returned iterator yourself, e.g., by applying the ?
operator on each yielded
item or by collect
ing the items from the returned iterator into a
Result<Container<_>, _>
as in the second example below.
Examples
Iterate over the decoded symbols in a for
loop
use constriction::{
stream::{model::DefaultLeakyQuantizer, stack::DefaultAnsCoder, Decode},
UnwrapInfallible,
};
// Create a decoder from some mock compressed data and create some mock entropy models.
let compressed = vec![0x2C63_D22E, 0x0000_0377];
let mut decoder = DefaultAnsCoder::from_compressed(compressed).unwrap();
let quantizer = DefaultLeakyQuantizer::new(-100i32..=100);
let entropy_models = (0..5).map(
|i| quantizer.quantize(probability::distribution::Gaussian::new((i * 10) as f64, 10.0))
);
// Decode the symbols and iterate over them in a `for` loop:
for symbol in decoder.decode_symbols(entropy_models) {
dbg!(symbol.unwrap_infallible()); // Prints the symbols `-3`, `12`, `19`, `28`, and `41`.
}
collect
the decoded symbols into a Result<Vec<M::Symbol>, _>
A useful trick when you have an iterator over Result
s is that you can collect
it
into Result<SomeContainer<_>, _>
:
// Assume same setup as in previous example.
let symbols = decoder.decode_symbols(entropy_models).collect::<Result<Vec<_>, _>>();
assert_eq!(symbols.unwrap_infallible(), [-3, 12, 19, 28, 41]);
This results in only a single memory allocation of the exact correct size, and it
automatically short-circuits upon error (which is moot in this example because, for
this particular entropy coder, both FrontendError
and BackendError
are
Infallible
, i.e., decoding cannot fail).
See Also
decode_symbol
if you want to decode only a single symbol;try_decode_symbols
if generating the entropy models may fail; anddecode_iid_symbols
if all symbols use the same entropy model.
sourcefn try_decode_symbols<'s, I, M, E>(
&'s mut self,
models: I
) -> TryDecodeSymbols<'s, Self, I::IntoIter, PRECISION> ⓘwhere
I: IntoIterator<Item = Result<M, E>> + 's,
M: DecoderModel<PRECISION>,
M::Probability: Into<Self::Word>,
Self::Word: AsPrimitive<M::Probability>,
fn try_decode_symbols<'s, I, M, E>(
&'s mut self,
models: I
) -> TryDecodeSymbols<'s, Self, I::IntoIter, PRECISION> ⓘwhere
I: IntoIterator<Item = Result<M, E>> + 's,
M: DecoderModel<PRECISION>,
M::Probability: Into<Self::Word>,
Self::Word: AsPrimitive<M::Probability>,
Decodes a sequence of symbols from a fallible iterator over entropy models.
This method is equivalent to decode_symbols
, except that it takes a fallible
iterator (i.e., an iterator that yields Result
s).
Just like decode_symbols
,
- this method is lazy, i.e., it doesn’t decode until you iterate over the returned iterator;
- the returned iterator implements
ExactSizeIterator
ifmodels
implementsExactSizeIterator
; and - you can, in principle, continue to decode after an error, but you’ll likely rather
want to short-circuit the returned iterator. This applies to both decoding errors
and to errors from the supplied iterator
models
.
The returned iterator yields
Ok(symbol)
on success;Err(TryCodingError::InvalidEntropyModel(e))
if the iteratormodels
yieldedErr(e)
; andErr(TryCodingError::CodingError(e))
if decoding resulted inErr(e)
.
This method may be useful for parameterized entropy models whose parameters have to satisfy certain constraints (e.g., they have to be positive), but they come from an untrusted source they may violate the constraints.
Example
use constriction::{
stream::{model::DefaultLeakyQuantizer, stack::DefaultAnsCoder, Decode, TryCodingError},
CoderError,
};
use core::convert::Infallible;
/// Helper function to decode symbols with Gaussian entropy models with untrusted parameters.
fn try_decode_gaussian(compressed: Vec<u32>, means: &[f64], stds: &[f64])
-> Result<Vec<i32>, TryCodingError<CoderError<Infallible, Infallible>, &'static str>>
{
let mut decoder = DefaultAnsCoder::from_compressed(compressed).unwrap();
let quantizer = DefaultLeakyQuantizer::new(-100i32..=100);
let entropy_models = means.iter().zip(stds).map(|(&mean, &std)| {
if std > 0.0 && mean.is_finite() {
Ok(quantizer.quantize(probability::distribution::Gaussian::new(mean, std)))
} else {
Err("argument error")
}
});
decoder.try_decode_symbols(entropy_models).collect::<Result<Vec<_>, _>>()
}
let compressed = vec![0x2C63_D22E, 0x0000_0377];
let means = [0.0, 10.0, 20.0, 30.0, 40.0];
let valid_stds = [10.0, 10.0, 10.0, 10.0, 10.0];
let invalid_stds = [10.0, 10.0, -1.0, 10.0, 10.0]; // Invalid: negative standard deviation
assert_eq!(
try_decode_gaussian(compressed.clone(), &means, &valid_stds),
Ok(vec![-3, 12, 19, 28, 41])
);
assert_eq!(
try_decode_gaussian(compressed, &means, &invalid_stds),
Err(TryCodingError::InvalidEntropyModel("argument error"))
);
sourcefn decode_iid_symbols<M>(
&mut self,
amt: usize,
model: M
) -> DecodeIidSymbols<'_, Self, M, PRECISION> ⓘwhere
M: DecoderModel<PRECISION> + Copy,
M::Probability: Into<Self::Word>,
Self::Word: AsPrimitive<M::Probability>,
fn decode_iid_symbols<M>(
&mut self,
amt: usize,
model: M
) -> DecodeIidSymbols<'_, Self, M, PRECISION> ⓘwhere
M: DecoderModel<PRECISION> + Copy,
M::Probability: Into<Self::Word>,
Self::Word: AsPrimitive<M::Probability>,
Decodes amt
symbols using the same entropy model for all symbols.
The return type implements ExactSizeIterator<Item = Result<M::Symbol, ...>>
.
Just like decode_symbols
,
- this method is lazy, i.e., it doesn’t decode until you iterate over the returned iterator; and
- you can, in principle, continue to decode after an error, but you’ll likely rather want to short-circuit the returned iterator.
While this method takes model
formally by value, you’ll typically want to pass the
DecoderModel
by reference (which is possible since any reference to a
DecoderModel
implements DecoderModel
too). The bound on M: Copy
prevents
accidental misuse in this regard. We provide the ability to pass the DecoderModel
by value as an opportunity for microoptimzations when dealing with models that can
be cheaply copied (see, e.g.,
LookupDecoderModel::as_view
).
If you want to decode each symbol with its individual entropy model, then consider
calling decode_symbols
instead. If you just want to decode a single symbol, then
call decode_symbol
instead.
sourcefn maybe_exhausted(&self) -> bool
fn maybe_exhausted(&self) -> bool
Checks if there might be no compressed data left for decoding.
If this method returns false
then there must be additional data left to decode. If
this method returns true
then it is unknown whether or not the decoder is
exhausted.
The default implementation always returns true
, which is always correct albeit not
particularly useful. Consider overwriting the default implementation and call
ReadWords::maybe_exhausted
on your backend if appropriate.
This method is useful for checking for consistency. If you have decoded all data
that you expect to be on a decoder but this method still returns false
then either
your code has a bug or the compressed data was corrupted. If this method returns
true
then that’s no guarantee that everything is correct, but at least there’s no
evidence of the contrary.
Calling this method can be awkward for entropy coders that implement
Decode<PRECISION>
for more than one value of PRECISION
. The method
Code::decoder_maybe_exhausted
is provided as a more convenient forwarding
method.