Trait constriction::stream::Encode

pub trait Encode<const PRECISION: usize>: Code {
    type FrontendError: Debug;
    type BackendError: Debug;
    fn encode_symbol<M>(
        &mut self, 
        symbol: impl Borrow<M::Symbol>, 
        model: M
    ) -> Result<(), CoderError<Self::FrontendError, Self::BackendError>>
    where
        M: EncoderModel<PRECISION>,
        M::Probability: Into<Self::Word>,
        Self::Word: AsPrimitive<M::Probability>;

    fn encode_symbols<S, M>(
        &mut self, 
        symbols_and_models: impl IntoIterator<Item = (S, M)>
    ) -> Result<(), CoderError<Self::FrontendError, Self::BackendError>>
    where
        S: Borrow<M::Symbol>,
        M: EncoderModel<PRECISION>,
        M::Probability: Into<Self::Word>,
        Self::Word: AsPrimitive<M::Probability>,
    { ... }
    fn try_encode_symbols<S, M, E>(
        &mut self, 
        symbols_and_models: impl IntoIterator<Item = Result<(S, M), E>>
    ) -> Result<(), TryCodingError<CoderError<Self::FrontendError, Self::BackendError>, E>>
    where
        S: Borrow<M::Symbol>,
        M: EncoderModel<PRECISION>,
        M::Probability: Into<Self::Word>,
        Self::Word: AsPrimitive<M::Probability>,
    { ... }
    fn encode_iid_symbols<S, M>(
        &mut self, 
        symbols: impl IntoIterator<Item = S>, 
        model: M
    ) -> Result<(), CoderError<Self::FrontendError, Self::BackendError>>
    where
        S: Borrow<M::Symbol>,
        M: EncoderModel<PRECISION> + Copy,
        M::Probability: Into<Self::Word>,
        Self::Word: AsPrimitive<M::Probability>,
    { ... }
    fn maybe_full(&self) -> bool { ... }
}

A trait for stream encoders (i.e., compressors)

This trait defines methods for encoding a single symbol or a sequence of symbols.

Naming Convention

This trait is deliberately called Encode and not Encoder. See corresponding comment for the Code trait for the reasoning.

Associated Types

type FrontendError: Debug

The error type for logical encoding errors.

This is often a DefaultEncoderFrontendError.

Frontend errors are errors that relate to the logical encoding process, such as when a user tries to encode a symbol that has zero probability under the provided entropy model. They are to be distinguished from backend errors, which depend on the used sources and/or sinks of compressed data, and which can be errors like “out of space” or “I/O error”.

type BackendError: Debug

The error type for writing out encoded data.

If you stick with the default backend(s) and don’t do anything fancy, then this will most likely be Infallible, which means that the compiler can optimize away error checks. You can explicitly express that you expect an Infallible error type by calling .unwrap_infallible() on a Result<T, Infallible> or on a Result<T, CoderError<Infallible, Infallible>> (you’ll have to bring the trait UnwrapInfallible into scope).

If you use a custom backend, then the BackendError will typically be the WriteError type of the of the backend. For example, it could signal an I/O error if you’re encoding directly to a file or socket rather than to an automatically growing in-memory buffer.

Required methods

fn encode_symbol<M>(
    &mut self,
    symbol: impl Borrow<M::Symbol>,
    model: M
) -> Result<(), CoderError<Self::FrontendError, Self::BackendError>> where
    M: EncoderModel<PRECISION>,
    M::Probability: Into<Self::Word>,
    Self::Word: AsPrimitive<M::Probability>, 

Encodes a single symbol with the given entropy model.

This is the most basic encoding method. If you want to encode more than a single symbol then you may want to call encode_symbols, try_encode_symbols, or encode_iid_symbols instead.

Note that:

• the symbol can be passed either by value or by reference;
• the model can also be passed by reference since, if M implements EncoderModel<PRECISION>, then &M does so too; and
• the PRECISION will typically be inferred from the entropy model, and it may not be larger than Word::BITS; this is enforced by run-time assertions (that get optimized away unless they fail) but it will be enforced at compile time in future versions of constriction as soon as the type system allows this.

Errors

Returns Err(CoderError::Frontend(e)) if there was a logic error e during encoding (such as trying to encode a symbol with zero probability under the provided entropy model). Returns Err(CoderError::Backend(e)) if writing compressed data lead to an I/O error e. Otherwise, returns Ok(()).

Example

use constriction::stream::{model::DefaultLeakyQuantizer, stack::DefaultAnsCoder, Encode};

// Create an ANS Coder and an entropy model.
let mut ans_coder = DefaultAnsCoder::new();
let quantizer = DefaultLeakyQuantizer::new(-100i32..=100);
let entropy_model = quantizer.quantize(probability::distribution::Gaussian::new(0.0, 10.0));

// Encode a symbol, passing both the symbol and the entropy model by reference:
ans_coder.encode_symbol(&12, &entropy_model).unwrap();

// Encode a symbol, passing both the symbol and the entropy model by value:
ans_coder.encode_symbol(-8, entropy_model).unwrap();

// Get the compressed bitstring.
let compressed = ans_coder.into_compressed();
dbg!(compressed);

Provided methods

fn encode_symbols<S, M>(
    &mut self,
    symbols_and_models: impl IntoIterator<Item = (S, M)>
) -> Result<(), CoderError<Self::FrontendError, Self::BackendError>> where
    S: Borrow<M::Symbol>,
    M: EncoderModel<PRECISION>,
    M::Probability: Into<Self::Word>,
    Self::Word: AsPrimitive<M::Probability>, 

Encodes a sequence of symbols, each with its individual entropy model.

The provided iterator has to yield pairs (symbol, entropy_model). The default implemnetation just calls encode_symbol for each item. You can overwrite the default implementation if your entropy coder can treat a sequence of symbols in a more efficient way.

This method short-circuits as soon as encoding leads to an error (see discussion of error states for encode_symbol).

This method encodes the symbols in the order in which they are yielded by the iterator. This is suitable for an encoder with “queue” semantics, like a RangeEncoder. If you’re using an encoder with “stack” semantics, such as an AnsCoder, then you may prefer encoding the symbols in reverse order (see AnsCoder::encode_symbols_reverse).

Note that:

• the symbols can be yielded either by value or by reference;
• the models can also be yielded either by value or by reference since, if M implements EncoderModel<PRECISION>, then &M does so too; and
• the iterator has to yield models of a fixed type (unless you want to Box up each model, which is most likely a bad idea); if you have entropy models of various types then just call either this method or encode_symbol several times manually.

See Also

• try_encode_symbols if generating the entropy models may fail; and
• encode_iid_symbols if all symbols use the same entropy model.

Example

use constriction::stream::{model::DefaultLeakyQuantizer, queue::DefaultRangeEncoder, Encode};

// Define the symbols we want to encode and the parameters of our entropy models.
let quantizer = DefaultLeakyQuantizer::new(-100i32..=100);
let symbols = [15, 3, -8, 2];
let means = [10.2, 1.5, -3.9, 5.1];
let stds = [7.1, 5.8, 10.9, 6.3];
let entropy_model = quantizer.quantize(probability::distribution::Gaussian::new(0.0, 10.0));

// Encode all symbols using range coding.
let mut encoder1 = DefaultRangeEncoder::new();
encoder1.encode_symbols(
    symbols.iter().zip(&means).zip(&stds).map(
        |((&symbol, &mean), &std)|
            (symbol, quantizer.quantize(probability::distribution::Gaussian::new(mean, std)))
    )
).unwrap();
let compressed1 = encoder1.into_compressed();

// The above is equivalent to:
let mut encoder2 = DefaultRangeEncoder::new();
for ((&symbol, &mean), &std) in symbols.iter().zip(&means).zip(&stds) {
    let model = quantizer.quantize(probability::distribution::Gaussian::new(mean, std));
    encoder2.encode_symbol(symbol, model).unwrap();
}
let compressed2 = encoder2.into_compressed();

assert_eq!(compressed1, compressed2);

fn try_encode_symbols<S, M, E>(
    &mut self,
    symbols_and_models: impl IntoIterator<Item = Result<(S, M), E>>
) -> Result<(), TryCodingError<CoderError<Self::FrontendError, Self::BackendError>, E>> where
    S: Borrow<M::Symbol>,
    M: EncoderModel<PRECISION>,
    M::Probability: Into<Self::Word>,
    Self::Word: AsPrimitive<M::Probability>, 

Encodes a sequence of symbols from a fallible iterator.

This method is equivalent to encode_symbols, except that it takes a fallible iterator (i.e., an iterator that yields Results). It encodes symbols as long as the iterator yields Ok((symbol, entropy_model)) and encoding succeeds. The method short-circuits as soon as either

• the iterator yields Err(e), in which case it returns Err(TryCodingError::InvalidEntropyModel(e)) to the caller; or
• encoding fails with Err(e), in which case it returns Err(TryCodingError::CodingError(e)).

This method may be useful for parameterized entropy models whose parameters have to satisfy certain constraints (e.g., they have to be positive), but they come from an untrusted source they may violate the constraints.

fn encode_iid_symbols<S, M>(
    &mut self,
    symbols: impl IntoIterator<Item = S>,
    model: M
) -> Result<(), CoderError<Self::FrontendError, Self::BackendError>> where
    S: Borrow<M::Symbol>,
    M: EncoderModel<PRECISION> + Copy,
    M::Probability: Into<Self::Word>,
    Self::Word: AsPrimitive<M::Probability>, 

Encodes a sequence of symbols, all with the same entropy model.

This method short-circuits as soon as encoding leads to an error (see discussion of error states for encode_symbol).

While this method takes model formally by value, you’ll typically want to pass the EncoderModel by reference (which is possible since any reference to an EncoderModel implements EncoderModel too). The bound on M: Copy prevents accidental misuse in this regard. We provide the ability to pass the EncoderModel by value as an opportunity for microoptimzations when dealing with models that can be cheaply copied (see, e.g., ContiguousCategoricalEntropyModel::as_view).

Note that this method encodes the symbols in the order in which they are yielded by the iterator. This is suitable for an encoder with “queue” semantics, like a RangeEncoder. If you’re using an encoder with “stack” semantics, such as an AnsCoder, then you may prefer encoding the symbols in reverse order (see AnsCoder::encode_iid_symbols_reverse).

If you want to encode each symbol with its individual entropy model, then consider calling encode_symbols instead. If you just want to encode a single symbol, then call encode_symbol instead.

fn maybe_full(&self) -> bool

Checks if there might not be any room to encode more data.

If this method returns false then encoding one more symbol must not fail due to a full backend (it may still fail for other reasons). If this method returns true then it is unknown whether or not encoding one more symbol will overflow the backend.

The default implementation always returns true, which is always correct albeit not particularly useful. Consider overwriting the default implementation and call WriteWords::maybe_full on your backend if appropriate.

Calling this method can be awkward for entropy coders that implement Encode<PRECISION> for more than one value of PRECISION. The method Code::encoder_maybe_full is provided as a more convenient forwarding method.

Implementors

impl<Word, State, Backend, const PRECISION: usize> Encode<PRECISION> for RangeEncoder<Word, State, Backend> where
    Word: BitArray + Into<State>,
    State: BitArray + AsPrimitive<Word>,
    Backend: WriteWords<Word>, 

type FrontendError = DefaultEncoderFrontendError

type BackendError = Backend::WriteError

fn encode_symbol<D>(
    &mut self,
    symbol: impl Borrow<D::Symbol>,
    model: D
) -> Result<(), CoderError<DefaultEncoderFrontendError, Self::BackendError>> where
    D: EncoderModel<PRECISION>,
    D::Probability: Into<Self::Word>,
    Self::Word: AsPrimitive<D::Probability>, 

fn maybe_full(&self) -> bool

impl<Word, State, Backend, const PRECISION: usize> Encode<PRECISION> for AnsCoder<Word, State, Backend> where
    Word: BitArray + Into<State>,
    State: BitArray + AsPrimitive<Word>,
    Backend: WriteWords<Word>, 

fn encode_symbol<M>(
    &mut self,
    symbol: impl Borrow<M::Symbol>,
    model: M
) -> Result<(), CoderError<DefaultEncoderFrontendError, Self::BackendError>> where
    M: EncoderModel<PRECISION>,
    M::Probability: Into<Self::Word>,
    Self::Word: AsPrimitive<M::Probability>, 

Encodes a single symbol and appends it to the compressed data.

This is a low level method. You probably usually want to call a batch method like encode_symbols or encode_iid_symbols instead. See examples there.

The bound impl Borrow<M::Symbol> on argument symbol essentially means that you can provide the symbol either by value or by reference, at your choice.

Returns Err(ImpossibleSymbol) if symbol has zero probability under the entropy model model. This error can usually be avoided by using a “leaky” distribution as the entropy model, i.e., a distribution that assigns a nonzero probability to all symbols within a finite domain. Leaky distributions can be constructed with, e.g., a LeakyQuantizer or with LeakyCategorical::from_floating_point_probabilities.

TODO: move this and similar doc comments to the trait definition.

type FrontendError = DefaultEncoderFrontendError

type BackendError = Backend::WriteError

fn maybe_full(&self) -> bool

impl<Word, State, CompressedBackend, RemainingBackend, const PRECISION: usize> Encode<PRECISION> for ChainCoder<Word, State, CompressedBackend, RemainingBackend, PRECISION> where
    Word: BitArray + Into<State>,
    State: BitArray + AsPrimitive<Word>,
    CompressedBackend: WriteWords<Word>,
    RemainingBackend: ReadWords<Word, Stack>, 

type FrontendError = EncoderFrontendError

type BackendError = BackendError<CompressedBackend::WriteError, RemainingBackend::ReadError>

fn encode_symbol<M>(
    &mut self,
    symbol: impl Borrow<M::Symbol>,
    model: M
) -> Result<(), EncoderError<Word, CompressedBackend, RemainingBackend>> where
    M: EncoderModel<PRECISION>,
    M::Probability: Into<Self::Word>,
    Self::Word: AsPrimitive<M::Probability>, 

fn maybe_full(&self) -> bool