Trait constriction::backends::ReadWords

pub trait ReadWords<Word, S: Semantics> {
    type ReadError: Debug;

    fn read(&mut self) -> Result<Option<Word>, Self::ReadError>;

    fn maybe_exhausted(&self) -> bool { ... }
}

A trait for sources of compressed data (mainly used by decoders).

See the module-level documentation for more information, in particular regarding the type parameter S: Semantics.

Required Associated Types

type ReadError: Debug

The error type that can occur when reading from the data source, or Infallible.

Note that “end of file” / “out of data” is not considered an error. The read method indicates “end of file” by returning Ok(None), not Err(...). If reading the data source cannot fail (except for “end of file”) then ReadError should be Infallible so that the compiler can optimize out any error checks (see also UnwrapInfallible).

Required Methods

fn read(&mut self) -> Result<Option<Word>, Self::ReadError>

Reads a single Word from the data source and advances the state of the data source accordingly (i.e., so that the next read won’t read the same Word again).

Returns

• Ok(Some(word)) if the read succeeded;
• Ok(None) if the backend is exhausted (i.e., there’s no more data left); or
• Err(err) if an error err other than “end of file” occurred during reading (e.g., a file system error)

Note that ReadWords::read has stricter requirements than the standard library’s Iterator::next. Once ReadWords::read indicates end of file by returning Ok(None), it must never return Ok(Some(_)) when called again (i.e., types that implement ReadWords have to be “fused”, in iterator terminology). Entropy coders may rely on this contract for correctness of the encoded and decoded data but not for memory safety.

Provided Methods

fn maybe_exhausted(&self) -> bool

Returns true if the data source could be out of data.

The default implementation always returns true since returning true makes no statement. Overwrite the default implementation if you may in some cases be able to say with certainty that there is still data left to be read, and return false in these cases.

If maybe_exhausted() returns false then the next call to read must return either Ok(Some(_)) or Err(_) but not Ok(None).

Examples found in repository

src/backends.rs (line 794)

    fn maybe_exhausted(&self) -> bool {
        self.0.maybe_exhausted()
    }
}

impl<Word, B: ReadWords<Word, Queue>> ReadWords<Word, Stack> for Reverse<B> {
    type ReadError = B::ReadError;

    #[inline(always)]
    fn read(&mut self) -> Result<Option<Word>, Self::ReadError> {
        self.0.read()
    }

    #[inline(always)]
    fn maybe_exhausted(&self) -> bool {
        self.0.maybe_exhausted()
    }

src/stream/chain.rs (line 1069)

    fn maybe_exhausted(&self) -> bool {
        self.compressed.maybe_exhausted() || self.remainders.maybe_full()
    }
}

impl<Word, State, CompressedBackend, RemaindersBackend, const PRECISION: usize> Encode<PRECISION>
    for ChainCoder<Word, State, CompressedBackend, RemaindersBackend, PRECISION>
where
    Word: BitArray + Into<State>,
    State: BitArray + AsPrimitive<Word>,
    CompressedBackend: WriteWords<Word>,
    RemaindersBackend: ReadWords<Word, Stack>,
{
    type FrontendError = EncoderFrontendError;
    type BackendError = BackendError<CompressedBackend::WriteError, RemaindersBackend::ReadError>;

    fn encode_symbol<M>(
        &mut self,
        symbol: impl Borrow<M::Symbol>,
        model: M,
    ) -> Result<(), EncoderError<Word, CompressedBackend, RemaindersBackend>>
    where
        M: EncoderModel<PRECISION>,
        M::Probability: Into<Self::Word>,
        Self::Word: AsPrimitive<M::Probability>,
    {
        // assert!(State::BITS >= Word::BITS + PRECISION);
        assert!(PRECISION <= Word::BITS);
        assert!(PRECISION > 0);

        let (left_sided_cumulative, probability) = model
            .left_cumulative_and_probability(symbol)
            .ok_or(CoderError::Frontend(EncoderFrontendError::ImpossibleSymbol))?;

        if self.heads.remainders
            < probability.get().into().into() << (State::BITS - Word::BITS - PRECISION)
        {
            self.refill_remainders_head()?;
            // At this point, the invariant on `self.heads.remainders` (see its doc comment) is
            // temporarily violated (but it will be restored below). This is how
            // `decode_symbol` can detect that it has to flush `remainders.state`.
        }

        let remainder = (self.heads.remainders % probability.get().into().into())
            .as_()
            .as_();
        let quantile = (left_sided_cumulative + remainder).into();
        self.heads.remainders = self.heads.remainders / probability.get().into().into();

        if PRECISION != Word::BITS
            && self.heads.compressed.get() < Word::one() << (Word::BITS - PRECISION)
        {
            unsafe {
                // SAFETY:
                // - `heads.compressed` is nonzero because it is a `NonZero`
                // - `heads.compressed`, has `Word::BITS` bits and we checked above that all its one
                //   bits are within theleast significant `Word::BITS - PRECISION` bits. Thus, the
                //   most significant `PRECISION` bits are 0 and the left-shift doesn't truncate.
                // Thus, the result of the left-shift is also noznero.
                self.heads.compressed =
                    (self.heads.compressed.get() << PRECISION | quantile).into_nonzero_unchecked();
            }
        } else {
            let word = if PRECISION == Word::BITS {
                quantile
            } else {
                let word = self.heads.compressed.get() << PRECISION | quantile;
                unsafe {
                    // SAFETY: if we're here then `heads.compressed >= 1 << (Word::BITS - PRECISION).
                    // Thus, shifting right by this amount of bits leaves at least one 1 bit.
                    self.heads.compressed = (self.heads.compressed.get()
                        >> (Word::BITS - PRECISION))
                        .into_nonzero_unchecked();
                }
                word
            };
            self.compressed
                .write(word)
                .map_err(BackendError::Compressed)?;
        }

        Ok(())
    }

    fn maybe_full(&self) -> bool {
        self.remainders.maybe_exhausted() || self.compressed.maybe_full()
    }

src/stream/queue.rs (line 716)

    pub fn maybe_exhausted(&self) -> bool {
        // The maximum possible difference between `point` and `lower`, even if the
        // compressed data was concatenated with a lot of one bits.
        let max_difference =
            ((State::one() << (State::BITS - Word::BITS)) << 1).wrapping_sub(&State::one());

        // The check for `self.state.range == State::max_value()` is for the special case of
        // an empty buffer.
        self.bulk.maybe_exhausted()
            && (self.state.range.get() == State::max_value()
                || self.point.wrapping_sub(&self.state.lower) < max_difference)
    }

Implementations on Foreign Types

impl<Word> ReadWords<Word, Stack> for Vec<Word>

type ReadError = Infallible

The only way how reading from a vector can fail is if the vector is empty, but that’s not considered an error (it returns Ok(None) instead).

fn read(&mut self) -> Result<Option<Word>, Self::ReadError>

Pops the word off the end of the vector (= top of the stack). If you instead want to keep the data unchanged (e.g., because you want to reuse it later) then wrap either the vector v or or the slice &v[..] in a Cursor.

fn maybe_exhausted(&self) -> bool

impl<Array> ReadWords<<Array as Array>::Item, Stack> for SmallVec<Array>where
    Array: Array,

type ReadError = Infallible

The only way how reading from a vector can fail is if the vector is empty, but that’s not considered an error (it returns Ok(None) instead).

fn read(&mut self) -> Result<Option<Array::Item>, Self::ReadError>

Pops the word off the end of the vector (= top of the stack). If you instead want to keep the data unchanged (e.g., because you want to reuse it later) then wrap either the vector v or or the slice &v[..] in a Cursor.

fn maybe_exhausted(&self) -> bool

Implementors

impl<Iter, S, Word> ReadWords<Word, S> for InfallibleIteratorReadWords<Iter>where
    Iter: Iterator<Item = Word>,
    S: Semantics,

Since InfallibleIteratorReadWords doesn’t implement WriteWords, it is allowed to implement ReadWords for all ReadWriteLogics

type ReadError = Infallible

impl<Iter, S, Word, ReadError> ReadWords<Word, S> for FallibleIteratorReadWords<Iter>where
    Iter: Iterator<Item = Result<Word, ReadError>>,
    S: Semantics,
    ReadError: Debug,

Since FallibleIteratorReadWords doesn’t implement WriteWords, it is allowed to implement ReadWords for all ReadWriteLogics

type ReadError = ReadError

impl<Word, B: ReadWords<Word, Queue>> ReadWords<Word, Stack> for Reverse<B>

type ReadError = <B as ReadWords<Word, Queue>>::ReadError

impl<Word, B: ReadWords<Word, Stack>> ReadWords<Word, Queue> for Reverse<B>

type ReadError = <B as ReadWords<Word, Stack>>::ReadError

impl<Word: Clone, Buf: AsRef<[Word]>> ReadWords<Word, Queue> for Cursor<Word, Buf>

type ReadError = Infallible

impl<Word: Clone, Buf: SafeBuf<Word>> ReadWords<Word, Stack> for Cursor<Word, Buf>

type ReadError = Infallible