Struct DecodeCursor

pub struct DecodeCursor<'a> { /* private fields */ }

Offers more flexible decoding than the top-level decode().

You can skip numbers you don’t need with skip(), and decode the parts of your input you need with decode_slice().

If you need maximum flexibility, you can use decode_sink() with a custom DecodeQuadSink implementation to receive numbers as they are decoded rather than storing them into a slice and then inspecting them.

Decode sinks

If you don’t want to write decoded numbers into a slice and inspect them later, you can use a custom sink. This is probably most useful when you want to minimize memory usage. For instance, you could mmap a file and scan through its contents with a custom sink without ever allocating on the heap.

There are two traits to represent a sink: one for complete quads (DecodeQuadSink), and one for any trailing leftover numbers that may not fill a quad (DecodeSingleSink). You will need to implement both with the appropriate Decoder::DecodedQuad type for the Decoder you are using. You can look at the implementations of TupleSink in the tests for examples.

Examples

Here’s a sink that calculates the maximum number in the input without writing the decoded input anywhere. (Unfortunately, due to a Rust bug I cannot include a SIMD example in a doc test.)

use std::cmp;

use rand::{Rng, seq::SliceRandom};

use stream_vbyte::{
    encode::encode,
    decode::{DecodeSingleSink, DecodeQuadSink, cursor::DecodeCursor},
    scalar::{Scalar, UnusedQuad},
    decode_quad_scalar
};

struct MaxSink {
    max: u32
}

impl MaxSink {
    fn new() -> MaxSink {
        MaxSink {
            max: 0
        }
    }
}

// Update the max as each number is decoded
impl DecodeSingleSink for MaxSink {
    fn on_number(&mut self, num: u32, _nums_decoded: usize) {
        self.max = cmp::max(self.max, num)
    }
}

// DecodeQuadSink is not used for `Scalar` decoder, but the type system insists
// on implementing it. This macro generates a stub impl.
decode_quad_scalar!(MaxSink);
// If this was using, say, the `Ssse3` decoder, would need a DecodeQuadSink impl to
// find the max `u32` in a `__m128i`, perhaps with `_mm_max_epu32` (SSE 4.1).

fn main() {
    let mut nums = vec![1, 2, 3, 5, 8, 13, 21, 34];

    // shuffle the numbers just so there's clearly nothing up our sleeve
    let mut rng = rand::thread_rng();
    nums.shuffle(&mut rng);

    let mut encoded = vec![0; nums.len() * 5];
    encode::<Scalar>(&nums, &mut encoded);

    let mut cursor = DecodeCursor::new(&encoded, nums.len());
    let mut sink = MaxSink::new();
    cursor.decode_sink::<Scalar, _>(&mut sink, nums.len());

    assert_eq!(34, sink.max);
}

Implementations

impl<'a> DecodeCursor<'a>

pub fn new(input: &'a [u8], count: usize) -> DecodeCursor<'a>

Create a new cursor.

pub fn skip(&mut self, to_skip: usize)

Skip to_skip numbers. to_skip must be a multiple of 4, and must not be greater than the count of remaining numbers that are in complete blocks of 4. In other words, if you have 7 numbers remaining (a block of 4 and a partial block of 3), the only count you can skip is 4.

Skipping numbers is several times faster than decoding them.

pub fn decode_slice<D: Decoder + WriteQuadToSlice>( &mut self, output: &mut [u32], ) -> usize

Decode into the output buffer.

If there is at least one complete quad of input remaining to decode, the buffer must be at least of size 4.

If there is only a final partial quad of input, the buffer must be at least as big as the remaining input.

Returns the number of numbers decoded by this invocation, which may be less than the size of the buffer.

pub fn decode_sink<D, S>( &mut self, sink: &mut S, max_numbers_to_decode: usize, ) -> usize

where D: Decoder, S: DecodeQuadSink<D> + DecodeQuadSink<Scalar>,

Decode at most max_numbers_to_decode numbers from the input and hand them to sink.

Decoding is done one quad at a time, except for the last quad, which may have fewer than four corresponding encoded numbers. Consequently, the number of numbers decoded will be a multiple of 4, unless max_numbers_to_decode includes the end of the encoded input, in which case the number of numbers will be all remaining numbers in the input regardless of whether that’s a multiple of 4 or not.

With each invocation of decode(), the nums_decoded parameter used in DecodeQuadSink.on_quad() will start counting up from 0 again.

Returns the number of numbers decoded.

pub fn input_consumed(&self) -> usize

Returns the total length of input scanned so far: the complete block of control bytes, plus any encoded numbers decoded.

pub fn has_more(&self) -> bool

Returns true iff there are more numbers to be decoded.

Trait Implementations

impl<'a> Debug for DecodeCursor<'a>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.