Crate streamvbyte

A integer compression library which wraps the c streamvbyte encoder/decoder

Encodes 32 bit integers into variable length byte sequences in O(n) time.

Input can be at most u32::MAX integers.

Examples

Encode an u32 slice into a new buf in with encode:

use streamvbyte::encode;
let out_bytes: Vec<u8> = encode(&[1,2,44,5123,43,534]);

…or by using the encode_to_buf function into an existing buffer:

use streamvbyte::{max_compressedbytes,encode_to_buf};
let input = vec![1,2,44,5123,43,534];
let max_bytes = max_compressedbytes(input.len());
let mut out_buf = vec![0;max_bytes];
let bytes_written = encode_to_buf(&input,&mut out_buf);
assert_eq!(bytes_written.unwrap(),10);

You can use encode_delta to encode increasing sequences more effectively:

use streamvbyte::encode_delta;
let out_bytes: Vec<u8> = encode_delta(&[1,2,44,64,71,534],0);

Decoding values works in much the same way:

use streamvbyte::{decode_delta,encode_delta};
let out_bytes: Vec<u8> = encode_delta(&[1,2,44,64,71,534],0);
let mut recovered = vec![0;6];
let bytes_read = decode_delta(&out_bytes,&mut recovered,0);
assert_eq!(out_bytes.len(),bytes_read);
assert_eq!(&recovered,&[1,2,44,64,71,534]);

Note: length of the output buf recovered needs to match the input length. This information needs to be stored external to compressed output.

Enums

StreamVbyteError

! Errors that can be emitted from the streamvbyte crate and the underlying -sys crate

Functions

decode

Decode a sequence of u32 integers from a vbyte encoded byte representation into an existing buffer output.

decode_delta

Decode a sequence of non decreasing u32 integers from a vbyte encoded byte representation into an existing buffer output.

encode

Encode a sequence of u32 integers into a vbyte encoded byte representation. Internally a buffer of length max_compressedbytes is allocated to store the compressed result.

encode_delta

Encode a sequence non decreasing of u32 integers into a vbyte encoded byte representation. Internally a buffer of length max_compressedbytes is allocated to store the compressed result.

encode_delta_to_buf

Encode a sequence non decreasing of u32 integers into a vbyte encoded byte representation into an existing buffer output.

encode_to_buf

Encode a sequence of u32 integers into a vbyte encoded byte representation into an existing buffer output.

max_compressedbytes

! Returns the maximum number of bytes required by the compressor to encode length u32s