Crate leb128fmt

Leb128fmt is a library to decode and encode LEB128 formatted numbers. LEB128 is a variable length integer compression format.

The library does not allocate memory and can be used in no_std and no_std::no_alloc environments.

Various functions are provided which encode and decode signed and unsigned integers with the number of bits in the function name. There are generic functions provided to read and write slices of encoded values as well.

There are encoding functions with the word fixed in the name which will write out a value using the maximum number of bytes for a given bit size. For instance, using encode_fixed_u32 will always use 5 bytes to write out the value. While always using the maximum number of bytes removes the benefit of compression, in some scenarios, it is beneficial to have a fixed encoding size.

Finally, there are macros provided which you can use to build your own encoding and decoding functions for unusual variants like signed 33 bit values.

Examples

Functions using Arrays

// Encode an unsigned 32 bit number:
let (output, written_len) = leb128fmt::encode_u32(43110).unwrap();
// The number of bytes written in the output array
assert_eq!(written_len, 3);
assert_eq!(&output[..written_len], &[0xE6, 0xD0, 0x02]);
// The entire output array. Note you should only use &output[..written_len] to copy
// into your output buffer
assert_eq!(output, [0xE6, 0xD0, 0x02, 0x00, 0x00]);

// Decode an unsigned 32 bit number:
let input = [0xE6, 0xD0, 0x02, 0x00, 0x00];
let (result, read_len) = leb128fmt::decode_u32(input).unwrap();
assert_eq!(result, 43110);
assert_eq!(read_len, 3);

Helper Functions

If you are reading from an input buffer, you can use is_last and max_len to determine the bytes to copy into the array.

let buffer = vec![0xFE, 0xFE, 0xE6, 0xD0, 0x02, 0xFE, 0xFE, 0xFE];
let pos = 2;
let end = buffer.iter().skip(pos).copied().position(leb128fmt::is_last).map(|p| pos + p);
if let Some(end) = end {
    if end <= pos + leb128fmt::max_len::<32>() {
        let mut input = [0u8; leb128fmt::max_len::<32>()];
        input[..=end - pos].copy_from_slice(&buffer[pos..=end]);
        let (result, read_len) = leb128fmt::decode_u32(input).unwrap();
        assert_eq!(result, 43110);
        assert_eq!(read_len, 3);
    } else {
        // invalid LEB128 encoding
    }
} else {
  if buffer.len() - pos < leb128fmt::max_len::<32>() {
     // Need more bytes in the buffer
  } else {
     // invalid LEB128 encoding
  }
}

Functions Using Slices

let mut buffer = vec![0xFE; 10];
let mut pos = 1;

// Encode an unsigned 64 bit number with a mutable slice:
let result = leb128fmt::encode_uint_slice::<u64, 64>(43110u64, &mut buffer, &mut pos);
// The number of bytes written in the output array
assert_eq!(result, Some(3));
assert_eq!(pos, 4);

assert_eq!(buffer, [0xFE, 0xE6, 0xD0, 0x02, 0xFE, 0xFE, 0xFE, 0xFE, 0xFE, 0xFE]);

// Decode an unsigned 64 bit number with a slice:
pos = 1;
let result = leb128fmt::decode_uint_slice::<u64, 64>(&buffer, &mut pos);
assert_eq!(result, Ok(43110));
assert_eq!(pos, 4);

Functions Using Fixed Sized Encoding

There may be several different ways to encode a value. For instance, 0 can be encoded as 32 bits unsigned:

let mut pos = 0;
assert_eq!(leb128fmt::decode_uint_slice::<u32, 32>(&[0x00], &mut pos), Ok(0));
pos = 0;
assert_eq!(leb128fmt::decode_uint_slice::<u32, 32>(&[0x80, 0x00], &mut pos), Ok(0));
pos = 0;
assert_eq!(leb128fmt::decode_uint_slice::<u32, 32>(&[0x80, 0x80, 0x00], &mut pos), Ok(0));
pos = 0;
assert_eq!(leb128fmt::decode_uint_slice::<u32, 32>(&[0x80, 0x80, 0x80, 0x00], &mut pos), Ok(0));
pos = 0;
assert_eq!(leb128fmt::decode_uint_slice::<u32, 32>(&[0x80, 0x80, 0x80, 0x80, 0x00], &mut pos), Ok(0));

There are functions provided to encode a value using the maximum number of bytes possible for a given bit size. Using the maximum number of bytes removes the benefit of compression, but it may be useful in a few scenarios.

For instance, if a binary format needs to store the size or offset of some data before the size of data is known, it can be beneficial to write a fixed sized 0 placeholder value first. Then, once the real value is known, the 0 placeholder can be overwritten without moving other bytes. The real value is also written out using the fixed maximum number of bytes.

// Encode an unsigned 32 bit number with all 5 bytes:
let output  = leb128fmt::encode_fixed_u32(43110).unwrap();
assert_eq!(output, [0xE6, 0xD0, 0x82, 0x80, 0x00]);

// Decode an unsigned 32 bit number:
let input = output;
let (result, read_len) = leb128fmt::decode_u32(input).unwrap();
assert_eq!(result, 43110);

// Note that all 5 bytes are read
assert_eq!(read_len, 5);

Macros

• decode_sint_arr
  Builds custom signed integer decode functions.
• decode_uint_arr
  Builds custom unsigned integer decode functions.
• encode_fixed_sint_arr
  Builds custom signed integer encode functions with the max byte length of byte arrays used.
• encode_fixed_uint_arr
  Builds custom unsigned integer encode functions with the max byte length of byte arrays used.
• encode_sint_arr
  Builds custom signed integer encode functions.
• encode_uint_arr
  Builds custom unsigned integer encode functions.

Structs

• Error
  Error when decoding a LEB128 value.

Traits

• SInt
  Sealed trait for supported signed integer types.
• UInt
  Sealed trait for supported unsigned integer types.

Functions

• decode_s32
  Decodes an unsigned LEB128 number.
• decode_s64
  Decodes an unsigned LEB128 number.
• decode_sint_slice
  Decodes an unsigned integer from a slice of bytes and starting at a given position.
• decode_u32
  Decodes an unsigned LEB128 number.
• decode_u64
  Decodes an unsigned LEB128 number.
• decode_uint_slice
  Decodes an unsigned integer from a slice of bytes and starting at a given position.
• encode_fixed_s32
  Encodes a value as a signed LEB128 number.
• encode_fixed_s64
  Encodes a value as a signed LEB128 number.
• encode_fixed_sint_slice
  Encodes a given value into an output slice using a fixed set of bytes.
• encode_fixed_u32
  Encodes an unsigned LEB128 number with using the maximum number of bytes for the given bits length.
• encode_fixed_u64
  Encodes an unsigned LEB128 number with using the maximum number of bytes for the given bits length.
• encode_fixed_uint_slice
  Encodes a given value into an output slice using a fixed set of bytes.
• encode_s32
  Encodes a value as a signed LEB128 number.
• encode_s64
  Encodes a value as a signed LEB128 number.
• encode_sint_slice
  Encodes a given value into an output slice using the fixed set of bytes.
• encode_u32
  Encodes a value as an unsigned LEB128 number.
• encode_u64
  Encodes a value as an unsigned LEB128 number.
• encode_uint_slice
  Encodes a given value into an output slice using the fixed set of bytes.
• is_last
  Returns true if this is the last byte in an encoded LEB128 value.
• max_len
  Returns the maximum byte length that is used to encode a value for a given number of BITS.