A set of primitives and Serde serializers for fast, prefix-free encoding which preserves lexicographic ordering of values.
It is intended for encoding keys and values in key-value databases.
In most existing designs, prefix-free encoding of byte sequences is performed by escaping
"end-of-sequence" bytes. This takes extra space, and makes it difficult to know sequence length
without processing the whole input buffer; this also complicates memory allocation for
deserialized data. Instead, we take advantage of the fact that exact record size is always
known in key-value databases. This implementation relies on "two-sided" buffer design:
sequence lengths are varint-encoded and pushed to the tail end of the buffer, so
it is possible to get original length of serialized byte sequence(s) by deserializing
a few bytes only.
For serialization, this implementation provides (very fast) calculation of exact size
of serialized data length before serialization itself. These features
enable effective and predictable buffer management for repetitive scans and no-heap
- encodings in both ascending and descending lexicographic orderings are supported
- encoding puts lengths of variable-size sequences to the end of serialized data, so resulting encoding is prefix-free and friendly to lexicographic ordering
- zero allocations, supports
- method to cheaply get exact size of serialized data without doing actual serialization, for effective buffer management
- space-efficient varint encoding for sequence lengths and discriminants
- easily customizable (endianness, encoding of primitive types etc.), with useful pre-sets
- reader/writer traits for double-ended buffers, so you can implement your own or use implementations provided by the crate
- no unsafe code
serde(on by default): include
serdeserializer and deserializer. If you need only primitives, you can opt out.
std(on by default): opt out for
#[no-std]use, you will lose some convenience methods which use
The underlying encoding format is simple and unlikely to change.
As a safeguard,
FormatVersion trait for all serializer parameter
Note: serializing with descending lexicographic order is particularly useful for key-value databases like rocksdb, where reverse iteration is slower than forward iteration.
Types and traits for working with serialization and deserialization buffers
Prefix-free encoding and decoding of byte sequences, with escaping
Serialization parameters traits and types
Ordered serialization/deserialization for primitive types and byte arrays.
Fast variable length serialization of unsigned integers with
Serialized object size calculator
Serialization and deserialization errors
lexicographic ordering for serialization
Calculate exact size of serialized data for a
Deserialize value from byte slice with
Deserialize value from mutable byte slice.
Create new default deserializer instance (with
Create new default serializer instance (with
A convenient Result type