Crate compressed_collections

source ·
Expand description

Collections which transparently compress data to reduce memory usage

This crate offers collections which automatically compress themselves, which reduces memory usage, sometimes substantially, allowing collections to be held in memory that would otherwise be too big.

So far, a stack is implemented, which can be used as a dropin replacement for a Vec stack. The only restriction on the datatypes in the collections is that they must be serde serializable.

For instance:

use compressed_collections::Stack;

let mut compressed_stack = Stack::new();
for _ in 0..(1024 * 1024 * 100) {
    compressed_stack.push(1);
}

This only allocates around 10MB (the default buffer size), whereas the equivalent vector would be around 100MB in size.

Design goals:

  • Provide collections with a subset of the API of the standard equivalent wherever possible for easy dropin use.
  • Only implement the efficient operations for each datastructure

Datastructures:

  • Stack
  • Deque
  • Map

Structs

Deque
A deque which automatically compresses itself over a certain size
Stack
A stack which automatically compresses itself over a certain size

Enums

ChunkSize
The amount of data to buffer before compressing.