rust-brotli-decompressor
Project Requirements
Direct no-stdlib port of the C brotli decompressor to Rust
no dependency on the Rust stdlib: this library would be ideal for decompressing within a rust kernel among other things.
This will be useful to see how C and Rust compare in an apples-to-apples comparison where the same algorithms and data structures and optimizations are employed.
The current expected performance losses come from
- an extra indirection in the hgroups
- array bounds checks on every access
- no ability to load a full aligned 64 bit or 128 bit item from a [u8]
the system also enables all syscalls to be "frontloaded" in the initial generation of a memory pool for the allocator. Afterwards, SECCOMP can be activated or other mechanisms can be used to secure the application, if desired
Usage
With the io::Read abstraction
let mut input = new;
then you can simply read input as you would any other io::Read class
With the Stream Copy abstraction
match BrotliDecompress
With manual memory management
There are 3 steps to using brotli without stdlib
- setup the memory manager
- setup the BrotliState
- in a loop, call BrotliDecompressStream
in Detail
// at global scope declare a MemPool type -- in this case we'll choose the heap to
// avoid unsafe code, and avoid restrictions of the stack size
declare_stack_allocator_struct!;
// at local scope, make a heap allocated buffers to hold uint8's uint32's and huffman codes
let mut u8_buffer = define_allocator_memory_pool!;
let mut u32_buffer = define_allocator_memory_pool!;
let mut hc_buffer = define_allocator_memory_pool!;
let heap_u8_allocator = new_allocator;
let heap_u32_allocator = new_allocator;
let heap_hc_allocator = new_allocator;
// At this point no more syscalls are going to be needed since everything can come from the allocators.
// Feel free to activate SECCOMP jailing or other mechanisms to secure your application if you wish.
// Now it's possible to setup the decompressor state
let mut brotli_state = new;
// at this point the decompressor simply needs an input and output buffer and the ability to track
// the available data left in each buffer
loop
This interface is the same interface that the C brotli decompressor uses
Also feel free to use custom allocators that invoke Box directly. This example illustrates a mechanism to avoid subsequent syscalls after the initial allocation