Crate snappy_framed [−] [src]
Normally, Snappy compression works entirely in memory, but
there is also a "Snappy framed" format which can be read and
written in streaming mode. We provide Read
and Write
implementations for framed snappy data.
The API to this library is designed to be similar to that of
flate2
, though we have not yet implemented
read::SnappyFramedEncoder
or write::SnappyFramedDecoder
.
A note about checksums
The "Snappy framed" format includes CRC-32C checksums, which can be
computed relatively efficiently on Intel processors. However,
implementations disagree on the byte order of the checksums! The Java
implementations, the snzip
command-line tool and the
SmallTalk implementation use one byte order, and the Node.js and Python
implementations use another.
For now, we have chosen to generate checksums in the format used by the
Java and snzip
libraries. When reading, we can either verify
Java-format checksums, or we can ignore the checksums entirely.
Limitations
This library is still a work in progress:
- Invalid streams will probably result in a panic.
- Decompression performance has been tuned a fair bit, except for CRCs, but there's probably an extra 25% or so to be gained by further tweaking.
- We currently assume that you will
write
data in large blocks when compressing, and we will generate poorly-compressed data if you make lots of small writes. This could be fixed by using an internal write buffer.
Modules
read |
Implementations of |
write |
Implementations of |