Crate lz4_flex

Pure Rust, high performance implementation of LZ4 compression.

A detailed explanation of the algorithm can be found here.

Overview

This crate provides two ways to use lz4. The first way is through the frame::FrameDecoder and frame::FrameEncoder types, which implement the std::io::Read and std::io::Write traits with the lz4 frame format. Unless you have a specific reason to the contrary, you should only use the lz4 frame format. Specifically, the lz4 frame format permits streaming compression or decompression.

The second way is through the decompress_size_prepended and compress_prepend_size functions. These functions provide access to the lz4 block format, and don’t support a streaming interface directly. You should only use these types if you know you specifically need the lz4 block format.

Example: compress data on stdin with frame format

This program reads data from stdin, compresses it and emits it to stdout. This example can be found in examples/compress.rs:

use std::io;
let stdin = io::stdin();
let stdout = io::stdout();
let mut rdr = stdin.lock();
// Wrap the stdout writer in a LZ4 Frame writer.
let mut wtr = lz4_flex::frame::FrameEncoder::new(stdout.lock());
io::copy(&mut rdr, &mut wtr).expect("I/O operation failed");
wtr.finish().unwrap();

Example: decompress data on stdin with frame format

This program reads data from stdin, decompresses it and emits it to stdout. This example can be found in examples/decompress.rs:

use std::io;
let stdin = io::stdin();
let stdout = io::stdout();
// Wrap the stdin reader in a LZ4 FrameDecoder.
let mut rdr = lz4_flex::frame::FrameDecoder::new(stdin.lock());
let mut wtr = stdout.lock();
io::copy(&mut rdr, &mut wtr).expect("I/O operation failed");

Example: block format roundtrip

use lz4_flex::{compress_prepend_size, decompress_size_prepended};
let input: &[u8] = b"Hello people, what's up?";
let compressed = compress_prepend_size(input);
let uncompressed = decompress_size_prepended(&compressed).unwrap();
assert_eq!(input, uncompressed);

Feature Flags

• safe-encode uses only safe rust for encode. enabled by default
• safe-decode uses only safe rust for encode. enabled by default
• checked-decode will add additional checks if safe-decode is not enabled, to avoid out of bounds access. This should be enabled for untrusted input.
• frame support for LZ4 frame format. implies std, enabled by default
• std enables dependency on the standard library. enabled by default

For maximum performance use no-default-features.

Modules

block

LZ4 Block Format

frame

LZ4 Frame Format

Functions

compress

Compress all bytes of input.

compress_into

Compress all bytes of input into output. The method chooses an appropriate hashtable to lookup duplicates and calls compress_into_with_table. output should be preallocated with a size of get_maximum_output_size.

compress_prepend_size

Compress all bytes of input into output. The uncompressed size will be prepended as a little endian u32. Can be used in conjunction with decompress_size_prepended

decompress

Decompress all bytes of input into a new vec.

decompress_into

Decompress all bytes of input into output. output should be preallocated with a size of of the uncompressed data.

decompress_size_prepended

Decompress all bytes of input into a new vec. The first 4 bytes are the uncompressed size in litte endian. Can be used in conjunction with compress_prepend_size