Crate bzip3_rs 

Pure-Rust port of the bzip3 compressor.

This crate reads and writes the bzip3 format. It has no C code and no unsafe code. Output is compatible with C bzip3 in both directions.

Each block passes through four stages: run-length coding, LZP, a Burrows-Wheeler transform, and an order-0 context-mixing coder. The crate checks a CRC32 value for each block.

Example

use bzip3_rs::{BlockSize, compress, decompress};

let data = b"hello world ".repeat(1000);
let coded = compress(&data, BlockSize::DEFAULT)?;
let plain = decompress(&coded)?;
assert_eq!(plain, data);

Where to start

• compress and decompress process a whole buffer.
• compress_with_jobs and decompress_with_jobs process blocks in parallel. See Jobs.
• Module stream processes std::io::Read and std::io::Write values.
• Modules read and write hold encoder and decoder adapters.
• BlockEncoder and BlockDecoder process one block at a time. BlockDecoder::recover_block_into keeps the data of a damaged block and returns a BlockStatus value.
• BlockSize holds the checked block size.

Frame formats

The crate uses two frame formats. They do not mix.

• The one-shot frame has a 13-byte header: magic, block size, and block count. compress and decompress use it.
• The streaming frame has a 9-byte header: magic and block size. Blocks follow until the input ends. Module stream, modules read and write, and the bzip3 tool use it.

Each decoder reads only its own frame. Use the decoder that matches the encoder.

use bzip3_rs::{BlockSize, decompress, stream};

let mut coded = Vec::new();
stream::compress_slice(b"hello world", &mut coded, BlockSize::MIN, bzip3_rs::Jobs::SERIAL)?;

// `decompress` reads the one-shot frame, so it rejects a streaming frame.
assert!(decompress(&coded).is_err());

Errors

Functions, constructors, and finish methods return error::Result.

The read and write adapters implement std::io::Read and std::io::Write. Their read and write methods return std::io::Error. A codec fault has the kind std::io::ErrorKind::InvalidData. The error::Error value is the source of that I/O error.

use std::io::{Cursor, ErrorKind, Read};
use bzip3_rs::error::Error;
use bzip3_rs::{BlockSize, Jobs, read, stream};

let mut coded = Vec::new();
stream::compress_slice(&b"hello world ".repeat(1000), &mut coded, BlockSize::MIN, Jobs::SERIAL)?;
let middle = coded.len() / 2;
coded[middle] ^= 0xFF; // Damage one payload byte.

let mut decoder = read::Decoder::new(Cursor::new(coded.as_slice()))?;
let fault = decoder.read_to_end(&mut Vec::new()).unwrap_err();
assert_eq!(fault.kind(), ErrorKind::InvalidData);
let codec = fault.get_ref().and_then(|source| source.downcast_ref::<Error>());
assert!(codec.is_some());

Parallelism

Every Jobs value gives the same output bytes. More threads use more memory, because each thread holds one block.

Cargo features

The library needs no feature. Feature cli builds the bzip3 tool.

Modules

error

Error type for the bzip3 port.

read

Read-based streaming compression wrappers.

stream

Parallel streaming compression over the 9-byte frame.

write

Write-based streaming compression wrappers.

Structs

BlockDecoder

Block decoder with constructor-owned working memory.

BlockEncoder

Block encoder with constructor-owned working memory.

BlockHeader

Sizes of one framed block, shared by both framings.

BlockSize

Block size in bytes, checked against the valid range.

StreamHeader

Streaming frame header: the magic bytes and the block size.

Enums

BlockStatus

State of the bytes that BlockDecoder::recover_block_into appended.

Jobs

Thread count for a codec call.

Constants

MAGIC_NUMBER

Magic bytes opening every frame.

Functions

compress

Compress data into one-shot framed output.

compress_bound

Estimate the largest compressed output for an input of input_size bytes.

compress_with_jobs

Compress data into one-shot framed output on a thread pool.

decompress

Decompress one-shot framed input.

decompress_with_jobs

Decompress one-shot framed input on a thread pool.

default_workers

Return the default worker count.