mbrotli

Brotli compression in safe Rust, with qualities 0–11, reusable encoder storage, streaming I/O, and caller-scheduled parallel compression. This crate provides compression only; it does not include a decoder.

Getting started

Requires Rust 1.89 or later.

[dependencies]
mbrotli = "0.1"

use mbrotli::{Compressor, EncoderConfig, Quality};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = EncoderConfig::default().with_quality(Quality::Q5);
    let mut encoder = Compressor::new(config)?;
    let input = "brotli ".repeat(1000);
    let compressed = encoder.compress(input.as_bytes())?;
    println!("{} -> {} bytes", input.len(), compressed.len());
    Ok(())
}

Set the quality explicitly to control compression effort. EncoderConfig::default() uses quality 11, the most expensive search.

Choosing an API

A Compressor owns reusable working buffers. Encoding takes &mut self; reuse the same compressor for successive streams.

For repeated operations, reuse both the compressor and the destination:

use mbrotli::{Compressor, EncoderConfig, Quality};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut encoder = Compressor::new(
        EncoderConfig::default().with_quality(Quality::Q5),
    )?;
    let mut output = Vec::new();
    for input in [b"first payload".as_slice(), b"second payload".as_slice()] {
        output.clear();
        let range = encoder.compress_into(input, &mut output)?;
        assert_eq!(range, 0..output.len());
    }
    Ok(())
}

Streaming

Call finish to terminate a writer's stream and recover its sink. Dropping the writer does not finish it. flush makes accepted input decodable without ending the stream; flush boundaries can affect compressed size.

use mbrotli::io::FinishError;
use mbrotli::{Compressor, EncoderConfig, InputSize, Quality, StreamConfig};
use std::io::Write;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut encoder = Compressor::new(
        EncoderConfig::default().with_quality(Quality::Q5),
    )?;
    let input = "brotli ".repeat(1000);
    let stream = StreamConfig::from(InputSize::Exact(input.len() as u64));
    let mut writer = encoder.writer(Vec::new(), stream)?;
    for chunk in input.as_bytes().chunks(512) {
        writer.write_all(chunk)?;
    }
    let streamed = writer.finish().map_err(FinishError::into_error)?;
    assert_eq!(streamed, encoder.compress(input.as_bytes())?);
    Ok(())
}

All serial APIs emit identical bytes with the same configuration, dictionary, declared input size, flush boundaries, and continuation offset. To match a one-shot call, use InputSize::Exact(input.len() as u64), offset zero, and no explicit flushes. Caller chunk sizes and available SIMD backends do not change the output.

Parallel compression emits one stream from independent segments. Its output is deterministic across task counts for fixed segment settings, but can differ in both bytes and size from serial compression.

Dictionaries and format support

Unsupported quality/feature combinations return errors. A decoder needs the same external dictionaries to decode a stream that references them. The experimental API may change in a patch release.

[dependencies]
mbrotli = { version = "0.1", features = ["experimental"] }

The encoder is a port of Google's Brotli v1.2.0, pinned in the repository's brotli-ffi/vendor/brotli submodule at 028fb5a. Tests compare ordinary output with equivalent C streaming settings and decode it with C. Native C one-shot shortcuts and arbitrary C chunk schedules can produce different bytes. Custom static search and framing have separate compatibility checks. Declared windows above 30 bits lack an independent end-to-end decoder check in this repository.

Documentation

• User guide: configuration, buffers, streaming, and errors.
• Dictionaries and extended formats: preparation, limits, and experimental features.
• Parallel compression: task scheduling, input sources, and staging.
• Benchmarks and profiling: workloads and reproducible commands.
• Development: build, checks, coverage, and fuzzing.
• Architecture: implementation mechanics and diagrams.

Runnable examples:

cargo run --example compress
cargo run --release --example parallel -- INPUT OUTPUT
cargo doc --no-deps --open