mbrotli

Brotli compression in safe Rust, with qualities 0–11, reusable encoder storage, streaming I/O, and caller-scheduled parallel compression. This crate provides compression only; it does not include a decoder.

Benchmark results

Median across all eight datasets, with each dataset weighted equally after normalizing to Google C Brotli. Higher speed and lower output are better; 1× matches C. Qualities with the strongest mbrotli median speed / C appear first. Cold serial APIs, window 22, i7-13700KF / WSL2; recorded 2026-09-07. Burli supports q0–q5. Equal quality does not imply equal output size.

Explore the median results, dataset charts, and measurement method and raw data.

Getting started

Requires Rust 1.89 or later.

[dependencies]
mbrotli = "0.1"

use mbrotli::{Compressor, EncoderConfig, Quality};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = EncoderConfig::default().with_quality(Quality::Q5);
    let mut encoder = Compressor::new(config)?;
    let input = "brotli ".repeat(1000);
    let compressed = encoder.compress(input.as_bytes())?;
    println!("{} -> {} bytes", input.len(), compressed.len());
    Ok(())
}

Set the quality explicitly to control compression effort. EncoderConfig::default() uses quality 11, the most expensive search.

Choosing an API

A Compressor owns reusable working buffers. Encoding takes &mut self; reuse the same compressor for successive streams.

For repeated operations, reuse both the compressor and the destination:

use mbrotli::{Compressor, EncoderConfig, Quality};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut encoder = Compressor::new(
        EncoderConfig::default().with_quality(Quality::Q5),
    )?;
    let mut output = Vec::new();
    for input in [b"first payload".as_slice(), b"second payload".as_slice()] {
        output.clear();
        let range = encoder.compress_into(input, &mut output)?;
        assert_eq!(range, 0..output.len());
    }
    Ok(())
}

Streaming

Call finish to terminate a writer's stream and recover its sink. Dropping the writer does not finish it. flush makes accepted input decodable without ending the stream; flush boundaries can affect compressed size.

use mbrotli::io::FinishError;
use mbrotli::{Compressor, EncoderConfig, InputSize, Quality, StreamConfig};
use std::io::Write;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut encoder = Compressor::new(
        EncoderConfig::default().with_quality(Quality::Q5),
    )?;
    let input = "brotli ".repeat(1000);
    let stream = StreamConfig::from(InputSize::Exact(input.len() as u64));
    let mut writer = encoder.writer(Vec::new(), stream)?;
    for chunk in input.as_bytes().chunks(512) {
        writer.write_all(chunk)?;
    }
    let streamed = writer.finish().map_err(FinishError::into_error)?;
    assert_eq!(streamed, encoder.compress(input.as_bytes())?);
    Ok(())
}

All serial APIs emit identical bytes with the same configuration, dictionary, declared input size, flush boundaries, and continuation offset. To match a one-shot call, use InputSize::Exact(input.len() as u64), offset zero, and no explicit flushes. Caller chunk sizes and available SIMD backends do not change the output.

Parallel compression emits one stream from independent segments. Its output is deterministic across task counts for fixed segment settings, but can differ in both bytes and size from serial compression.

Dictionaries and format support

Unsupported quality/feature combinations return errors. A decoder needs the same external dictionaries to decode a stream that references them. The experimental API may change in a patch release.

[dependencies]
mbrotli = { version = "0.1", features = ["experimental"] }

The encoder is a port of Google's Brotli v1.2.0, pinned in the repository's brotli-ffi/vendor/brotli submodule at 028fb5a. Tests compare ordinary output with equivalent C streaming settings and decode it with C. Native C one-shot shortcuts and arbitrary C chunk schedules can produce different bytes. Custom static search and framing have separate compatibility checks. Declared windows above 30 bits lack an independent end-to-end decoder check in this repository.

Correctness

Each claim this crate makes about its bytes is checked by a machine against an oracle it does not own: the pinned C encoder for the bytes, the C decoder for validity, and the crate's own alternative paths for internal agreement. The correctness proof states every claim, names the oracle that checks it, and records one complete run of all of them, over both the standard and the experimental flow, with the commands to repeat it.

The proof also states its limits: a defect shared with Google Brotli v1.2.0 would not be detected, byte identity is claimed only for equivalent C streaming settings, and fuzzing is evidence for the inputs it executed.

Documentation

• User guide: configuration, buffers, streaming, and errors.
• Dictionaries and extended formats: preparation, limits, and experimental features.
• Parallel compression: task scheduling, input sources, and staging.
• Benchmark results: median comparisons and vertical speed/size charts by quality and dataset.
• Benchmarks and profiling: workloads and reproducible commands.
• Correctness proof: every claim, the oracle that checks it, and one complete run of all of them.
• Development: build, checks, coverage, and fuzzing.
• Architecture: implementation mechanics and diagrams.

Runnable examples:

cargo run --example compress
cargo run --release --example parallel -- INPUT OUTPUT
cargo doc --no-deps --open