mbrotli

Brotli compression and decompression in safe Rust.

A port of Google's Brotli encoder with a Rust-native API, plus a native Rust decoder. Reuse working memory between payloads, integrate directly with std::io::Read and std::io::Write, drive either codec incrementally, and opt into caller-scheduled parallel compression when a workload benefits from it.

Quick start · Benchmarks · Read / Write · API guide · Compatibility · Changelog

Why mbrotli?

• Safe implementation, not a C wrapper. Production code in this crate forbids unsafe. SIMD-accelerated compression uses fearless_simd.
• Reference-compatible encoding. Qualities 0–11, with byte-for-byte comparisons against Google Brotli under equivalent streaming settings.
• Native Rust I/O. Compress and decompress through one-shot Vec/slice APIs, std::io::Read / Write adapters, or explicit incremental sessions.
• Reusable state. Keep codec workspaces and destination buffers across requests instead of rebuilding them for every payload.
• You control the threads. Run parallel compression with scoped threads, Rayon, or your own scheduler. The library does not create a thread pool.

Performance

Compression performance is generally close to or faster than Google C Brotli, depending on the workload and quality setting.

Recorded 2026-09-07 · Intel Core i7-13700KF · WSL2 · window 22 · cold serial APIs. Each value is the median of per-dataset ratios across eight equally weighted datasets, including empty and tiny inputs. A speed ratio above 1× is faster than C; a size ratio below 1× is smaller. These are compression results; the separate decoder comparison follows below.

The benchmark includes encoder construction, allocation, compression, and disposal. Results apply to the recorded revision and machine.

Per-dataset results and methodology · Raw measurements · Reproduce the benchmarks

Recorded 2026-09-10 · Intel Core i7-13700KF · WSL2 · window 22 · cold serial APIs. Google C, mbrotli, Rust brotli and Burli decode identical C-generated streams at source qualities 0–11. SIMD Brotli shares Rust brotli's decoder and is omitted; Burli decodes every source quality. All 384 cases restore the original bytes.

Across the eight equally weighted inputs, the median speed / Burli is 1.013× (previously 0.967× in a separate run), and the median speed / Google C ranges from 3.1× to 3.7× by source quality. Empty and tiny inputs retain the same weight as larger datasets. The chart shows per-quality medians of C-time/decoder-time ratios; above 1× is faster than C. Throughput in the quality pages counts restored bytes. Construction, allocation, decode and disposal are timed. C receives the known output capacity; Rust brotli includes its native 4 KiB I/O adapter. These are cold API measurements on the recorded machine, with results varying by workload and source quality.

Per-quality and dataset results · Raw measurements · Methodology and limits

Quick start

Requires Rust 1.89 or later. Add it to your project:

[dependencies]
mbrotli = "0.3"

use mbrotli::{Compressor, EncoderConfig, Quality};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let input = "brotli ".repeat(1000);
    let config = EncoderConfig::default().with_quality(Quality::Q5);
    let mut encoder = Compressor::new(config)?;

    let compressed = encoder.compress(input.as_bytes())?;
    println!("{} -> {} bytes", input.len(), compressed.len());
    Ok(())
}

Set the quality explicitly for your workload. EncoderConfig::default() uses quality 11, the most expensive compression search.

Choose your API

Compression and decompression expose the same core I/O shapes. Pick the shape that matches how your application already moves bytes; parallel compression is a separate execution strategy, not another streaming API.

Reuse memory between payloads

Compressor and Decompressor own reusable working state. Keep the codec and your output buffer alive across operations when allocation reuse matters.

use mbrotli::{Compressor, EncoderConfig, Quality};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut encoder = Compressor::new(
        EncoderConfig::default().with_quality(Quality::Q5),
    )?;
    let mut output = Vec::new();

    for input in [b"first payload".as_slice(), b"second payload".as_slice()] {
        output.clear(); // Keep the allocation; compress_into appends.
        let written = encoder.compress_into(input, &mut output)?;
        assert_eq!(written, 0..output.len());
    }
    Ok(())
}

Read and Write streaming

Both codecs provide synchronous adapters for the standard Rust I/O traits. The two adapter shapes are complementary:

• reader(...) wraps an input Read and exposes transformed bytes through Read.
• writer(...) wraps an output Write and accepts source bytes through Write.

That means you can plug Brotli into an existing pull-based or push-based pipeline without first collecting the whole payload in memory.

Compression I/O

Compressor::reader consumes uncompressed bytes from a Read source and yields compressed bytes. Compressor::writer accepts uncompressed bytes and writes compressed bytes to its sink. Encoder writers must be explicitly finished; dropping one abandons the stream.

use mbrotli::{Compressor, EncoderConfig, InputSize, Quality};
use std::io::{Read, Write};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let payload = b"streamed payload";
    // Pull model: read compressed bytes from an uncompressed source.
    let mut encoder = Compressor::new(
        EncoderConfig::default().with_quality(Quality::Q5),
    )?;
    let stream = InputSize::Exact(payload.len() as u64).into();
    let mut reader = encoder.reader(&payload[..], stream)?;
    let mut compressed_from_reader = Vec::new();
    reader.read_to_end(&mut compressed_from_reader)?;

    // Push model: write uncompressed bytes into a compressed sink.
    let mut encoder = Compressor::new(
        EncoderConfig::default().with_quality(Quality::Q5),
    )?;
    let stream = InputSize::Exact(payload.len() as u64).into();
    let mut writer = encoder.writer(Vec::new(), stream)?;
    writer.write_all(payload)?;
    let compressed_from_writer = writer
        .finish()
        .map_err(mbrotli::io::FinishError::into_error)?;

    assert_eq!(compressed_from_reader, compressed_from_writer);
    Ok(())
}

flush() makes accepted input decodable without ending the encoder stream, and flush boundaries can affect compressed bytes. Use finish() when the stream is complete.

Decompression I/O

Decompressor::reader consumes a compressed Read source and yields the decompressed payload. Decompressor::writer accepts compressed bytes and writes the decompressed payload to its sink.

use mbrotli::{DecodeStreamConfig, DecoderConfig, Decompressor};
use std::io::{Read, Write};

fn decode_with_reader(compressed: &[u8]) -> Result<Vec<u8>, Box<dyn std::error::Error>> {
    let mut decoder = Decompressor::new(DecoderConfig::default())?;
    let mut reader = decoder.reader(compressed, DecodeStreamConfig::default())?;
    let mut output = Vec::new();
    reader.read_to_end(&mut output)?;
    Ok(output)
}

fn decode_with_writer(compressed: &[u8]) -> Result<Vec<u8>, Box<dyn std::error::Error>> {
    let mut decoder = Decompressor::new(DecoderConfig::default())?;
    let mut writer = decoder.writer(Vec::new(), DecodeStreamConfig::default())?;
    writer.write_all(compressed)?;
    Ok(writer
        .finish()
        .map_err(mbrotli::io::FinishError::into_error)?)
}

The decoder adapters use bounded internal buffering. Reader read-ahead can be recovered with into_parts(), while decoder-writer finalization reports truncated or invalid input instead of silently accepting an incomplete stream.

Parallel compression

Parallel compression is independent of the Read/Write adapters. It changes how one compression job is scheduled: mbrotli splits the input into segments, exposes work items to the caller, and assembles the completed segments back into one Brotli stream. The library does not create or own a thread pool.

use mbrotli::compressor::parallel::{
    BatchConfig, ParallelCompressor, ParallelConfig, TaskCount,
};
use mbrotli::{EncoderConfig, Quality};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let input = vec![b'a'; 8 << 20];
    let mut encoder = ParallelCompressor::new(
        EncoderConfig::default().with_quality(Quality::Q5),
        ParallelConfig::default(),
    )?;
    let mut batch = encoder.prepare_slice(
        &input,
        BatchConfig::auto(TaskCount::available()?),
    )?;
    let tasks = batch.take_tasks()?;

    std::thread::scope(|scope| {
        for task in tasks {
            scope.spawn(move || task.run());
        }
    });

    let mut output = Vec::new();
    batch.finish_into(&mut output)?;
    println!("{} -> {} bytes", input.len(), output.len());
    Ok(())
}

For fixed segment settings, parallel output is deterministic across task counts, but it can differ in bytes and size from serial compression. The example stages compressed segments in memory; see the parallel guide for budgets, disk staging, file input, and other executors.

Native decompression

Decompressor provides reusable Vec/slice APIs, incremental sessions, and synchronous reader/writer adapters. Vec appends are rolled back on failure. The decoder specializes command loops and history copies for the selected CPU backend using safe fearless_simd abstractions.

Configure limits for untrusted input. Numeric budgets are unlimited by default. This example accepts standard windows and sets explicit input, output, and workspace budgets; choose limits appropriate for your application.

use mbrotli::{DecodeLimits, DecoderConfig, Decompressor, WindowLimit};

fn decode_payload(input: &[u8]) -> Result<Vec<u8>, Box<dyn std::error::Error>> {
    let limits = DecodeLimits::default()
        .with_max_input_bytes(Some(1 << 20))
        .with_max_output_bytes(Some(8 << 20))
        .with_max_workspace_bytes(Some(32 << 20));
    let config = DecoderConfig::default()
        .with_window_limit(WindowLimit::standard(24)?)
        .with_limits(limits);
    let mut decoder = Decompressor::new(config)?;

    Ok(decoder.decompress(input)?)
}

The workspace budget excludes caller-owned output and borrowed dictionaries; it is not a total process-memory limit. Retain the decoder across calls when reuse matters. See decoder configuration and semantics and compatibility evidence.

Select only the codecs you need

The default feature set is std, compression, and decompression. Disable default features to select one codec or use no_std with alloc. For an alloc-backed decoder:

[dependencies]
mbrotli = { version = "0.3", default-features = false, features = ["no_std", "decompression"] }

Add "compression" for both codecs, or use "std" instead of "no_std" for standard I/O support. no_std requires a global allocator; it excludes I/O adapters, parallel compression, experimental framing, and profiling, and uses compile-time SIMD selection. Cargo features are additive: another dependency can re-enable a codec or std. Leave std and hotpath* disabled throughout the dependency graph for a std-free build.

Compatibility

The encoder is ported from Google Brotli v1.2.0 (028fb5a); the test reference in brotli-ffi/vendor/brotli is pinned to upstream master at 4508218e (2026-09-01), which still reports version 1.2.0. Ordinary encoding at qualities 0–11 and windows 10–24 is compared byte-for-byte with equivalent C streaming settings.

That comparison requires matching configuration, dictionary, declared input size, flush boundaries, and continuation offset. C one-shot shortcuts or arbitrary C chunk schedules can produce different bytes. Within mbrotli's serial APIs, matching those settings preserves output across input chunk sizes, SIMD backends, and buffer reuse. This is format compatibility, not a drop-in replacement for another crate's Rust API.

Unsupported combinations return errors. External dictionary references require the same dictionaries at the decoder. Large Window and shared-dictionary streams require a decoder that supports the corresponding extension. C-based end-to-end validation of Large Window encoding covers windows up to 30 bits; wider declarations use separate checks and do not have the same independent C-decoder evidence.

Enable the experimental Cargo feature to use the gated formats. Their API may change in a patch release, and custom static encoding and framing have separate validation from the ordinary encoder's byte-identity checks. See the dictionary and format guide.

Validation

The repository includes differential tests against the pinned C encoder and decoder, cross-API and cross-backend checks, AFL++ fuzz targets, Miri checks, and AddressSanitizer workflows. The most recent campaign ran 59 AFL++ workers over both surfaces and both feature builds for eight hours — 914.9 million executions, no crash, and one fixed defect in how the high-quality match finder reserved its forest. Read the encoder verification report and the separate native decoder report for tested configurations, dated results, reproduction commands, and limitations. Testing and fuzzing are evidence, not formal verification.

The crate enforces #![cfg_attr(not(test), forbid(unsafe_code))]. This describes mbrotli's production implementation, not every transitive dependency: SIMD intrinsics are encapsulated by fearless_simd, and differential tests use a C FFI reference.

Documentation and contributing

API reference · User guide · Dictionaries · Parallel compression · Architecture

Bug reports with a minimal input, codec settings, and reproduction steps are welcome. For performance reports, include the CPU, compiler, feature set, and workload. The development guide covers local checks, coverage, and fuzzing.

License

MIT. The encoder builds on the work of the Google Brotli project.