Struct PreparedDictionary

Source

pub struct PreparedDictionary { /* private fields */ }

Expand description

An indexed set of prefix dictionaries, ready to compress against.

Built by DictionaryBuilder. Immutable and shareable: every method takes &self, so one dictionary can back any number of compressors at once, on any number of threads, with no synchronisation of this crate’s making.

§Examples

use mbrotli::dictionary::DictionaryBuilder;
use mbrotli::{Compressor, EncoderConfig, Quality};

let dictionary = DictionaryBuilder::new()
    .add_prefix(&b"HTTP/1.1 200 OK\r\nContent-Type: "[..])
    .build()?;

let mut encoder = Compressor::new(EncoderConfig::default().with_quality(Quality::Q5))?;
let payload = b"Content-Type: text/html; charset=utf-8";

let with = encoder.compress_with_dictionary(&dictionary, payload)?;
let without = encoder.compress(payload)?;
assert!(with.len() < without.len());

Sharing one dictionary between workers needs no lock, because nothing in it is mutable:

use mbrotli::dictionary::DictionaryBuilder;
use mbrotli::{Compressor, EncoderConfig, Quality};
use std::sync::Arc;

let dictionary = Arc::new(
    DictionaryBuilder::new()
        .add_prefix(&b"a shared prefix, indexed once"[..])
        .build()?,
);

let workers: Vec<_> = (0..4)
    .map(|worker| {
        let dictionary = Arc::clone(&dictionary);
        std::thread::spawn(move || {
            let config = EncoderConfig::default().with_quality(Quality::Q5);
            let mut encoder = Compressor::new(config).expect("a legal configuration");
            encoder
                .compress_with_dictionary(&dictionary, format!("worker {worker}").as_bytes())
                .expect("compression")
        })
    })
    .collect();

for worker in workers {
    assert!(!worker.join().expect("the worker finished").is_empty());
}

Implementations§

Source §

impl PreparedDictionary

Source

pub fn attachment_count(&self) -> usize

Returns how many prefix dictionaries were attached.

Zero for a dictionary containing only custom static words/transforms.

§Examples

use mbrotli::dictionary::DictionaryBuilder;

let dictionary = DictionaryBuilder::new()
    .add_prefix(&b"oldest"[..])
    .add_prefix(&b"newest"[..])
    .build()?;

assert_eq!(dictionary.attachment_count(), 2);

Source

pub fn source_bytes(&self) -> usize

Returns how many dictionary bytes the caller handed over.

Includes prefix bytes and custom word/transform source bytes, but not serialized framing or built-in dictionary bytes. The decoder must attach the same dictionaries in the same order.

§Examples

use mbrotli::dictionary::DictionaryBuilder;

let dictionary = DictionaryBuilder::new().add_prefix(&b"twelve bytes"[..]).build()?;

assert_eq!(dictionary.source_bytes(), 12);

Source

pub fn retained_bytes(&self) -> usize

Returns how much memory this dictionary occupies.

Counts the dictionary bytes and the prepared indexes together. Reading it needs no synchronisation, because there is none to take.

§Examples

use mbrotli::dictionary::DictionaryBuilder;

let dictionary = DictionaryBuilder::new()
    .add_prefix(&b"long enough to be worth indexing"[..])
    .build()?;

assert!(dictionary.retained_bytes() > dictionary.source_bytes());

Source

pub fn backward_distance(&self, offset: u64, max_backward: u64) -> Option<u64>

Returns the backward distance that addresses offset in the prefix.

RFC 9841 places the attached prefix immediately beyond the ordinary sliding window: distances 1..=max_backward are the stream’s own history, max_backward + 1 is the last prefix byte, and max_backward + prefix_length is the very first. Custom static source bytes do not contribute to prefix_length. max_backward is the largest distance the window can express at the position the copy starts from.

Returns None for an offset past the end of the prefix, and when the distance would not fit a u64. Nothing wraps.

§Examples

use mbrotli::dictionary::DictionaryBuilder;

let dictionary = DictionaryBuilder::new()
    .add_prefix(&b"oldest"[..])
    .add_prefix(&b"newest"[..])
    .build()?;

// Twelve prefix bytes: the last is one past the window, the first twelve past.
assert_eq!(dictionary.backward_distance(11, 1000), Some(1001));
assert_eq!(dictionary.backward_distance(0, 1000), Some(1012));
assert_eq!(dictionary.backward_distance(12, 1000), None);

Source

pub fn prefix_offset(&self, distance: u64, max_backward: u64) -> Option<u64>

Returns the prefix offset a backward distance addresses.

The inverse of PreparedDictionary::backward_distance, and the mapping a decoder performs. Returns None when the distance falls inside the ordinary sliding window or past the end of the prefix.

§Examples

use mbrotli::dictionary::DictionaryBuilder;

let dictionary = DictionaryBuilder::new()
    .add_prefix(&b"oldest"[..])
    .add_prefix(&b"newest"[..])
    .build()?;

assert_eq!(dictionary.prefix_offset(1001, 1000), Some(11));
assert_eq!(dictionary.prefix_offset(1012, 1000), Some(0));
// Inside the window, and past the whole prefix.
assert_eq!(dictionary.prefix_offset(1000, 1000), None);
assert_eq!(dictionary.prefix_offset(1013, 1000), None);