Trait Matcher 

pub trait Matcher {
    // Required methods
    fn get_next_space(&mut self) -> Vec<u8> ;
    fn get_last_space(&mut self) -> &[u8] ;
    fn commit_space(&mut self, space: Vec<u8>);
    fn skip_matching(&mut self);
    fn start_matching(
        &mut self,
        handle_sequence: impl for<'a> FnMut(Sequence<'a>),
    );
    fn reset(&mut self, level: CompressionLevel);
    fn window_size(&self) -> u64;

    // Provided methods
    fn fill_in_place(
        &mut self,
        _capacity: usize,
        _fill: &mut dyn FnMut(&mut Vec<u8>) -> (usize, bool),
    ) -> Option<(usize, bool)> { ... }
    fn uncommitted_input(&self) -> &[u8]  { ... }
    fn commit_filled(&mut self, _len: usize) { ... }
    fn reserve_for_frame(&mut self, _bytes: usize) { ... }
    fn skip_matching_with_hint(&mut self, _incompressible_hint: Option<bool>) { ... }
    fn set_source_size_hint(&mut self, _size: u64) { ... }
    fn set_dictionary_size_hint(&mut self, _sizes: DictionarySizes) { ... }
    fn clear_param_overrides(&mut self) { ... }
    fn prime_with_dictionary(
        &mut self,
        _dict_content: &[u8],
        _offset_hist: [u32; 3],
    ) { ... }
    fn dictionary_is_resident(&self) -> bool { ... }
    fn reapply_resident_dictionary(&mut self, _offset_hist: [u32; 3]) { ... }
    fn restore_primed_dictionary(&mut self, _level: CompressionLevel) -> bool { ... }
    fn capture_primed_dictionary(&mut self, _level: CompressionLevel) { ... }
    fn invalidate_primed_dictionary(&mut self) { ... }
    fn seed_dictionary_entropy(
        &mut self,
        _huff: Option<&HuffmanTable>,
        _ll: Option<&FSETable>,
        _ml: Option<&FSETable>,
        _of: Option<&FSETable>,
    ) { ... }
    fn supports_dictionary_priming(&self) -> bool { ... }
    fn block_samples_match_dict(&self, _block: &[u8]) -> bool { ... }
    fn heap_size(&self) -> usize { ... }
}

Trait used by the encoder that users can use to extend the matching facilities with their own algorithm making their own tradeoffs between runtime, memory usage and compression ratio

This trait operates on buffers that represent the chunks of data the matching algorithm wants to work on. Each one of these buffers is referred to as a space. One or more of these buffers represent the window the decoder will need to decode the data again.

This library asks the Matcher for a new buffer using get_next_space to allow reusing of allocated buffers when they are no longer part of the window of data that is being used for matching.

The library fills the buffer with data that is to be compressed and commits them back to the matcher using commit_space.

Then it will either call start_matching or, if the space is deemed not worth compressing, skip_matching is called.

This is repeated until no more data is left to be compressed.

Required Methods

fn get_next_space(&mut self) -> Vec<u8>

Get a space where we can put data to be matched on. Will be encoded as one block. The maximum allowed size is 128 kB.

fn get_last_space(&mut self) -> &[u8]

Get a reference to the last committed space

fn commit_space(&mut self, space: Vec<u8>)

Commit a space to the matcher so it can be matched against

fn skip_matching(&mut self)

Just process the data in the last committed space for future matching.

fn start_matching(&mut self, handle_sequence: impl for<'a> FnMut(Sequence<'a>))

Process the data in the last committed space for future matching AND generate matches for the data

fn reset(&mut self, level: CompressionLevel)

Reset this matcher so it can be used for the next new frame

fn window_size(&self) -> u64

The size of the window the decoder will need to execute all sequences produced by this matcher.

Must return a positive (non-zero) value; returning 0 causes StreamingEncoder to reject the first write with an invalid-input error (InvalidInput with std, Other with no_std).

Must remain stable for the lifetime of a frame. It may change only after reset() is called for the next frame (for example because the compression level changed).

Provided Methods

fn fill_in_place( &mut self, _capacity: usize, _fill: &mut dyn FnMut(&mut Vec<u8>) -> (usize, bool), ) -> Option<(usize, bool)>

Read the next block straight into the matcher’s own history buffer, skipping the scratch buffer that commit_space otherwise has to copy in.

fill is handed the history buffer with room reserved for capacity more bytes and returns (appended, eof). The bytes are readable through uncommitted_input but are NOT yet part of the match window: the caller chooses the block boundary (the pre-split pass needs the bytes to decide) and then calls commit_filled. Whatever is left over stays in the buffer and becomes the head of the next block, so a carried split remainder costs no copy.

Returns None if this matcher has no in-place ingest, which is the default: the caller then keeps the staged-copy path.

fn uncommitted_input(&self) -> &[u8]

Bytes ingested by fill_in_place that no block has claimed yet. Empty unless that hook is implemented.

fn commit_filled(&mut self, _len: usize)

Claim len bytes from the head of uncommitted_input as the next block.

fn reserve_for_frame(&mut self, _bytes: usize)

Size the ingest buffer for a frame of bytes up front, so filling it block by block doesn’t walk a doubling chain of reallocations. Clamped internally to the buffer’s eviction ceiling, so an over-long or absent hint can never reserve more than a bounded window. No-op unless fill_in_place is implemented.

fn skip_matching_with_hint(&mut self, _incompressible_hint: Option<bool>)

Hint-aware skip path used internally to thread a precomputed block incompressibility verdict to matcher backends.

Default implementation preserves backwards compatibility for external custom matchers by delegating to skip_matching.

fn set_source_size_hint(&mut self, _size: u64)

Provide a hint about the total uncompressed size for the next frame.

Implementations may use this to select smaller hash tables and windows for small inputs, matching the C zstd source-size-class behavior. Called before reset when the caller knows the input size (e.g. from pledged content size or file metadata).

The default implementation is a no-op for custom matchers and test stubs. The built-in runtime matcher (MatchGeneratorDriver) overrides this hook and applies the hint during level resolution.

fn set_dictionary_size_hint(&mut self, _sizes: DictionarySizes)

Hint the sizes of the dictionary that will be primed into the next frame. The built-in runtime matcher resolves the frame’s cParams from the dictionary’s CDict tier (upstream ZSTD_createCDict, keyed by the serialized size) and sizes its dictionary tables from the content. Default no-op for custom matchers and test stubs; consumed at the next reset.

fn clear_param_overrides(&mut self)

Drop any per-frame fine-grained parameter overrides installed via the public parameter API, reverting to plain level-based geometry at the next reset. Called by FrameCompressor::set_compression_level so switching back to a bare level after a customized frame does not keep the old overrides sticky. Default no-op for custom matchers.

fn prime_with_dictionary( &mut self, _dict_content: &[u8], _offset_hist: [u32; 3], )

Prime matcher state with dictionary history before compressing the next frame. Default implementation is a no-op for custom matchers that do not support this.

fn dictionary_is_resident(&self) -> bool

Whether the most recent reset re-borrowed a resident attach-mode dictionary (kept the dict bytes + cached index in place). When true the caller MUST skip Self::prime_with_dictionary and only reapply the offset history via Self::reapply_resident_dictionary.

fn reapply_resident_dictionary(&mut self, _offset_hist: [u32; 3])

Reapply the dictionary’s offset history to a re-borrowed frame — the cheap tail of priming, without the dict commit / re-index. Default no-op.

fn restore_primed_dictionary(&mut self, _level: CompressionLevel) -> bool

CDict-equivalent fast path for repeated frames sharing one dictionary. Restore the matcher state captured by Self::capture_primed_dictionary at the SAME level (a table copy) instead of re-running Self::prime_with_dictionary (which re-hashes every dictionary position). Returns true when a matching snapshot was restored; false (the default) means the caller must prime then capture.

fn capture_primed_dictionary(&mut self, _level: CompressionLevel)

Snapshot the post-prime matcher state for the given level so later frames can Self::restore_primed_dictionary it. Default no-op.

fn invalidate_primed_dictionary(&mut self)

Drop any captured prime snapshot (dictionary or level changed). Default no-op.

fn seed_dictionary_entropy( &mut self, _huff: Option<&HuffmanTable>, _ll: Option<&FSETable>, _ml: Option<&FSETable>, _of: Option<&FSETable>, )

Seed matcher cost model with dictionary entropy tables before the next frame. Default implementation is a no-op for custom matchers.

fn supports_dictionary_priming(&self) -> bool

Returns whether this matcher can consume dictionary priming state and produce dictionary-dependent sequences. Defaults to false for custom matchers.

fn block_samples_match_dict(&self, _block: &[u8]) -> bool

Whether a sample of block hashes to a match in an attached dictionary. The raw-fast-path uses this to avoid skipping the scan on a block that looks incompressible but compresses against the dictionary (an external match the block’s own content cannot reveal). Defaults to false for custom matchers (and the no-dict case), leaving the content-only verdict.

fn heap_size(&self) -> usize

Heap bytes this matcher’s allocations hold (tables, history, scratch), excluding the inline struct itself. Lets a context report its true footprint via ZSTD_sizeof_CCtx. Defaults to 0 for custom matchers.

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl Matcher for MatchGeneratorDriver