Struct FrameDecoder 

pub struct FrameDecoder { /* private fields */ }

Low level Zstandard decoder that can be used to decompress frames with fine control over when and how many bytes are decoded.

This decoder is able to decode frames only partially and gives control over how many bytes/blocks will be decoded at a time (so you don’t have to decode a 10GB file into memory all at once). It reads bytes as needed from a provided source and can be read from to collect partial results.

If you want to just read the whole frame with an io::Read without having to deal with manually calling FrameDecoder::decode_blocks you can use the provided crate::decoding::StreamingDecoder wich wraps this FrameDecoder.

Workflow is as follows:

use structured_zstd::decoding::BlockDecodingStrategy;

use std::io::{Read, Write};

// no_std environments can use the crate's own Read traits
use structured_zstd::io::{Read, Write};

fn decode_this(mut file: impl Read) {
    //Create a new decoder
    let mut frame_dec = structured_zstd::decoding::FrameDecoder::new();
    let mut result = Vec::new();

    // Use reset or init to make the decoder ready to decode the frame from the io::Read
    frame_dec.reset(&mut file).unwrap();

    // Loop until the frame has been decoded completely
    while !frame_dec.is_finished() {
        // decode (roughly) batch_size many bytes
        frame_dec.decode_blocks(&mut file, BlockDecodingStrategy::UptoBytes(1024)).unwrap();

        // read from the decoder to collect bytes from the internal buffer
        let bytes_read = frame_dec.read(result.as_mut_slice()).unwrap();

        // then do something with it
        do_something(&result[0..bytes_read]);
    }

    // handle the last chunk of data
    while frame_dec.can_collect() > 0 {
        let x = frame_dec.read(result.as_mut_slice()).unwrap();

        do_something(&result[0..x]);
    }
}

fn do_something(data: &[u8]) {
    std::io::stdout().write_all(data).unwrap();
}

Implementations

impl FrameDecoder

pub fn new() -> FrameDecoder

This will create a new decoder without allocating anything yet. init()/reset() will allocate all needed buffers if it is the first time this decoder is used else they just reset these buffers with not further allocations

pub fn workspace_size(&self) -> usize

Heap bytes currently held by the decoder’s lazily-grown workspace: the decode-window buffer plus the per-block literal/content buffers and the entropy tables. Returns 0 before the first frame is initialised (no workspace allocated yet). The window allocation dominates and grows with the frame’s window size; this is the value to track for decode-time memory pressure, mirroring the workspace term of upstream ZSTD_sizeof_DCtx. Shared dictionaries (ref-counted handles) are not counted, matching upstream excluding refDDict memory.

pub fn set_content_checksum(&mut self, mode: ContentChecksum)

Select how the frame’s optional content checksum is handled (compute, expose, verify, or skip). See ContentChecksum. Default ContentChecksum::EmitOnly. Takes effect on the next decode; safe to call between frames on a reused decoder.

pub fn enable_per_block_checksums(&mut self)

Available on crate features hash and lsm only.

Opt in to per-block XXH64 verification during decode. Default off; zero cost when disabled. Each block’s decompressed bytes are XXH64-hashed (low 32 bits) and appended to Self::computed_block_checksums as the decode progresses. Callers compare the captured digests against externally-stored expected values (e.g. from a per-block sidecar in the containing application protocol).

Behind all(feature = "lsm", feature = "hash") — the XXH64 primitive lives behind the hash feature, so this method only compiles when both are enabled.

pub fn computed_block_checksums(&self) -> &[u32]

Available on crate features hash and lsm only.

Per-block XXH64 (low 32 bits) digests captured during the current frame’s decode. Empty unless Self::enable_per_block_checksums was called before Self::decode_all / Self::reset.

Reset at the start of every new frame.

Behind all(feature = "lsm", feature = "hash").

pub fn expect_dict_id(&mut self, expected: Option<u32>)

Available on crate feature lsm only.

Pin the expected Dictionary_ID for the next frame.

When expected is set, Self::init / Self::reset validate it against the parsed frame header BEFORE any block decode work runs. A mismatch returns crate::decoding::errors::FrameDecoderError::UnexpectedDictId before any block decode and before any output is produced. Scratch buffer allocation / reservation for the decode pipeline happens during frame-header parsing, which is already complete when this validation fires — the cost of scratch sizing is paid even on a mismatched header. The guarantee is “no block decode, no XXH64 init, no partial output”, not “zero allocation”.

Some(0) is treated as “no dictionary expected”: a frame whose header omits the optional Dictionary_ID field (flag value 0) passes the check; a frame that carries an explicit non-zero id fails.

None (default) disables the check.

Primary use case: post-AEAD-decrypt sanity check in wire-format consumers (e.g. lsm-tree’s encrypted block format pins the dict_id baked into the AAD against the inner zstd frame’s dict_id to defeat dict-substitution attacks).

NOT a replacement for AEAD authentication. NOT the same semantic as upstream zstd ZSTD_d_windowLogMax (which is a ceiling-style limit, separate concern).

pub fn expect_window_descriptor(&mut self, expected: Option<u8>)

Available on crate feature lsm only.

Pin the expected raw Window_Descriptor byte (RFC 8878 §3.1.1.1.2 layout: (exp << 3) | mantissa) for the next frame.

When expected is set, Self::init / Self::reset validate it against the parsed frame header BEFORE any block decode work runs. A mismatch returns crate::decoding::errors::FrameDecoderError::UnexpectedWindowDescriptor.

Single-segment frames omit the Window_Descriptor byte from the wire entirely. Setting an expectation while receiving a single-segment frame fails the check with found: None — there is no on-wire byte to match against, which is reported explicitly rather than silently passing.

None (default) disables the check.

Byte-exact equality, NOT a ceiling. Upstream zstd ZSTD_d_windowLogMax is a separate ceiling-style limit available through the C FFI surface; this method is for strict equality validation against a pinned expectation (e.g. lsm-tree’s wire format pins the window descriptor from the AAD to defeat decompression-bomb-swap attacks).

pub fn set_magicless(&mut self, magicless: bool)

Enable or disable magicless frame format (ZSTD_f_zstd1_magicless). When set to true, subsequent Self::init / Self::reset calls expect the frame header to begin directly with the frame-header descriptor — no 4-byte magic number prefix. Default false. Must match the encoder’s magicless setting; the format is unambiguous only when the caller knows it out-of-band.

Note: magicless mode also disables skippable-frame detection. The 0x184D2A50..=0x184D2A5F skippable-frame magic range is only recognised when the 4-byte magic prefix is consumed, so decode_all / init / reset will treat a skippable frame at the head of a magicless stream as a malformed frame header (bad descriptor / window-size error) instead of skipping it. Mixed-format streams that interleave skippable frames must be pre-split by the caller; set_magicless(true) is only safe when the entire stream is known to be magicless zstd frames.

pub fn set_max_window_size( &mut self, size: u64, ) -> Result<(), FrameDecoderError>

Set the largest window a frame may declare when this decoder holds the window itself: Self::init, Self::reset, Self::decode_blocks and the streaming paths. Upstream zstd’s ZSTD_DCtx_setMaxWindowSize (and ZSTD_d_windowLogMax, as 1 << log). The default, MAXIMUM_ALLOWED_WINDOW_SIZE, is upstream’s; raise it to decode frames from zstd --long=28 and wider, lower it to cap the memory untrusted input may claim. Decoding into a caller’s buffer (Self::decode_all and its siblings) holds no window, so only MAX_DECODER_WINDOW_SIZE bounds it there, as upstream’s ZSTD_decompressDCtx.

A ceiling lowered below the window of a frame already started binds that frame too: its next decode step fails with FrameDecoderError::WindowSizeTooBig instead of reserving the window, until the ceiling is raised again or the next frame starts.

Errors

FrameDecoderError::WindowCeilingOutOfRange when size lies outside 1 KiB (the smallest window a frame declares) ..= MAX_DECODER_WINDOW_SIZE; the ceiling in force is kept.

Examples

use structured_zstd::decoding::FrameDecoder;

let mut decoder = FrameDecoder::new();
decoder.set_max_window_size(1 << 30).unwrap();
assert_eq!(decoder.max_window_size(), 1 << 30);
assert!(decoder.set_max_window_size(1 << 9).is_err());

pub fn max_window_size(&self) -> u64

The window ceiling in force; see Self::set_max_window_size.

pub fn set_max_block_size(&mut self, size: Option<u32>)

Refuse any block that holds more than size bytes, compressed or decoded, from the next frame on: for input known to come from an encoder that keeps its blocks that small (upstream’s ZSTD_d_maxBlockSize, the decoding side of ZSTD_c_maxBlockSize). size is clamped to 1 KiB ..= 128 KiB; None restores the format’s 128 KiB. A frame whose window is smaller still holds its blocks to the window. Applies on every path, including decoding into a caller’s buffer, as upstream’s one-shot honours it too.

use structured_zstd::decoding::FrameDecoder;
use structured_zstd::encoding::{compress_to_vec, CompressionLevel};

let frame = compress_to_vec(&[7u8; 4096][..], CompressionLevel::Fastest);
let mut out = [0u8; 4096];
let mut decoder = FrameDecoder::new();
decoder.set_max_block_size(Some(1024));
// One RLE block of 4 KiB holds more than 1 KiB.
assert!(decoder.decode_all(&frame, &mut out).is_err());
decoder.set_max_block_size(None);
assert_eq!(decoder.decode_all(&frame, &mut out).unwrap(), 4096);

pub fn awaits_checksum(&self) -> bool

Whether every block of the frame is decoded and only its trailing content checksum is still to be read, as Self::read_content_checksum takes it.

pub fn input_needed(&self, pending: &[u8]) -> usize

How many bytes from the start of pending the current frame’s next step needs in hand: a block header, a whole block, or the trailing checksum; 0 once the frame wants no more input. Upstream zstd’s ZSTD_nextSrcSizeToDecompress. A block stating more than the frame’s block maximum needs only its header, whose decode then reports it.

A caller whose input arrives in pieces holds them until this many are in hand, since Self::decode_from_to moves only on whole steps.

use structured_zstd::decoding::FrameDecoder;
use structured_zstd::encoding::{compress_slice_to_vec, CompressionLevel};
let frame = compress_slice_to_vec(&[7u8; 64], CompressionLevel::Default);
let mut source = frame.as_slice();
let mut decoder = FrameDecoder::new();
decoder.reset(&mut source).unwrap();
// One byte of a block header: the header itself is the next step.
assert_eq!(decoder.input_needed(&source[..1]), 3);
// The rest of the frame holds the whole first block.
assert!(decoder.input_needed(source) <= source.len());

pub fn decode_block( &mut self, header: [u8; 3], content: &[u8], target: &mut [u8], ) -> Result<usize, FrameDecoderError>

Decode one block of the frame begun by Self::reset, its 3-byte header and its content given apart, and hand all of its output to target at once (upstream ZSTD_decompressContinue on a block). The output also stays in the window for the blocks after it, so target may be reused for the next one. Returns the bytes written.

A frame whose last block is decoded takes no more blocks: its checksum, when it has one, goes to Self::read_content_checksum, and the next frame starts with Self::reset.

Errors

As a block Self::decode_from_to decodes, plus FrameDecoderError::TargetTooSmall when target cannot take the block’s output, and FrameDecoderError::NotYetInitialized when no frame is waiting for a block. A block refused before it is decoded (a bad header, a block past the maximum, content of the wrong length) leaves the frame as it was, as upstream checks the size before it changes any state; one that fails while decoding, or whose output target cannot take, ends the frame.

use structured_zstd::decoding::FrameDecoder;
use structured_zstd::encoding::{compress_to_vec, CompressionLevel};

let frame = compress_to_vec(&b"one block at a time"[..], CompressionLevel::Fastest);
let mut source = frame.as_slice();
let mut decoder = FrameDecoder::new();
decoder.reset(&mut source).unwrap();
let (header, rest) = source.split_first_chunk::<3>().unwrap();
let mut out = [0u8; 64];
let n = decoder.decode_block(*header, rest, &mut out).unwrap();
assert_eq!(&out[..n], b"one block at a time");
assert!(decoder.is_finished());

pub fn block_content_len( &self, header: [u8; 3], ) -> Result<(usize, bool), FrameDecoderError>

How many content bytes follow the block header header of the current frame (its size, or one byte for an RLE block), and whether it is the frame’s last block; the header is checked against the frame’s block maximum (upstream ZSTD_getcBlockSize). What Self::decode_block wants alongside it.

Errors

A reserved block type or a block past the maximum, as Self::decode_block reports them; FrameDecoderError::NotYetInitialized when no frame is waiting for a block.

pub fn read_content_checksum( &mut self, checksum: [u8; 4], ) -> Result<(), FrameDecoderError>

Take the 4-byte content checksum that ends the frame whose blocks Self::decode_block decoded, and check it in ContentChecksum::Verify mode.

Errors

FrameDecoderError::ChecksumMismatch when it disagrees with the output; FrameDecoderError::NotYetInitialized when the frame is not waiting for its checksum.

pub fn begin_blocks( &mut self, dict: Option<&DictionaryHandle>, ) -> Result<(), FrameDecoderError>

Start a run of blocks outside any frame (upstream ZSTD_decompressBegin before ZSTD_decompressBlock and ZSTD_insertBlock): each block decodes against the output before it, as far back as the window ceiling, and against dict when given. Compressed blocks go to Self::decode_block with a header saying so, and output the caller already holds to Self::insert_block.

use structured_zstd::decoding::FrameDecoder;

let mut decoder = FrameDecoder::new();
decoder.begin_blocks(None).unwrap();
decoder.insert_block(b"history the next block can match").unwrap();

pub fn insert_block(&mut self, content: &[u8]) -> Result<(), FrameDecoderError>

Take content, output the caller already holds, into the history of the run Self::begin_blocks started (upstream ZSTD_insertBlock), for a block the encoder could not compress and the caller stored as is.

Errors

FrameDecoderError::NotYetInitialized when no run of blocks is open.

pub fn init(&mut self, source: impl Read) -> Result<(), FrameDecoderError>

init() will allocate all needed buffers if it is the first time this decoder is used else they just reset these buffers with not further allocations

Note that all bytes currently in the decodebuffer from any previous frame will be lost. Collect them with collect()/collect_to_writer()

equivalent to reset()

pub fn init_with_dict_handle( &mut self, source: impl Read, dict: &DictionaryHandle, ) -> Result<(), FrameDecoderError>

Initialize the decoder for a new frame using a pre-parsed dictionary handle.

If the frame header has a dictionary ID, this validates it against dict.id() and returns FrameDecoderError::DictIdMismatch on mismatch.

If the header omits the optional dictionary ID, this still applies the provided dictionary handle.

Warning

This method always applies dict unless the frame header contains a non-matching dictionary ID. Callers must only use this API when they already know the frame was encoded with the provided dictionary, even if the frame header omits the dictionary ID or encodes an explicit dictionary ID of 0.

Passing a dictionary for a frame that was not encoded with it can silently corrupt the decoded output.

pub fn reset(&mut self, source: impl Read) -> Result<(), FrameDecoderError>

reset() will allocate all needed buffers if it is the first time this decoder is used else they just reset these buffers with not further allocations

Note that all bytes currently in the decodebuffer from any previous frame will be lost. Collect them with collect()/collect_to_writer()

equivalent to init()

pub fn reset_with_dict_handle( &mut self, source: impl Read, dict: &DictionaryHandle, ) -> Result<(), FrameDecoderError>

Reset this decoder for a new frame using a pre-parsed dictionary handle.

If the frame header has a dictionary ID, this validates it against dict.id() and returns FrameDecoderError::DictIdMismatch on mismatch.

If the header omits the optional dictionary ID, this still applies the provided dictionary handle.

Warning

This method always applies dict unless the frame header contains a non-matching dictionary ID. Callers must only use this API when they already know the frame was encoded with the provided dictionary, even if the frame header omits the dictionary ID or encodes an explicit dictionary ID of 0.

Passing a dictionary for a frame that was not encoded with it can silently corrupt the decoded output.

pub fn reset_with_dict_handles( &mut self, source: impl Read, dicts: &[DictionaryHandle], ) -> Result<(), FrameDecoderError>

reset_with_dict_handle with a choice of dictionaries: the frame decodes with the one its header names, and a frame that names none takes the last of dicts. A frame naming a dictionary missing from dicts fails with FrameDecoderError::DictIdMismatch. An empty dicts decodes without a dictionary, as Self::reset does.

use structured_zstd::decoding::{Dictionary, DictionaryHandle, FrameDecoder};

let first = DictionaryHandle::from_dictionary(
    Dictionary::from_raw_content(1, b"first dictionary content".to_vec()).unwrap(),
);
let second = DictionaryHandle::from_dictionary(
    Dictionary::from_raw_content(2, b"second dictionary content".to_vec()).unwrap(),
);
// A frame without a dictionary ID: the last dictionary applies.
let frame = structured_zstd::encoding::compress_to_vec(
    &b"payload"[..],
    structured_zstd::encoding::CompressionLevel::Fastest,
);
let mut decoder = FrameDecoder::new();
decoder
    .reset_with_dict_handles(&frame[..], &[first, second])
    .unwrap();

pub fn add_dict(&mut self, dict: Dictionary) -> Result<(), FrameDecoderError>

Add a dictionary that can be selected dynamically by frame dictionary ID.

Returns FrameDecoderError::DictAlreadyRegistered if the ID is already registered (either as owned or shared).

pub fn add_dict_from_bytes( &mut self, raw_dictionary: &[u8], ) -> Result<(), FrameDecoderError>

Parse and add a serialized dictionary blob.

pub fn add_dict_handle( &mut self, dict: DictionaryHandle, ) -> Result<(), FrameDecoderError>

Available on target_has_atomic=ptr only.

Add a pre-parsed dictionary handle for reuse across decoders.

This API is available on targets with pointer-width atomics (target_has_atomic = "ptr").

Returns FrameDecoderError::DictAlreadyRegistered if the ID is already registered (either as owned or shared).

pub fn force_dict(&mut self, dict_id: u32) -> Result<(), FrameDecoderError>

pub fn content_size(&self) -> u64

Returns how many bytes the frame contains after decompression

pub fn get_checksum_from_data(&self) -> Option<u32>

Returns the checksum that was read from the data. Only available after all bytes have been read. It is the last 4 bytes of a zstd-frame

pub fn get_calculated_checksum(&self) -> Option<u32>

Available on crate feature hash only.

Returns the checksum that was calculated while decoding. Only a sensible value after all decoded bytes have been collected/read from the FrameDecoder. Returns None when the frame header has content_checksum_flag = 0: no hash is computed for such frames (the post-decode XXH64 pass was a 63 % decode-wall hotspot on flag-off frames; skipping it when the frame format declares no trailing digest avoids that wasted work).

pub fn verify_content_size(&self) -> Result<(), FrameDecoderError>

Check that a finished frame produced exactly the Frame_Content_Size its header declared (RFC 8878 3.1.1.1.4), returning FrameDecoderError::FrameContentSizeMismatch otherwise. No-op for a frame that declares no size or has not decoded its last block yet.

decode_all, decode_from_to and the streaming reader call this automatically. Callers driving decode_blocks directly invoke it once the frame is finished, as upstream zstd checks at its last block.

Examples

use structured_zstd::decoding::{BlockDecodingStrategy, FrameDecoder};
use structured_zstd::encoding::{compress_to_vec, CompressionLevel};

let frame = compress_to_vec(&b"declared and delivered"[..], CompressionLevel::Fastest);
let mut source = frame.as_slice();
let mut decoder = FrameDecoder::new();
decoder.init(&mut source).unwrap();
decoder.decode_blocks(&mut source, BlockDecodingStrategy::All).unwrap();
assert!(decoder.is_finished());
decoder.verify_content_size().unwrap();
assert_eq!(decoder.collect().unwrap(), b"declared and delivered");

pub fn verify_content_checksum(&self) -> Result<(), FrameDecoderError>

Available on crate feature hash only.

Compare the frame’s stored content checksum against the digest the decoder computed, returning FrameDecoderError::ChecksumMismatch on disagreement. No-op unless the mode is ContentChecksum::Verify and the frame carries a trailing checksum.

decode_all and the streaming reader call this automatically. Callers driving decode_blocks directly invoke it themselves once per frame, after the frame is fully decoded AND fully drained (e.g. via collect), so both the stored value and the running digest are final.

pub fn bytes_read_from_source(&self) -> u64

Counter for how many bytes have been consumed while decoding the frame

pub fn dictionary(&self) -> Option<&DictionaryHandle>

The dictionary the decoder keeps from the last frame it started with one, which Self::workspace_size leaves out because the handle is shared: an owner adding up its footprint counts it with the other handles it holds (see DictionaryHandle::exclusive_heap_size).

Examples

use structured_zstd::decoding::FrameDecoder;

assert!(FrameDecoder::new().dictionary().is_none());

pub fn is_finished(&self) -> bool

Whether the current frames last block has been decoded yet If this returns true you can call the drain* functions to get all content (the read() function will drain automatically if this returns true)

pub fn blocks_decoded(&self) -> usize

Counter for how many blocks have already been decoded

pub fn decode_blocks( &mut self, source: impl Read, strat: BlockDecodingStrategy, ) -> Result<bool, FrameDecoderError>

Decodes blocks from a reader. It requires that the framedecoder has been initialized first. The Strategy influences how many blocks will be decoded before the function returns This is important if you want to manage memory consumption carefully. If you don’t care about that you can just choose the strategy “All” and have all blocks of the frame decoded into the buffer

pub fn decode_blocks_partial( &mut self, source: impl Read, start_block: u32, end_block: u32, resume: Option<ResumeInput<'_>>, emit_resume: bool, ) -> Result<PartialDecode, FrameDecoderError>

Available on crate feature lsm only.

Decode the inner blocks [start_block, end_block) of the current frame and return their decompressed bytes as one contiguous buffer.

Serves two consumer needs with one call:

• Range-query performance: decode only the inner zstd blocks that cover a key range instead of the whole frame. Blocks before start_block are decoded into the window (zstd blocks share one window, so a leading block’s bytes may be the match source for an in-range block and cannot simply be skipped) but their output is not returned; blocks at or after end_block are not decoded at all, which is the trailing-block work saving. Map a decompressed byte offset to a block index with FrameEmitInfo::decompressed_byte_range.
• Best-effort recovery: if a block decode fails, decoding stops, the clean prefix of in-range output is preserved in PartialDecode::data, and the failure is reported via PartialDecode::stopped_at. Passing (0, u32::MAX) decodes the whole frame, stopping at the first corrupt block (pure recovery).

end_block is exclusive; pass u32::MAX to decode to the end of the frame. Call on a freshly reset decoder (it decodes from the frame’s first block).

Resume (cold incremental / top-up)

A plain call drains its in-range output from the match window on return, so two consecutive calls cannot resume one another and growing a decoded extent would mean re-decoding the covering prefix from block 0 (O(extent) per growth, O(N²) for a forward walk). The resume / emit_resume arguments make a symmetric one-call grow-loop possible:

• emit_resume = true captures the cross-block carry-over state (entropy tables + repcode history + the next block index / output offset) into PartialDecode::resume_state. The entropy-table snapshot clone is only paid when this is set. The snapshot is None when the decode reaches the frame’s last block (PartialDecode::frame_finished): there is no following block to resume from, so an incremental walk stops on frame_finished rather than on a None snapshot.
• resume = Some(ResumeInput) continues from a previously emitted ResumeState WITHOUT re-decompressing the preceding blocks: the match window is primed from ResumeInput::window_prime and the entropy/repcode tables are restored from the state, so a Repeat_Mode resume block resolves byte-identically to a contiguous decode — even across a dropped (cold) decoder.

When resume is Some, decoding resumes at ResumeState::block_index and the start_block argument is ignored (pass resume.state.block_index()); position source at that block’s compressed frame offset (FrameEmitInfo::blocks[block_index].offset_in_frame). After a resumed call, bytes_read_from_source and any stopped_at offsets are relative to the repositioned source.

Dictionaries: ResumeState does NOT carry the dictionary content. For a dictionary frame, attach the dictionary to the resuming decoder the same way as for a fresh decode — reset with the dictionary registered (or reset_with_dict_handle) BEFORE this call — so dict-sourced matches near the frame start resolve. The caller already holds the dictionary (it supplied it at encode time), so re-supplying it on resume is free; storing it in the snapshot would only duplicate it. The resume guard records the applied dictionary’s identity and rejects (FrameDecoderError::ResumeFrameMismatch) a resume whose active dictionary differs from the one the snapshot was captured under.

Errors

Returns FrameDecoderError::NotYetInitialized if the decoder has not been reset, FrameDecoderError::WindowSizeTooBig if the window ceiling was lowered below the frame’s window after it started (see set_max_window_size), FrameDecoderError::InvalidBlockRange if the effective start exceeds end_block, FrameDecoderError::ResumeWindowTooShort if resume’s window_prime is shorter than the match window the resume block can reach back into (min(window_size, output_offset)), and FrameDecoderError::ResumeFrameMismatch if the snapshot was captured from a frame with a different decode shape / dictionary, or (with the hash feature) a window_prime whose content does not match what was captured — all rejected up front rather than silently mis-resolving matches. A corrupt block is NOT an Err here: it is reported via PartialDecode::stopped_at so the clean prefix survives.

pub fn collect(&mut self) -> Option<Vec<u8>>

Collect every decoded byte not collected yet. While the frame is still decoding the window stays inside as history, already handed out; once it is finished (Self::is_finished) the buffer is emptied.

pub fn collect_to_writer(&mut self, w: impl Write) -> Result<usize, Error>

Self::collect into w.

pub fn can_collect(&self) -> usize

How many decoded bytes are waiting to be collected: every byte decoded and not collected yet, whether or not the frame is finished.

pub fn decode_from_to( &mut self, source: &[u8], target: &mut [u8], ) -> Result<(usize, usize), FrameDecoderError>

Decodes blocks from the source slice and reads from the decodebuffer into the target slice, one block at a time: output is handed to target as each block completes, and no further block is decoded while target cannot take what is already decoded. The decode buffer so holds one window plus one block however much input is supplied; call again with the unread input to continue. The source slice may contain only parts of a frame but must contain at least one full block to make progress

By all means use decode_blocks if you have a io.Reader available. This is just for compatibility with other decompressors which try to serve an old-style c api

Every decoded byte goes to target as soon as it has room, the window staying inside as history, so a frame that is not finished yet still hands over everything its blocks so far decode to.

Returns (read, written). Both zero means the call made no progress: the source holds no full block and the buffer no undelivered output, so the same input cannot advance. read == 0 with written > 0 is progress of the other kind: target filled from output already buffered, and the same input decodes further once the caller offers more room.

Note that no kind of block can be bigger than 128kb. So to be safe use at least 128*1024 (max block content size) + 3 (block_header size) + 18 (max frame_header size) bytes as your source buffer

You may call this function with an empty source after all bytes have been decoded. This is equivalent to just call decoder.read(&mut target)

pub fn decode_all( &mut self, input: &[u8], output: &mut [u8], ) -> Result<usize, FrameDecoderError>

Decode multiple frames into the output slice.

input must contain an exact number of frames. Skippable frames are allowed and will be skipped during decode.

output must be large enough to hold the decompressed data. If you don’t know how large the output will be, use FrameDecoder::decode_blocks instead.

This calls FrameDecoder::init, and all bytes currently in the decoder will be lost.

Returns the number of bytes written to output.

pub fn decode_all_with_skippable_visitor<F>( &mut self, input: &[u8], output: &mut [u8], visitor: F, ) -> Result<usize, FrameDecoderError>

where F: FnMut(u8, &[u8]),

Available on crate feature lsm only.

Decode multiple frames into the output slice, invoking visitor for every skippable frame encountered before advancing past it.

input must contain an exact number of frames. Skippable frames (RFC 8878 §3.1.2 magic numbers 0x184D2A50..=0x184D2A5F) are allowed and will be both visited AND skipped: the visitor gets (magic_variant, payload) where magic_variant is the low nibble of the magic (magic - 0x184D2A50, range 0..=15) and payload is a borrowed slice of the on-wire payload bytes (the skippable frame’s Frame_Size field worth of data) into input — no allocation.

The visitor sees skippable frames in stream order; interleaved regular zstd frames continue to decompress into output exactly as decode_all does.

output must be large enough to hold the decompressed data. Returns the number of bytes written to output.

Example

use structured_zstd::decoding::FrameDecoder;

let mut decoder = FrameDecoder::new();
let mut output = vec![0u8; 1024];
let mut collected: Vec<(u8, Vec<u8>)> = Vec::new();
let n = decoder.decode_all_with_skippable_visitor(
    input,
    &mut output,
    |variant, payload| collected.push((variant, payload.to_vec())),
)?;

pub fn decode_all_with_dict_handle( &mut self, input: &[u8], output: &mut [u8], dict: &DictionaryHandle, ) -> Result<usize, FrameDecoderError>

Decode multiple frames into the output slice using a pre-parsed dictionary handle.

input must contain an exact number of frames. Skippable frames are allowed and will be skipped during decode.

output must be large enough to hold the decompressed data. If you don’t know how large the output will be, use FrameDecoder::decode_blocks instead.

This calls FrameDecoder::init_with_dict_handle, and all bytes currently in the decoder will be lost.

Warning

Each decoded frame is initialized with dict, even when a frame header omits the optional dictionary ID. Callers must only use this API when they already know the input frames were encoded with the provided dictionary; otherwise decoded output can be silently corrupted.

pub fn decode_all_with_dict_handles( &mut self, input: &[u8], output: &mut [u8], dicts: &[DictionaryHandle], ) -> Result<usize, FrameDecoderError>

decode_all_with_dict_handle with a choice of dictionaries per frame, as reset_with_dict_handles makes it: each frame decodes with the dictionary its header names, or with the last of dicts when it names none. An empty dicts is decode_all.

use structured_zstd::decoding::{Dictionary, DictionaryHandle, FrameDecoder};

let dict = DictionaryHandle::from_dictionary(
    Dictionary::from_raw_content(7, b"shared dictionary content".to_vec()).unwrap(),
);
let frame = structured_zstd::encoding::compress_to_vec(
    &b"payload"[..],
    structured_zstd::encoding::CompressionLevel::Fastest,
);
let mut out = [0u8; 7];
let mut decoder = FrameDecoder::new();
let n = decoder
    .decode_all_with_dict_handles(&frame, &mut out, &[dict])
    .unwrap();
assert_eq!(&out[..n], b"payload");

pub fn decode_all_with_dict_bytes( &mut self, input: &[u8], output: &mut [u8], raw_dictionary: &[u8], ) -> Result<usize, FrameDecoderError>

Decode multiple frames into the output slice using a serialized dictionary.

Warning

Each decoded frame is initialized with the parsed dictionary, even when a frame header omits the optional dictionary ID. Callers must only use this API when they already know the input frames were encoded with that dictionary; otherwise decoded output can be silently corrupted.

pub fn decode_all_to_vec( &mut self, input: &[u8], output: &mut Vec<u8>, ) -> Result<(), FrameDecoderError>

Decode multiple frames into the extra capacity of the output vector.

input must contain an exact number of frames.

output must have enough spare capacity to hold the decompressed data. This adds no extra slack: exact-fit output is now eligible for the direct decode path, so a Vec::with_capacity(fcs) is decoded straight into without a growth/reallocation. It will NOT grow the vector to fit the decompressed payload itself; the caller’s pre-allocated capacity must already cover the data. If you don’t know how large the output will be, use FrameDecoder::decode_blocks instead.

This calls FrameDecoder::init, and all bytes currently in the decoder will be lost.

The length of the output vector is updated to include the decompressed data. The length is not changed if an error occurs.

Trait Implementations

impl Default for FrameDecoder

fn default() -> Self

Returns the “default value” for a type.

impl Read for FrameDecoder

Read bytes from the decode_buffer that are no longer needed. While the frame is not yet finished this will retain window_size bytes, else it will drain it completely

fn read(&mut self, target: &mut [u8]) -> Result<usize, Error>

Pull some bytes from this source into the specified buffer, returning how many bytes were read.

fn read_vectored(&mut self, bufs: &mut [IoSliceMut<'_>]) -> Result<usize, Error>

Like read, except that it reads into a slice of buffers.

fn is_read_vectored(&self) -> bool

🔬This is a nightly-only experimental API. (can_vector)

Determines if this Reader has an efficient read_vectored implementation.

fn read_to_end(&mut self, buf: &mut Vec<u8>) -> Result<usize, Error>

Reads all bytes until EOF in this source, placing them into buf.

fn read_to_string(&mut self, buf: &mut String) -> Result<usize, Error>

Reads all bytes until EOF in this source, appending them to buf.

fn read_exact(&mut self, buf: &mut [u8]) -> Result<(), Error>

Reads the exact number of bytes required to fill buf.

fn read_buf(&mut self, buf: BorrowedCursor<'_, u8>) -> Result<(), Error>

🔬This is a nightly-only experimental API. (read_buf)

Pull some bytes from this source into the specified buffer.

fn read_buf_exact( &mut self, cursor: BorrowedCursor<'_, u8>, ) -> Result<(), Error>

🔬This is a nightly-only experimental API. (read_buf)

Reads the exact number of bytes required to fill cursor.

fn by_ref(&mut self) -> &mut Self

where Self: Sized,

Creates a “by reference” adapter for this instance of Read.

fn bytes(self) -> Bytes<Self>

where Self: Sized,

Transforms this Read instance to an Iterator over its bytes.

fn chain<R>(self, next: R) -> Chain<Self, R>

where R: Read, Self: Sized,

Creates an adapter which will chain this stream with another.

fn take(self, limit: u64) -> Take<Self>

where Self: Sized,

Creates an adapter which will read at most limit bytes from it.

fn read_array<const N: usize>(&mut self) -> Result<[u8; N], Error>

where Self: Sized,

🔬This is a nightly-only experimental API. (read_array)

Read and return a fixed array of bytes from this source.

fn read_le<T>(&mut self) -> Result<T, Error>

where T: FromEndianBytes, Self: Sized,

🔬This is a nightly-only experimental API. (read_le)

Read and return a type (e.g. an integer) in little-endian order.

fn read_be<T>(&mut self) -> Result<T, Error>

where T: FromEndianBytes, Self: Sized,

🔬This is a nightly-only experimental API. (read_le)

Read and return a type (e.g. an integer) in big-endian order.