Crate zeekstd

This crate provides a Rust implementation of the Zstandard Seekable Format, as outlined in the specification.

The seekable format splits compressed data into a series of independent “frames”, each compressed individually, so that decompression of a section in the middle of an archive only requires zstd to decompress at most a frame’s worth of extra data, instead of the entire archive.

The frames are appended, so that the decompression of the entire payload still regenerates the original content, using any compliant zstd decoder.

Zeekstd uses the bindings from the zstd_safe crate.

Compression

A seekable Encoder will start new frames automatically at 2MiB of uncompressed data. See EncodeOptions to change this and other compression parameters.

use std::{fs::File, io};
use zeekstd::Encoder;

let mut input = File::open("foo")?;
let output = File::create("foo.zst")?;
let mut encoder = Encoder::new(output)?;
io::copy(&mut input, &mut encoder)?;
// End compression and write the seek table
encoder.finish()?;

Decompression

By default, the seekable Decoder decompresses everything, from the first to the last frame.

use std::{fs::File, io};
use zeekstd::Decoder;

let input = File::open("seekable.zst")?;
let mut output = File::create("data")?;
let mut decoder = Decoder::new(input)?;
io::copy(&mut decoder, &mut output)?;

It can also decompress only specific frames.

decoder.set_lower_frame(2);
decoder.set_upper_frame(3);
io::copy(&mut decoder, &mut io::stdout())?;

Re-exports

pub use seek_table::SeekTable;

Modules

seek_table

Structs

BytesWrapper

A seekable wrapper around a byte slice.

DecodeOptions

Options that configure how data is decompressed.

Decoder

Decompresses data from a seekable source.

EncodeOptions

Options that configure how data is compressed.

Encoder

A single-use seekable encoder.

Error

The errors that may occur when working with this crate.

RawEncoder

A reusable, seekable encoder.

Enums

FrameSizePolicy

A policy that controls when new frames are started automatically.

Constants

SEEKABLE_MAGIC_NUMBER

The magic number of the seek table integrity field.

SEEKABLE_MAX_FRAMES

The maximum number of frames in a seekable archive.

SEEKABLE_MAX_FRAME_SIZE

The maximum size of the uncompressed data of a frame.

SEEK_TABLE_INTEGRITY_SIZE

The size of the seek table integrity field.

Traits

Seekable

Represents a seekable source.

Type Aliases

CompressionLevel

Represents the compression level used by zstd.

Result

A Result alias where the Err case is zeekstd::Error.