Module compress 

Streaming compression abstractions backed by zrip (zstd).

Design

Both Compressor and Decompressor are stream-oriented: they accept bounded input chunks and produce bounded output chunks, so the heap footprint of processing a file does not grow with the file size. The STREAM_BUF_SIZE sliding window keeps memory constant (~64 KiB typical, capped well below 4 MiB even for adversarial frames).

Wire format

The compressed body is a concatenation of independent zstd frames: each Compressor::compress call emits one complete frame when the next call arrives (or on Compressor::finish).

The decompressor buffers compressed bytes until a complete frame is present (frame boundaries are detected by a small built-in zstd frame parser), then decodes exactly that frame with zrip::decompress_with_limit. A frame may therefore span several decompress() inputs, and one input may contain several frames.

Constant memory

• Compressor: buffers at most one input chunk internally plus the zrip encoder workspace (~150 KiB).
• Decompressor: holds at most one compressed frame (MAX_PENDING_FRAME) and one decompressed frame (MAX_FRAME_OUTPUT) at a time.

The server should feed Compressor::compress in STREAM_BUF_SIZE windows (64 KiB), keeping frames ~64 KiB and the client’s transient buffers small.

Structs

ZripCompressor

zstd compressor emitting one independent frame per input chunk.

ZripDecompressor

zstd decompressor reassembling frames across arbitrary chunk boundaries.

Enums

CompressionFormat

Compression formats understood by libfw.

Constants

MAX_FRAME_OUTPUT

Safety ceiling for the decompressed size of a single frame.

MAX_OUTPUT_PER_CALL

Default cap on the total bytes a single Decompressor::decompress call may append to the output buffer.

MAX_PENDING_FRAME

Maximum compressed bytes buffered while waiting for a frame boundary.

ZRIP_DEFAULT_LEVEL

Default zrip compression level (Fast strategy, good for network transfer).

Traits

Compressor

Streaming compressor. Feed it bounded input chunks; it appends the compressed bytes produced so far to the provided output buffer.

Decompressor

Streaming decompressor. Feed it arbitrary compressed bytes; decoded output is appended to the provided buffer.

Functions

compressor

Construct a compressor for format.

decompressor

Construct a decompressor for format.

decompressor_with_limit

Construct a decompressor for format with an explicit per-call output budget (bytes a single Decompressor::decompress call may append).