Skip to main content

Module compressor

Module compressor 

Source
Expand description

L2 Blob Compressor.

Stateless, deep module encapsulating compression of blob payloads spilled to the L2 (on-disk) tier of [BlobCache]. The purpose is to shrink durable blob footprints without harming hot-path latency or wasting CPU on payloads that are already incompressible.

§Design

The module is intentionally small and side-effect free:

  • All operations are static — no internal state, no allocator pinning, no threading concerns. Inputs are &[u8] slices, outputs are owned Vec<u8>.
  • Compression is best-effort: when the input is small, when the content type already represents a compressed media format, or when the zstd output fails to shrink the input meaningfully, the original bytes are returned inside Compressed::Raw and no encode is performed (or its result is discarded).
  • The compressed variant carries the original byte length so decompression can pre-allocate exactly and verify the encoded length on decode.

Wiring into [BlobCache] L2 put/get is performed in a follow-up slice; this module is additive and has no callers in this commit.

Structs§

CompressOpts
Configuration knobs for L2BlobCompressor::compress.
L2BlobCompressor
Stateless compressor for L2 blob payloads.

Enums§

CompressError
Errors produced by the compressor.
Compressed
Storage-ready representation of a blob payload after the compressor has inspected it. Raw is byte-equivalent to the input; Zstd carries an encoded payload plus the original byte length for verification.

Constants§

DEFAULT_MAX_RATIO
Default cutoff ratio above which the compressed bytes are discarded. 0.95 means we require at least a 5% reduction to keep the encoded form.
DEFAULT_MIN_BYTES
Default minimum payload size eligible for compression. Sub-kilobyte payloads rarely benefit and the framing overhead can exceed any savings.
DEFAULT_ZSTD_LEVEL
Default zstd compression level — favours encode speed over ratio. The L2 tier is meant to amplify capacity, not to be the smallest possible store.