Skip to main content

Module encoding

Module encoding 

Source
Expand description

Per-object content encoding (Content-Encoding), opt in per write.

ObjectStore::put_encoded compresses a body before it is stored and records the encoding beside it (GCS: the object’s contentEncoding; LocalDir and Mem: the backend’s own metadata). Reads decode it:

  • Decoded once. get inflates into one fresh buffer and returns it as Bytes; from there the body is shared by reference count like any other, so zero-copy consumers (an mmap’d file written from it, a Python buffer view) are unaffected.
  • No ranges. A byte range of a gzip stream is meaningless, and handing back compressed bytes would be silent corruption, so get_range of an encoded object is Error::EncodedRange. Encode only objects that are read whole.
  • Bounded. A small object can inflate without bound (a decompression bomb). Decoding stops at the store’s inflated-size limit — MAX_INFLATED_BYTES unless the store was built with_max_inflated — with Error::TooLarge; a truncated or damaged stream is Error::Corrupt. Neither ever panics.

The codec is feature gzip (flate2 on the pure-Rust miniz_oxide backend). Without it the type still exists, and every encoded write or read is Error::UnsupportedEncoding rather than raw compressed bytes.

Enums§

ContentEncoding
How an object’s stored bytes are encoded. No encoding (identity) is Option::<ContentEncoding>::None throughout the API.

Constants§

MAX_INFLATED_BYTES
Default bound on a decoded body: 1 GiB. A gzip stream inflates at most ~1032:1, so the bound, not the stored size, is what caps the memory one read can claim. Raise or lower it per store with with_max_inflated.