Skip to main content

Crate libz_rs_sys

Crate libz_rs_sys 

Source
Expand description

This crate is a C API for zlib-rs. The API is broadly equivalent to libz-sys and zlib-ng-sys.

From a rust perspective, this API is not very ergonomic. Use the flate2 crate for a more ergonomic rust interface to zlib.

§Features

custom-prefix

Add a custom prefix to all exported symbols.

The value of the LIBZ_RS_SYS_PREFIX is used as a prefix for all exported symbols. For example:

> LIBZ_RS_SYS_PREFIX="MY_CUSTOM_PREFIX" cargo build -p libz-rs-sys --features=custom-prefix
   Compiling libz-rs-sys v0.2.1 (/home/folkertdev/rust/zlib-rs/libz-rs-sys)
    Finished `dev` profile [optimized + debuginfo] target(s) in 0.21s
> objdump -tT target/debug/liblibz_rs_sys.so | grep "uncompress"
0000000000081028 l     O .got	0000000000000000              _ZN7zlib_rs7inflate10uncompress17he7d985e55c58a189E$got
000000000002c570 l     F .text	00000000000001ef              _ZN7zlib_rs7inflate10uncompress17he7d985e55c58a189E
0000000000024330 g     F .text	000000000000008e              MY_CUSTOM_PREFIXuncompress
0000000000024330 g    DF .text	000000000000008e  Base        MY_CUSTOM_PREFIXuncompress

c-allocator, rust-allocator

Pick the default allocator implementation that is used if no zalloc and zfree are configured in the input z_stream.

  • c-allocator: use malloc/free for the implementation of zalloc and zfree
  • rust-allocator: the rust global allocator for the implementation of zalloc and zfree

The rust-allocator is the default when this crate is used as a rust dependency, and slightly more efficient because alignment is handled by the allocator. When building a dynamic library, it may make sense to use c-allocator instead.

std

Assume that std is available. When this feature is turned off, this crate is compatible with #![no_std].

gz

Exports the gz* family of functions (gzread, gzwrite, etc.). This feature is off by default because these functions are rarely used from rust, and defining these functions requires a dependency on libc.

§Example

This example compresses (“deflates”) the string "Hello, World!" and then decompresses (“inflates”) it again.

let mut strm = libz_rs_sys::z_stream::default();

let version = libz_rs_sys::zlibVersion();
let stream_size = core::mem::size_of_val(&strm) as i32;

let level = 6; // the default compression level
let err = unsafe { libz_rs_sys::deflateInit_(&mut strm, level, version, stream_size) };
assert_eq!(err, libz_rs_sys::Z_OK);

let input = "Hello, World!";
strm.avail_in = input.len() as _;
strm.next_in = input.as_ptr();

let mut output = [0u8; 32];
strm.avail_out = output.len() as _;
strm.next_out = output.as_mut_ptr();

let err = unsafe { libz_rs_sys::deflate(&mut strm, libz_rs_sys::Z_FINISH) };
assert_eq!(err, libz_rs_sys::Z_STREAM_END);

let err = unsafe { libz_rs_sys::deflateEnd(&mut strm) };
assert_eq!(err, libz_rs_sys::Z_OK);

let deflated = &mut output[..strm.total_out as usize];

let mut strm = libz_rs_sys::z_stream::default();
let err = unsafe { libz_rs_sys::inflateInit_(&mut strm, version, stream_size) };
assert_eq!(err, libz_rs_sys::Z_OK);

strm.avail_in = deflated.len() as _;
strm.next_in = deflated.as_ptr();

let mut output = [0u8; 32];
strm.avail_out = output.len() as _;
strm.next_out = output.as_mut_ptr();

let err = unsafe { libz_rs_sys::inflate(&mut strm, libz_rs_sys::Z_FINISH) };
assert_eq!(err, libz_rs_sys::Z_STREAM_END);

let err = unsafe { libz_rs_sys::inflateEnd(&mut strm) };
assert_eq!(err, libz_rs_sys::Z_OK);

let inflated = &output[..strm.total_out as usize];

assert_eq!(inflated, input.as_bytes())

§Memory Management

Zlib does not assume a global allocator, but instead accepts zalloc and zfree functions for allocating and deallocating memory.

ⓘ
type alloc_func = unsafe extern "C" fn(*mut c_void, c_uint, c_uint) -> *mut c_void;
type free_func = unsafe extern "C" fn(*mut c_void, *mut c_void);

struct z_stream {
    // ...
    zalloc: Option<alloc_func>,
    zfree: Option<free_func>,
    opaque: *mut c_void,
    // ..
}

These functions can run arbitrary logic, so long as they satisfy their contract. This mechanism can be used e.g. on embedded systems to use an array in static memory as the working memory for (de)compression. For more information see the z_stream documentation.

For convenience we provide two default memory allocators:

  • rust uses the rust global allocator
  • c uses malloc and free

When used as a rust crate, the default is to use the rust global allocator. This behavior can be overridden using feature flags to either configure the C allocator as the default, or to configure no default allocator at all. If no default is configured, and no custom allocation and deallocation functions are provided, initialization will return an error.

§Compression Levels

The zlib library supports compression levels 0 up to and including 9. The level indicates a tradeoff between time spent on the compression versus the compression ratio, the factor by which the input is reduced in size:

  • level 0: no compression at all
  • level 1: fastest compression
  • level 6: default (a good tradeoff between speed and compression ratio)
  • level 9: best compression

Beyond this intuition, the exact behavior of the compression levels is not specified. The implementation of zlib-rs follows the implementation of zlib-ng, and deviates from the one in stock zlib.

In particular, our compression level 1 is extremely fast, but also just does not compress that well. On the silesia-small.tar input file, we see these output sizes:

implementationcompression leveloutput size (mb)
-015.74
stock1 7.05
rs1 8.52
rs2 6.90
rs4 6.55

But, zlib-rs is much faster than stock zlib. In our benchmarks, it is only at level 4 that we spend roughly as much time as stock zlib on level 1:

implementationcompression levelwall time (ms)
stock1185
rs2139
rs4181

In our example, the main options are:

  • level 1: worse compression, but much faster
  • level 2: equivalent compression, but significantly faster
  • level 4: better compression, at the same speed

In summary, when you upgrade from stock zlib, we recommend that you benchmark on your data and target platform, and pick the right compression level for your use case.

§Safety

Most of the functions in this module are unsafe fns, meaning that their behavior may be undefined if certain assumptions are broken by the caller. In most cases, documentation in this module refers to the safety assumptions of standard library functions.

In most cases, pointers must be either NULL or satisfy the requirements of &*ptr or &mut *ptr. This requirement maps to the requirements of pointer::as_ref and pointer::as_mut for immutable and mutable pointers respectively.

For pointer and length pairs, describing some sequence of elements in memory, the requirements of core::slice::from_raw_parts or core::slice::from_raw_parts_mut apply. In some cases, the element type T is converted into MaybeUninit<T>, meaning that while the slice must be valid, the elements in the slice can be uninitialized. Using uninitialized buffers for output is more performant.

Finally, some functions accept a string argument, which must either be NULL or satisfy the requirements of core::ffi::CStr::from_ptr.

Structs§

gz_header
gzip header information passed to and from zlib routines. See RFC 1952 for more details on the meanings of these fields.
z_stream
The current stream state

Enums§

gzFile_sgz
In the zlib C API, this structure exposes just enough of the internal state of an open gzFile to support the gzgetc C macro. Since Rust code won’t be using that C macro, we define gzFile_s as an empty structure.
internal_state

Constants§

Z_ASCII
Z_BEST_COMPRESSION
Z_BEST_SPEED
Z_BINARY
Z_BLOCK
Z_BUF_ERROR
Z_DATA_ERROR
Z_DEFAULT_COMPRESSION
Z_DEFAULT_STRATEGY
Z_DEFLATED
Z_ERRNO
Z_FILTERED
Z_FINISH
Z_FIXED
Z_FULL_FLUSH
Z_HUFFMAN_ONLY
Z_MEM_ERROR
Z_NEED_DICT
Z_NO_COMPRESSION
Z_NO_FLUSH
Z_OK
Z_PARTIAL_FLUSH
Z_RLE
Z_STREAM_END
Z_STREAM_ERROR
Z_SYNC_FLUSH
Z_TEXT
Z_TREES
Z_UNKNOWN
Z_VERSION_ERROR

Functions§

adler32⚠
Calculates the adler32 checksum of a sequence of bytes.
adler32_combine
Combines the checksum of two slices into one.
adler32_combine64
Combines the checksum of two slices into one.
adler32_z⚠
Calculates the adler32 checksum of a sequence of bytes.
compress⚠
Compresses source into dest, and writes the final deflated size into destLen.
compress2⚠
Compresses source into dest, and writes the final deflated size into destLen.
compressBound
Returns an upper bound on the compressed size after compress or compress2 on sourceLen bytes.
crc32⚠
Calculates the crc32 checksum of a sequence of bytes.
crc32_combine
Combines the checksum of two slices into one.
crc32_combine64
Combines the checksum of two slices into one.
crc32_combine_gen
Return the operator corresponding to length len2, to be used with crc32_combine_op. len2 must be non-negative.
crc32_combine_gen64
Return the operator corresponding to length len2, to be used with crc32_combine_op. len2 must be non-negative.
crc32_combine_op
Give the same result as crc32_combine, using op in place of len2. op is is generated from len2 by crc32_combine_gen. This will be faster than crc32_combine if the generated op is used more than once.
crc32_z⚠
Calculates the crc32 checksum of a sequence of bytes.
deflate⚠
Compresses as much data as possible, and stops when the input buffer becomes empty or the output buffer becomes full.
deflateBound⚠
Returns an upper bound on the compressed size after deflation of sourceLen bytes.
deflateCopy⚠
Sets the destination stream as a complete copy of the source stream.
deflateEnd⚠
Deallocates all dynamically allocated data structures for this stream.
deflateGetDictionary⚠
Returns the sliding dictionary being maintained by deflate.
deflateInit2_⚠
Initializes the state for compression
deflateInit_⚠
Initializes the state for compression
deflateParams⚠
Dynamically update the compression level and compression strategy.
deflatePending⚠
Returns the number of bytes and bits of output that have been generated, but not yet provided in the available output.
deflatePrime⚠
Inserts bits in the deflate output stream.
deflateReset⚠
This function is equivalent to deflateEnd followed by deflateInit_, but does not free and reallocate the internal compression state.
deflateSetDictionary⚠
Initializes the compression dictionary from the given byte sequence without producing any compressed output.
deflateSetHeader⚠
Provides gzip header information for when a gzip stream is requested by deflateInit2_.
deflateTune⚠
Fine tune deflate’s internal compression parameters.
get_crc_table
The CRC table used by the crc32 checksum algorithm.
gzbuffer⚠gz
Set the internal buffer size used by this library’s functions for file to size. The default buffer size is 128 KB. This function must be called after gzopen or gzdopen, but before any other calls that read or write the file (including gzdirect). The buffer memory allocation is always deferred to the first read or write. Three times size in buffer space is allocated.
gzclearerr⚠gz
Clear the error and end-of-file state for file.
gzclose⚠gz
Close an open gzip file and free the internal data structures referenced by the file handle.
gzclose_r⚠gz
Close a gzip file that was opened for reading.
gzclose_w⚠gz
Close a gzip file that was opened for writing.
gzdirect⚠gz
Check whether file is in direct mode (reading or writing literal bytes without compression).
gzdopen⚠gz
Given an open file descriptor, prepare to read or write a gzip file. NOTE: This is similar to gzopen, but for cases where the caller already has the file open.
gzeof⚠gz
Check whether a read operation has tried to read beyond the end of file.
gzerror⚠gz
Retrieve the zlib error code and a human-readable string description of the most recent error on a gzip file stream.
gzflush⚠gz
Flush all pending output buffered in file. The parameter flush is interpreted the same way as in the deflate function. The return value is the zlib error number (see gzerror). gzflush is permitted only when writing.
gzfread⚠gz
Read and decompress up to nitems items of size size from file into buf, otherwise operating as gzread does. This duplicates the interface of C stdio’s fread(), with size_t request and return types.
gzfwrite⚠gz
Compress and write nitems items of size size from buf to file, duplicating the interface of C stdio’s fwrite, with size_t request and return types.
gzgetc⚠gz
Read one decompressed byte from file.
gzgetc_⚠gz
Backward-compatibility alias for gzgetc.
gzgets⚠gz
Read decompressed bytes from file into buf, until len-1 characters are read, or until a newline character is read and transferred to buf, or an end-of-file condition is encountered. If any characters are read or if len is one, the string is terminated with a null character. If no characters are read due to an end-of-file or len is less than one, then the buffer is left untouched.
gzoffset⚠gz
Return the current compressed (actual) read or write offset of file. This offset includes the count of bytes that precede the gzip stream, for example when appending or when using gzdopen for reading. When reading, the offset does not include as yet unused buffered input. This information can
gzoffset64⚠gz
Return the current compressed (actual) read or write offset of file. This offset includes the count of bytes that precede the gzip stream, for example when appending or when using gzdopen for reading. When reading, the offset does not include as yet unused buffered input. This information can
gzopen⚠gz
Open a gzip file for reading or writing.
gzopen64⚠gz
Open a gzip file for reading or writing.
gzprintf⚠gz and gzprintf
Convert, format, compress, and write the variadic arguments ... to a file under control of the string format, as in fprintf.
gzputc⚠gz
Compress and write c, converted to an unsigned 8-bit char, into file.
gzputs⚠gz
Compress and write the given null-terminated string s to file, excluding the terminating null character.
gzread⚠gz
Read and decompress up to len uncompressed bytes from file into buf. If the input file is not in gzip format, gzread copies up to len bytes into the buffer directly from the file.
gzrewind⚠gz
Rewind file to the start. This function is supported only for reading.
gzseek⚠gz
Set the starting position to offset relative to whence for the next gzread or gzwrite on file. The offset represents a number of bytes in the uncompressed data stream. The whence parameter is defined as in lseek(2), but only SEEK_CUR (relative to current position) and SEEK_SET (absolute from start of the uncompressed data stream) are supported.
gzseek64⚠gz
Set the starting position to offset relative to whence for the next gzread or gzwrite on file. The offset represents a number of bytes in the uncompressed data stream. The whence parameter is defined as in lseek(2), but only SEEK_CUR (relative to current position) and SEEK_SET (absolute from start of the uncompressed data stream) are supported.
gzsetparams⚠gz
Dynamically update the compression level and strategy for file. See the description of deflateInit2_ for the meaning of these parameters. Previously provided data is flushed before applying the parameter changes.
gztell⚠gz
Return the starting position for the next gzread or gzwrite on file. This position represents a number of bytes in the uncompressed data stream, and is zero when starting, even if appending or reading a gzip stream from the middle of a file using gzdopen.
gztell64⚠gz
Return the starting position for the next gzread or gzwrite on file. This position represents a number of bytes in the uncompressed data stream, and is zero when starting, even if appending or reading a gzip stream from the middle of a file using gzdopen.
gzungetc⚠gz
Push c back onto the stream for file to be read as the first character on the next read. At least one character of push-back is always allowed.
gzvprintf⚠gz and gzprintf
Convert, format, compress, and write the variable argument list to a file under control of the string format, as in vfprintf.
gzwrite⚠gz
Compress and write the len uncompressed bytes at buf to file.
inflate⚠
Decompresses as much data as possible, and stops when the input buffer becomes empty or the output buffer becomes full.
inflateBack⚠
Decompresses as much data as possible, and stops when the input buffer becomes empty or the output buffer becomes full.
inflateBackEnd⚠
Deallocates all dynamically allocated data structures for this stream.
inflateBackInit_⚠
Initializes the state for decompression
inflateCopy⚠
Sets the destination stream as a complete copy of the source stream.
inflateEnd⚠
Deallocates all dynamically allocated data structures for this stream.
inflateGetDictionary⚠
Returns the sliding dictionary being maintained by inflate.
inflateGetHeader⚠
Requests that gzip header information be stored in the provided gz_header structure.
inflateInit2_⚠
Initializes the state for decompression
inflateInit_⚠
Initializes the state for decompression
inflateMark⚠
Gives information about the current location of the input stream.
inflatePrime⚠
Inserts bits in the inflate input stream.
inflateReset⚠
Equivalent to inflateEnd followed by inflateInit_, but does not free and reallocate the internal decompression state.
inflateReset2⚠
This function is the same as inflateReset, but it also permits changing the wrap and window size requests.
inflateSetDictionary⚠
Initializes the decompression dictionary from the given uncompressed byte sequence.
inflateSync⚠
Skips invalid compressed data until
uncompress⚠
Inflates source into dest, and writes the final inflated size into destLen.
uncompress2⚠
Inflates source into dest like uncompress, and writes the final inflated size into destLen and the number of source bytes consumed into sourceLen.
zError
Get the error message for an error. This could be the value returned by e.g. compress or inflate.
zlibCompileFlags
Return flags indicating compile-time options.
zlibVersion
The version of the zlib library.

Type Aliases§

Bytef
alloc_func
free_func
gzFilegz
File handle for an open gzip file.
gz_headerp
in_func
out_func
size_t
uInt
uLong
uLongf
voidp
voidpc
voidpf
z_off64_tNot (Windows and GNU)
z_off_tNon-WebAssembly
z_streamp