Skip to main content

bz3_min_memory_needed

libbzip3_sys

Function bz3_min_memory_needed 

Source 
pub unsafe extern "C" fn bz3_min_memory_needed(
    block_size: i32,
) -> usize
Expand description

@brief Calculate the minimal memory required for compression with the given block size. This includes all internal buffers and state structures. This calculates the amount of bytes that will be allocated by a call to bz3_new().

@details Memory allocation and usage patterns:

bz3_new():

  • Allocates all memory upfront:
    • Core state structure (sizeof(struct bz3_state))
    • Swap buffer (bz3_bound(block_size) bytes)
    • SAIS array (BWT_BOUND(block_size) * sizeof(int32_t) bytes)
    • LZP lookup table ((1 << LZP_DICTIONARY) * sizeof(int32_t) bytes)
    • Compression state (sizeof(state))
  • All memory remains allocated until bz3_free()

Additional memory may be used depending on API used from here.

§Low Level APIs

  1. bz3_encode_block() / bz3_decode_block():
    • Uses pre-allocated memory from bz3_new()
    • No additional memory allocation except for libsais (usually ~16KiB)
    • Peak memory usage of physical RAM varies with compression stages:
      • LZP: Uses LZP lookup table + swap buffer
      • BWT: Uses SAIS array + swap buffer
      • Entropy coding: Uses compression state (cm_state) + swap buffer

Using the higher level API, bz3_compress, expect an additional allocation of bz3_bound(block_size).

In the parallel version bz3_encode_blocks, each thread gets its own state, so memory usage is n_threads * bz3_compress_memory_needed().

§High Level APIs

  1. bz3_compress():

    • Allocates additional temporary compression buffer (bz3_bound(block_size) bytes) in addition to the memory amount returned by this method call and libsais.
    • Everything is freed after compression completes

  2. bz3_decompress():

    • Allocates additional temporary compression buffer (bz3_bound(block_size) bytes) in addition to the memory amount returned by this method call and libsais.
    • Everything is freed after compression completes

Memory remains constant during operation, with except of some small allocations from libsais during BWT stage. That is not accounted by this function, though it usually amounts to ~16KiB, negligible. The worst case of BWT is 2*block_size technically speaking.

No dynamic (re)allocation occurs outside of that.

@param block_size The block size to be used for compression @return The total number of bytes required for compression, or 0 if block_size is invalid