Module analyze_utils

Utility functions for analyzing and processing bit-packed data.

This module provides low-level utilities for:

• Size estimation and compression
• Bit manipulation and ordering
• Bitstream reader/writer creation and management

Core Functions

• size_estimate: Estimates compressed data size based on LZ matches and entropy
• get_zstd_compressed_size: Calculates actual compressed size using zstandard
• calculate_file_entropy: Computes Shannon entropy of input data
• reverse_bits: Reverses bits in a u64 value

Bitstream Utilities

• create_bit_reader: Creates a BitReaderContainer with specified endianness
• create_bit_writer: Creates a BitWriterContainer with specified endianness
• create_bit_writer_with_owned_data: Creates writer containing copied data
• get_writer_buffer: Retrieves underlying buffer from a writer
• bit_writer_to_reader: Converts a writer into a reader

Types

• BitReaderContainer: Wrapper around bit readers supporting both endians
• BitWriterContainer: Wrapper around bit writers supporting both endians

Enums

BitReaderContainer

Wrapper around the BitReader type that allows it to be used with either endian.

BitWriterContainer

Tracks statistics about individual bits in a field

Functions

bit_writer_to_reader

Converts a BitWriterContainer instance into a BitReaderContainer instance.

calculate_file_entropy

Calculates the entropy of a given input

create_bit_reader

Creates a BitReaderContainer instance based on the given BitOrder.

create_bit_writer

Creates a BitWriterContainer instance based on the given BitOrder.

create_bit_writer_with_owned_data

Creates a BitWriterContainer instance based on the given BitOrder. This copies the supplied data into a new buffer, which is then owned by the container.

get_writer_buffer

Retrieves the buffer behind a BitWriterContainer instance.

get_zstd_compressed_size

Determines the actual size of the compressed data by compressing with a realistic compressor.

reverse_bits

Reverses the bits of a u64 value

size_estimate

Estimate size of a compressed data based on precalculated LZ matches and entropy