Crate hypeerlog 

hypeerlog

A blazingly fast HyperLogLog++ implementation designed for high-throughput, distributed cardinality estimation.

This crate faithfully implements the Google HyperLogLog++ paper, including standard bias correction and linear counting for small cardinalities.

The HyperLogLog algorithm is a probabilistic data structure used to estimate the number of distinct elements in a set. It operates using a fixed amount of memory while keeping the relative estimation error exceptionally small.

Features

• Flexible Hashing: Employs a custom, ultra-fast Murmur3 hasher by default (the gold standard for HyperLogLog sketches), while offering full generic support to drop in your own custom hasher implementation.
• Configurable Accuracy: Define your own precision or maximum relative error bounds to explicitly tune the exact accuracy vs. memory footprint trade-off required for your workload.
• Production Ready: Rigorously tested, fully micro-benchmarked, and meticulously optimized for raw performance.

Note on Design: This crate intentionally omits the sparse register representation described in the paper. By focusing entirely on a flattened dense register footprint, it removes serialization layout overheads and yields cleaner optimization paths for distributed network/storage engines where fixed-size states are highly desirable.

Estimating Cardinality

use hypeerlog::Hypeerlog;

let elems = vec![1, 2, 3, 4, 5, 6, 7, 1, 1, 2];

let mut hll = Hypeerlog::new();
hll.insert_many(&elems);

// The estimation is guaranteed to be within the typical HLL error bounds (e.g., ~2%).
assert_eq!(hll.cardinality().floor(), 7.0); 

Distributed Workloads & Merging

HyperLogLog sketches are perfectly additive. You can distribute massive datasets across multiple independent workers, compute highly efficient local sketches, and merge them later to find the global unique count.

use hypeerlog::Hypeerlog;

let elems = vec![1, 2, 3, 4, 5, 6, 7, 1, 1, 2];

let mut hll_one = Hypeerlog::new();
hll_one.insert_many(&elems[0..5]);

let mut hll_two = Hypeerlog::new();
hll_two.insert_many(&elems[5..]);

// Merge the second sketch into the first
let merged = hll_one.merge(hll_two).unwrap();
assert_eq!(merged.cardinality().floor(), 7.0);

no_std Support

This crate features a highly constrained, lightweight memory profile, making it a perfect fit for resource-constrained or bare-metal environments.

To enable standalone usage, activate the no_std feature in your Cargo.toml:

[dependencies]
hypeerlog = { version = "0.3.1", features = ["no_std"] }

All core estimation and merging features remain fully available in no_std mode via safe heap allocations handled contextually by the alloc crate.

Macros

hll

A convinient macro to create a Hypeerlog by directly adding elements to it, similar to the vec! macro

Structs

Hypeerlog

A probabilistic cardinality estimator based on the HyperLogLog++ algorithm.

Enums

HypeerlogError

All errors that can be returned. This happens when merging or loading

Functions

p_from_rel_error

Convert a relative error value into the corresponding precision

rel_error_from_p

Get the relative error corresponding to a specific p value