flowstats 0.1.2

Collection of stream analytics algorithms: cardinality, quantiles, frequency, sampling, and more
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151

//! # Flowstats
//!
//! Production-grade streaming algorithms for Rust.
//!
//! Flowstats provides high-performance implementations of probabilistic data structures
//! and streaming algorithms, designed for real-time analytics and large-scale data processing.
//!
//! ## Features
//!
//! - **Cardinality Estimation**: Count distinct elements with HyperLogLog
//! - **Frequency Estimation**: Track item frequencies with Count-Min Sketch
//! - **Heavy Hitters**: Find top-K elements with Space-Saving
//! - **Quantile Estimation**: Compute percentiles with t-digest
//! - **Full Mergeability**: All sketches support distributed merge operations
//! - **Error Bounds**: Formal guarantees on approximation accuracy
//!
//! ## Quick Start
//!
//! ```rust
//! use flowstats::prelude::*;
//!
//! // Count distinct users
//! let mut hll = HyperLogLog::new(14);
//! for user_id in ["alice", "bob", "charlie", "alice"] {
//!     hll.insert(user_id);
//! }
//! println!("Distinct users: ~{}", hll.estimate());
//!
//! // Track request latencies
//! let mut digest = TDigest::new(100.0);
//! for latency in [12.5, 45.2, 23.1, 67.8, 15.3] {
//!     digest.add(latency);
//! }
//! println!("p99 latency: {:?}", digest.quantile(0.99));
//!  
//! ```
//!
//! ## Distributed Computing
//!
//! All sketches implement the [`Sketch`](traits::Sketch) trait which includes
//! a `merge` operation, allowing sketches to be combined across distributed workers:
//!
//! ```rust
//! use flowstats::cardinality::HyperLogLog;
//! use flowstats::traits::Sketch;
//!
//! let mut worker1 = HyperLogLog::new(14);
//! let mut worker2 = HyperLogLog::new(14);
//!
//! // Each worker processes its partition
//! worker1.insert("user_a");
//! worker2.insert("user_b");
//!
//! // Merge results
//! worker1.merge(&worker2).unwrap();
//! ```
//!
//! ## Feature Flags
//!
//! Algorithm families (pick what you need):
//! - `cardinality` (default): HyperLogLog for distinct counting
//! - `frequency` (default): Count-Min Sketch, Space-Saving (tbd)
//! - `quantiles` (default): t-digest for percentiles
//! - `membership`: (default) Bloom filter
//! - `sampling`: Reservoir and weighted sampling (tbd)
//! - `sets`: Theta sketch for set operations (tbd)
//! - `statistics`: Running moments, entropy (tbd)
//! - `full`: Enable all algorithm families
//!
//! Platform features:
//! - `std` (default): Standard library support
//! - `serde`: Enable serialization

#![cfg_attr(not(feature = "std"), no_std)]
#![cfg_attr(docsrs, feature(doc_cfg))]

#[cfg(not(feature = "std"))]
extern crate alloc;

mod math;
pub mod traits;

#[cfg(feature = "cardinality")]
#[cfg_attr(docsrs, doc(cfg(feature = "cardinality")))]
pub mod cardinality;

#[cfg(feature = "quantiles")]
#[cfg_attr(docsrs, doc(cfg(feature = "quantiles")))]
pub mod quantiles;

#[cfg(feature = "frequency")]
#[cfg_attr(docsrs, doc(cfg(feature = "frequency")))]
pub mod frequency;

#[cfg(feature = "membership")]
#[cfg_attr(docsrs, doc(cfg(feature = "membership")))]
pub mod membership;

#[cfg(feature = "sampling")]
#[cfg_attr(docsrs, doc(cfg(feature = "sampling")))]
pub mod sampling;

#[cfg(feature = "statistics")]
#[cfg_attr(docsrs, doc(cfg(feature = "statistics")))]
pub mod statistics;

pub mod prelude {
    pub use crate::traits::*;

    #[cfg(feature = "cardinality")]
    pub use crate::cardinality::HyperLogLog;

    #[cfg(feature = "quantiles")]
    pub use crate::quantiles::TDigest;

    #[cfg(feature = "frequency")]
    pub use crate::frequency::CountMinSketch;

    #[cfg(all(feature = "frequency", feature = "std"))]
    pub use crate::frequency::SpaceSaving;

    #[cfg(feature = "membership")]
    pub use crate::membership::BloomFilter;

    #[cfg(feature = "sampling")]
    pub use crate::sampling::ReservoirSampler;

    #[cfg(feature = "statistics")]
    pub use crate::statistics::RunningStats;
}

#[cfg(feature = "cardinality")]
pub use cardinality::HyperLogLog;

#[cfg(feature = "quantiles")]
pub use quantiles::TDigest;

#[cfg(feature = "frequency")]
pub use frequency::CountMinSketch;

#[cfg(all(feature = "frequency", feature = "std"))]
pub use frequency::SpaceSaving;

#[cfg(feature = "membership")]
pub use membership::BloomFilter;

#[cfg(feature = "sampling")]
pub use sampling::ReservoirSampler;

#[cfg(feature = "statistics")]
pub use statistics::RunningStats;