load 0.2.0

Load generator for benchmark tasks with Zipf distribution
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117

# Zipf Distribution for Load Testing

## Overview

[Zipf distribution](https://en.wikipedia.org/wiki/Zipf%27s_law) models real-world access patterns where a small number of items account for the majority of requests. This pattern appears frequently in:

- Web cache access patterns (hot data)
- Database query frequency distributions
- File system access patterns
- Social media content popularity

Using zipf-distributed workloads in benchmarking creates realistic test scenarios that closely mirror production traffic patterns.

## Mathematical Foundation

The Zipf distribution follows a power law, $P(x) = C \cdot x^{-s}$, where $s > 0$ is the
shape parameter (higher values create steeper distributions) and $C$ is a normalization
constant. Values are drawn using
[inverse transform sampling](https://en.wikipedia.org/wiki/Inverse_transform_sampling).

See [formulas.md](formulas.md) for the full derivation — the
normalization constant, the inverse-transform step, and the $s = 1$ special case.


## Usage Examples

### Basic Zipf Generation

```rust
use load::zipf::Zipf;

// Create a zipf generator for range [1, 100] with shape parameter 1.1
let zipf = Zipf::new(1.0..100.0, 1.1).unwrap();

// Generate zipf-distributed values
let value1 = zipf.sample(0.1);  // Low u → high rank (popular items)
let value2 = zipf.sample(0.9);  // High u → low rank (unpopular items)

// Assert formatted values (4 decimal places)
assert_eq!(format!("{:.4}", value1), "1.4565");
assert_eq!(format!("{:.4}", value2), "56.6416");
```

### Load Testing Scenario

```rust
use load::zipf::Zipf;

// Generate 10,000 accesses to a 1,000-item cache
// with zipf distribution (s=1.2 for realistic web traffic)
let cache_accesses: Vec<usize> = Zipf::indices_access(1..1001, 1.2).unwrap()
    .take(10)  // Number of accesses
    .collect();

// Assert exact sequence with deterministic seed
assert_eq!(cache_accesses, vec![2, 13, 6, 1, 223, 546, 4, 121, 2, 2]);
```

### Custom RNG for Reproducible Tests

```rust
use load::zipf::{Zipf, ZipfIterator};

// Create reproducible workload for benchmarking
let zipf = Zipf::new(1.0..100.0, 1.5).unwrap();
let mut iter = ZipfIterator::with_seed(zipf, 42);
let workload: Vec<usize> = iter.map(|x| x as usize).take(5).collect();

// Assert exact sequence with seed 42
assert_eq!(workload, vec![3, 3, 5, 2, 1]);
```

### Streaming Processing (Memory Efficient)

```rust
use load::zipf::Zipf;
use std::collections::HashMap;

// Process large workloads without allocating all indices upfront
let mut hit_counts = HashMap::new();

for index in Zipf::indices_access(1..11, 1.2).unwrap().take(100) {
    *hit_counts.entry(index).or_insert(0) += 1;

    // Process each access immediately (e.g., cache lookup, database query)
    // This approach uses constant memory regardless of workload size
}

let most_accessed = hit_counts.iter().max_by_key(|&(_, count)| count);
assert_eq!(most_accessed, Some((&1, &38)));
```

## Performance Benchmarks

Benchmark results for Zipf distribution operations:

```text
running 8 tests
test bench_zipf_array_access   ... bench:          15.27 ns/iter (+/- 0.38)
test bench_zipf_batch_16       ... bench:         120.66 ns/iter (+/- 3.01)
test bench_zipf_batch_64       ... bench:         471.88 ns/iter (+/- 10.56)
test bench_zipf_indices_access ... bench:          14.92 ns/iter (+/- 0.69)
test bench_zipf_iter           ... bench:          14.37 ns/iter (+/- 0.35)
test bench_zipf_sample_s_1     ... bench:           0.27 ns/iter (+/- 0.02)
test bench_zipf_sample_s_1_07  ... bench:           0.27 ns/iter (+/- 0.02)
test bench_zipf_sample_s_2     ... bench:           0.27 ns/iter (+/- 0.02)
```

## Parameters Guide

- **Range [a, b]**: Defines the output range. For array indices, use `[1, array_length+1]`
- **Shape parameter s**: Controls distribution steepness
  - `s = 1.0`: Classical Zipf (web requests, word frequency)
  - `s = 1.2`: Typical web cache patterns
  - `s = 2.0`: Very steep distribution (80/20 rule becomes 95/5)
- **Array length**: Size of the dataset being accessed
- **Iterator control**: Use `.take(n)` to limit the number of accesses, or process indefinitely