balter 0.1.1

A load/stress testing framework.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74

![Balter](assets/balter.svg)

Balter, short for *Build A Load TestER*, is a load/stress testing framework for Rust designed to be flexible, efficient, and simple to use. Balter aims to minimize the conceptual overhead of load testing, and builds off of Tokio and the async ecosystem.

- See the [How To Use](#how-to-use) section for a guide on how to get started.
- See the [Limitations](#limitations) section for current things to be aware of (this is Beta software).
- See the [Developer Notes](#developer-notes) section for tips on modifying Balter.

## Features

- Run a given Scenario at a specific Transactions Per Second (TPS).
- Run a given Scenario to saturate the target (increase TPS until error rate is 3%).
- Distributed Scenario running; spin up multiple instances and they will coordinate [Experimental]

## WIP

- [ ] More robust distributed logic
- [ ] mTLS
- [ ] Peer discovery via DNS
- [ ] Autoscaling hooks

## Limitations

Balter is a Beta framework, which means there are some rough edges. This section lists the most important to be aware of. These are issues that are being worked on (with haste!) so hopefully this section will be gone soon :)

- The current statistical engine powering the TPS and Saturate functionality is not perfect
	- It cannot handle _changing_ loads well. It finds a set point and then sits there.
	- A highly variable measurement may cause it to find a bad set point.

- The distributed functionality is experimental.
	- Inefficient Gossip protocol being used
	- No transaction security (no TLS, mTLS, or WSS) - use at your own risk (and in a private VPC)
	- Likely to run into weird error cases

## How To Use

Balter is simple to get set up with. If you want to run a load test with a single Server, you include `balter = 0.1` in your Cargo.toml, and then:

```
use balter::prelude::*;

#[tokio::main]
async fn main() {
    my_scenario
        .tps(500)
        .duration(Duration::from_secs(30))
        .await;

    // OR

    my_scenario()
        .saturate()
        .duration(Duration::from_secs(30))
        .await;
}

#[scenario]
fn my_scenario() {
    my_transaction().await;
}

#[transaction]
fn my_transaction() -> Result<_, _> {
    // Some request logic
}
```

## Developer Notes

The Balter repository is set up to be easy to get started with development. It uses Nix to facilitate the environment setup via `shell.nix` (if you haven't yet drank the Nixaide, open up that file and it will give you an idea of the programs you'll want). The two most important for testing are [just](https://github.com/casey/just) and [cpulimit](https://github.com/opsengine/cpulimit) which are both used for running the test environment. (`Just` simplifies running the test scripts, but isn't actually needed)

Balter currently has most of its testing as integration tests, run via `just {test name}` which calls the test scripts in `test-scripts/`. Unfortunately, `cpulimit` is going to be based on the computer you're running on, so it might require tweaking some values in the `test-scripts/`. For a basic test to get you started, `just basic-saturate` will do.

The various integration test scripts all start up a `mock-service` (code in `mock-service/`) which is a simple HTTP server with a few endpoints to make it easier to test various functionality. `/api_10ms` is an endpoint which takes 10ms to respond. `/api_max_tps` is an endpoint which has a set MAX_TPS it accepts before responding with errors (and also coincidentally takes 10ms to respond).