hurley 0.1.5

A curl-like HTTP client with performance testing capabilities
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146

# Hurley: A High-Performance HTTP Client and Load Testing Tool Engineered in Rust

This article examines the technical architecture, capabilities, and use cases of **hurley**, a project developed in Rust that functions as both a general-purpose HTTP client and a performance testing tool. It explores the efficiency advantages gained by managing API testing and performance analysis through a unified tool within software development processes.

---

## 1. Introduction and Motivation

With the proliferation of microservices architectures and distributed systems, communication via the HTTP protocol has become the lifeblood of the software ecosystem. In this context, developers face two fundamental needs: (1) A flexible HTTP client to verify the functional correctness of API endpoints, and (2) Performance testing tools to analyze system behavior under load.

Typically, distinct toolsets are employed for these two requirements (specialized HTTP clients vs. load testing tools like `wrk` or `Apache Benchmark`). **hurley** aims to minimize context switching in development and testing processes and offer a unified testing experience by consolidating these two functions into a single command-line interface (CLI).

---

## 2. Core Capabilities and HTTP Client Mode

hurley features a client mode that demonstrates full compliance with modern HTTP standards. It supports all fundamental operations required by RESTful architectures.

### 2.1. Protocol Support and Request Structure

The tool supports all standard HTTP methods (GET, POST, PUT, DELETE, PATCH, HEAD). Request configuration can be flexibly structured via command-line arguments:

*   **Header Management:** Defining custom HTTP headers using the `-H` parameter.
*   **Payload Management:** Inline data submission via the `-d` parameter or file-based data streaming via the `-f` parameter.
*   **Redirect Policies:** Automatic tracking of HTTP 3xx series responses using the `-L` parameter.

```bash
# Example: POST request containing custom headers and payload
hurley -X POST https://api.example.com/v1/resource \
  -H "Content-Type: application/json" \
  -H "X-Client-ID: system-a" \
  -d '{"key": "value", "timestamp": 1678900000}'
```

---

## 3. Performance Testing and Load Simulation

A distinguishing feature of the tool is its ability to instantaneously transform existing HTTP requests into a load test without requiring external configuration.

### 3.1. Concurrency Model

hurley is built upon Rust's `Tokio` asynchronous runtime. This architecture allows for the management of a high number of concurrent connections while utilizing system resources (CPU and Memory) at minimal levels. The intensity of the test scenario is determined by the `-c` (concurrency) and `-n` (total requests) parameters.

### 3.2. Parameterized and Dataset-Based Testing

To simulate real-world traffic patterns, hurley supports data-driven test scenarios. Users can provide a data file (CSV or JSON) containing rows of variables, and write request templates using `{{column_name}}` placeholders.

This approach is critical for eliminating the misleading effects of cache mechanisms (cache warming bias) and measuring the general stability of the system with realistic, varying inputs.

*   **Data Files:** Supported via the `--data-file` flag, accepting CSV files with headers or JSON arrays of objects.
*   **Variable Substitution:** Placeholders like `{{user_id}}` can be used in the URL path, HTTP headers, and the request body.
*   **Row Cycling:** During a load test, requests sequentially cycle through the data rows, ensuring all data is exercised deterministically.

```bash
# Example: Parameterized load test using a CSV data file
hurley -X POST https://api.example.com/users/{{user_id}} \
  -H "Authorization: Bearer {{api_token}}" \
  -d '{"role": "{{role}}"}' \
  --data-file users.csv -c 10 -n 1000
```

```json
/* Example JSON Dataset Schema (Alternative to CSV) */
[
  { "user_id": 101, "api_token": "abc...", "role": "admin" },
  { "user_id": 102, "api_token": "def...", "role": "user" }
]
```

---

## 4. Performance Metrics and Statistical Analysis

In reporting test results, the tool goes beyond average values to present statistical distribution analyses. **Percentile** metrics are vital for detecting tail latency.

The fundamental metrics reported include:

*   **Throughput:** The number of requests processed per second (RPS).
*   **Latency Distribution:**
    *   **P50 (Median):** The completion time for 50% of requests.
    *   **P95 and P99:** System performance in the slowest 5% and 1% segments. These values are critical indicators for Service Level Agreement (SLA) compliance.
    *   **Jitter:** The standard deviation and range of variation in response times.
    *   **Endpoint Breakdown:** Detailed performance metrics separated for each endpoint (when using datasets).

```text
📊 Statistical Summary
   Total Requests:      1000
   Error Rate:          0.00%
   Requests/sec:        450.25

📈 Latency Distribution (Percentiles)
   p50 (Median):        45.12 ms
   p95:                120.45 ms
   p99:                210.88 ms

📍 Endpoint Breakdown (Details)
   GET /api/users/101
   └── p95: 110.20 ms
   POST /api/orders
   └── p95: 145.50 ms
```

---

## 5. Technical Architecture

The project is constructed upon the performance and reliability-oriented libraries of the Rust ecosystem:

*   **Asynchronous I/O:** Non-blocking I/O operations via `tokio` and `reqwest` libraries.
*   **Statistical Computation:** High dynamic range histogram analysis via the `hdrhistogram` library.
*   **Error Management:** Deterministic management of runtime errors via a strong type system and the `thiserror` library.

These architectural choices ensure that the tool delivers performance at the C/C++ level without compromising memory safety.

---

## 6. Installation

hurley can be easily installed via Cargo, Rust's package manager, or built from source.

### Via Cargo (Recommended)

```bash
cargo install hurley
```

### From Source

```bash
git clone https://github.com/dursunkoc/hurley.git
cd hurley
cargo build --release
```

The binary will be available at `target/release/hurley`.

---

## 7. Conclusion

hurley is a unified tool designed to remove the barriers between "functional testing" and "performance testing" processes in the modern API development cycle. By combining the performance advantages offered by the Rust language with a user-friendly interface, it offers strong analysis capabilities to developers and system engineers.

The project continues to be developed as open source, with plans to add features such as distributed testing capabilities and HTTP/3 support to the roadmap.

Project Source Code: [https://github.com/dursunkoc/hurley](https://github.com/dursunkoc/hurley)