bytehaul 0.1.3

Async HTTP download library with resume, multi-connection, rate limiting, and checksum verification
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114

# bytehaul

[![Tests](https://img.shields.io/github/actions/workflow/status/triwinds/bytehaul/test.yml?branch=main&logo=githubactions&label=tests)](https://github.com/triwinds/bytehaul/actions/workflows/test.yml)
[![Crates.io](https://img.shields.io/crates/v/bytehaul?logo=rust)](https://crates.io/crates/bytehaul)
[![Docs.rs](https://img.shields.io/docsrs/bytehaul?logo=docs.rs)](https://docs.rs/bytehaul)
[![PyPI](https://img.shields.io/pypi/v/bytehaul?logo=pypi)](https://pypi.org/project/bytehaul/)
[![Python](https://img.shields.io/pypi/pyversions/bytehaul?logo=python&logoColor=white)](https://pypi.org/project/bytehaul/)
[![License](https://img.shields.io/github/license/triwinds/bytehaul)](LICENSE)

A Rust async HTTP download library with Python bindings (also available on PyPI), supporting resume, multi-connection downloads, write-back cache, rate limiting, and checksum verification.

## Documentation

- [简体中文 README](docs/README.zh-CN.md)
- [Advanced Usage (Rust)](docs/advanced.md) | [进阶用法](docs/advanced.zh-CN.md)
- [Python Bindings Guide](bindings/python/README.md) | [Python 使用文档](docs/python.zh-CN.md)

## Features

- **Single & multi-connection downloads** — automatic Range probing and fallback
- **Pause / resume** — cooperative pause with persisted control files for later continuation
- **Write-back cache** — piece-based aggregation to reduce random I/O
- **Memory budget & backpressure** — semaphore-based flow control
- **Retry with exponential backoff** — configurable max retries, respects `Retry-After`
- **Rate limiting** — shared token-bucket across all workers
- **SHA-256 checksum verification** — post-download integrity check
- **Cancellation** — cooperative cancel via stop signal
- **Progress reporting** — real-time speed, ETA, downloaded bytes, and state
- **Shared network configuration** — proxy, custom DNS servers, and IPv6 toggle on the downloader client

## Installation

### Rust

Add `bytehaul` to your project via Cargo:

```bash
cargo add bytehaul
```

Or add it manually to your `Cargo.toml`:

```toml
[dependencies]
bytehaul = "0.1.3"
```

### Python

```bash
pip install bytehaul
```

Requires Python 3.9+. A single wheel per platform covers all supported Python versions (abi3).

## Quick Start (Rust)

```rust
use bytehaul::{DownloadSpec, Downloader};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let downloader = Downloader::builder().build()?;

    let spec = DownloadSpec::new("https://example.com/largefile.zip")
        .output_path("largefile.zip");

    let handle = downloader.download(spec);
    handle.wait().await?;
    println!("Download complete!");
    Ok(())
}
```

For configuration, progress monitoring, cancellation, and more, see the [Advanced Usage Guide](docs/advanced.md).

If you omit `output_path`, bytehaul will automatically choose a filename from `Content-Disposition`, then the URL path, and finally `download`. You can combine that with `.output_dir("downloads")` to control the destination directory. Absolute `output_path` values are still accepted when `output_dir` is not set.

## Quick Start (Python)

```python
import bytehaul

# Simple one-line download
bytehaul.download("https://example.com/file.bin", output_path="output.bin")

# Automatic filename detection into a directory
bytehaul.download("https://example.com/file.bin", output_dir="downloads")

# With options
bytehaul.download(
    "https://example.com/file.bin",
    output_path="output.bin",
    max_connections=8,
    max_download_speed=1_000_000,  # 1 MB/s
)
```

For the full Python API (object API, progress, cancellation, error handling, etc.), see the [Python Bindings Guide](bindings/python/README.md).

## Architecture

```
DownloadManager
  └─ DownloadSession
       ├─ Scheduler (piece assignment, segment reclamation)
       ├─ HttpWorker ×N (Range requests, retry)
       │    └─ channel ─→ Writer (WriteBackCache → FileWriter)
       └─ ControlStore (atomic save/load/delete)
```

## License

MIT. See [LICENSE](LICENSE).