bytehaul 0.1.1

Async HTTP download library with resume, multi-connection, rate limiting, and checksum verification
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104

# bytehaul

A Rust async HTTP download library with resume, multi-connection, write-back cache, rate limiting, and checksum verification.

## Documentation

- [简体中文 README](docs/README.zh-CN.md)
- [Advanced Usage (Rust)](docs/advanced.md) | [进阶用法](docs/advanced.zh-CN.md)
- [Python Bindings Guide](bindings/python/README.md) | [Python 使用文档](docs/python.zh-CN.md)

## Features

- **Single & multi-connection downloads** — automatic Range probing and fallback
- **Resume / breakpoint continuation** — control file persistence with atomic save
- **Write-back cache** — piece-based aggregation to reduce random I/O
- **Memory budget & backpressure** — semaphore-based flow control
- **Retry with exponential backoff** — configurable max retries, respects `Retry-After`
- **Rate limiting** — shared token-bucket across all workers
- **SHA-256 checksum verification** — post-download integrity check
- **Cancellation** — cooperative cancel via watch channel
- **Progress reporting** — real-time speed, downloaded bytes, state
- **Shared network configuration** — proxy, custom DNS servers, and IPv6 toggle on the downloader client

## Installation

### Rust

Add `bytehaul` to your project via Cargo:

```bash
cargo add bytehaul
```

Or add it manually to your `Cargo.toml`:

```toml
[dependencies]
bytehaul = "0.1"
```

### Python

```bash
pip install bytehaul
```

Requires Python 3.9+. A single wheel per platform covers all supported Python versions (abi3).

## Quick Start (Rust)

```rust
use bytehaul::{DownloadSpec, Downloader};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let downloader = Downloader::builder().build()?;

    let spec = DownloadSpec::new(
        "https://example.com/largefile.zip",
        "largefile.zip",
    );

    let handle = downloader.download(spec);
    handle.wait().await?;
    println!("Download complete!");
    Ok(())
}
```

For configuration, progress monitoring, cancellation, and more, see the [Advanced Usage Guide](docs/advanced.md).

## Quick Start (Python)

```python
import bytehaul

# Simple one-line download
bytehaul.download("https://example.com/file.bin", "output.bin")

# With options
bytehaul.download(
    "https://example.com/file.bin",
    "output.bin",
    max_connections=8,
    max_download_speed=1_000_000,  # 1 MB/s
)
```

For the full Python API (object API, progress, cancellation, error handling, etc.), see the [Python Bindings Guide](bindings/python/README.md).

## Architecture

```
DownloadManager
  └─ DownloadSession
       ├─ Scheduler (piece assignment, segment reclamation)
       ├─ HttpWorker ×N (Range requests, retry)
       │    └─ channel ─→ Writer (WriteBackCache → FileWriter)
       └─ ControlStore (atomic save/load/delete)
```

## License

MIT. See [LICENSE](LICENSE).