Crate ripget 

Fast, multi-part downloads with a simple API.

ripget prioritizes speed by downloading large files in parallel with HTTP range requests. The default configuration uses 10 parallel ranges and 16MB buffers, similar in spirit to aria2c.

Features

• Download files as fast as possible using HTTP multiplexing
• Overwrites existing output files by default
• Interactive CLI progress bar in terminals
• Automatic retry with exponential backoff for network throttling or disconnects
• Per-range idle timeout reconnects after 15 seconds without data
• Configurable parallelism with simple overrides
• Windowed streaming mode for sequential consumers
• Async library API powered by tokio and reqwest

Install

cargo install ripget

CLI usage

ripget "https://example.com/assets/large.bin"

Override the buffer size:

ripget --cache-size 8mb "https://example.com/assets/large.bin"

Override the output name:

ripget "https://example.com/assets/large.bin" my_file.blob

Environment overrides

• RIPGET_THREADS: override the default parallel range count
• RIPGET_USER_AGENT: override the HTTP user agent
• RIPGET_CACHE_SIZE: override the read buffer size (e.g. 8mb)

CLI options

• --threads <N>: override the default parallel range count
• --user-agent <UA>: override the HTTP user agent
• --silent: disable the progress bar
• --cache-size <SIZE>: override the read buffer size (e.g. 8mb)

Library usage

let report = ripget::download_url(
    "https://example.com/assets/large.bin",
    "large.bin",
    None,
    None,
)
.await?;
println!("downloaded {} bytes", report.bytes);

Override the user agent programmatically:

let options = ripget::DownloadOptions::new()
    .user_agent(format!("my-app/{}", env!("CARGO_PKG_VERSION")));
let report = ripget::download_url_with_options(
    "https://example.com/assets/large.bin",
    "large.bin",
    options,
)
.await?;
println!("downloaded {} bytes", report.bytes);

Windowed streaming (double-buffered range download):

let options = ripget::WindowedDownloadOptions::new(10 * 1024 * 1024)
    .threads(8);
let mut stream = ripget::download_url_windowed(
    "https://example.com/assets/large.bin",
    options,
)
.await?;
let mut file = tokio::fs::File::create("large.bin").await?;
tokio::io::copy(&mut stream, &mut file).await?;
let report = stream.finish().await?;
println!("streamed {} bytes", report.bytes);

Windowed streaming uses two in-memory buffers sized at window_size / 2 (total resident memory ~= window_size, plus HTTP read buffers). The reader consumes directly from the current cold buffer; when it drains, it waits for the hot buffer to finish and then swaps without extra copies. If the stream is dropped or finish() is called early, the background download is cancelled and finish() returns a report for the bytes read.

For async readers with a known length:

use tokio::io::AsyncWriteExt;
let data = b"hello from a stream".to_vec();
let (mut tx, rx) = tokio::io::duplex(64);
let data_clone = data.clone();
tokio::spawn(async move {
    let _ = tx.write_all(&data_clone).await;
});
let report = ripget::download_reader(rx, "out.bin", data.len() as u64).await?;
println!("downloaded {} bytes", report.bytes);

Retry behavior

ripget retries network failures and most HTTP statuses with exponential backoff to handle throttling or transient outages. Only 404 and 500 responses are treated as fatal. Each range reconnects if no data arrives for 15 seconds. If the server does not support range requests, ripget logs a warning and falls back to a single-threaded download.

Limitations

• The server should report the full size. When range requests are unsupported, ripget falls back to a single-threaded download.

License

Licensed under either of:

• Apache License, Version 2.0 (LICENSE-APACHE)
• MIT license (LICENSE-MIT)

Structs

DownloadOptions

Configuration for URL downloads.

DownloadReport

Information about a completed download.

StreamReport

Information about a completed windowed stream download.

WindowedDownload

Reader for a windowed URL download.

WindowedDownloadOptions

Configuration for windowed URL downloads.

Enums

RipgetError

Error type for ripget operations.

Constants

BUFFER_SIZE

Fixed read buffer size used for streaming data.

DEFAULT_THREADS

Default number of parallel ranges used by ripget.

Traits

ProgressReporter

Reports download progress for integrations like CLI progress bars.

Functions

download_reader

Copy an async reader into a file path.

download_reader_with_progress

Copy an async reader into a file path.

download_url

Download a URL to a file path using parallel range requests.

download_url_windowed

Download a URL as a sequential reader using a hot/cold window.

download_url_with_options

Download a URL to a file path using parallel range requests with options.

download_url_with_progress

Download a URL to a file path using parallel range requests with progress.

Type Aliases

Progress

Shared progress reporter handle.

Result

Result type for ripget operations.