rget-cli 0.3.1

High-performance resumable download manager
Documentation

Install

Requires Rust 1.85+ and ~/.cargo/bin on your PATH.

cargo install rget-cli

Published as rget-cli, installs a command called rget. The bare rget name on crates.io belongs to an unrelated project last published in 2017. Before the first release, install from git instead: cargo install --git https://github.com/cesarferreira/rget

Verify:

rget --help

git clone https://github.com/cesarferreira/rget.git
cd rget
cargo install --path . --locked
# or
make install-release

Debug install (faster compile, larger binary):

make install

Run without installing:

make build-release
./target/release/rget

Quickstart

Give it a URL. It works out the filename, uses as many connections as the server supports, and writes straight into the destination file.

rget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.7.tar.xz

  ↓ linux-6.7.tar.xz
  █████████████████████████▍░░░░░░░░░░░░░░░░░░   58.0%
  78.3 MiB / 135 MiB · 91.5 MiB/s · ETA 1s
  8 connections · 18/32 chunks

The bar advances in eighth-of-a-character steps, so it moves smoothly rather than jumping a whole cell at a time, and the block redraws in place instead of scrolling. Colour is on when stderr is a terminal and off otherwise — piping, or setting NO_COLOR, gives you clean text with no escape codes at all.

The first time you run it, rget asks where downloads should go, prefilled with your system's Downloads folder:

Where should rget save downloads?
  Folder [~/Downloads]:

Press Enter to accept, or type somewhere else. It only asks once. --dir overrides it for a single download, and rget config --dir <path> changes it for good. If stdin is not a terminal — a script, a pipeline, --json, --quiet — it never asks and quietly uses the system Downloads folder, so automation behaves identically to a terminal.

If anything interrupts it, run the same command again. No --resume flag, no .part files to clean up:

⟳ Resuming linux-6.7.tar.xz
  36.70 MiB of 135 MiB already downloaded
  ✓ SHA-256 verified
  ✓ linux-6.7.tar.xz
    135 MiB · in 5s · 26.4 MiB/s average

That survives Ctrl+C, SIGKILL, a dropped network, a closed terminal and a reboot. Progress is kept in a central SQLite database, so it survives across runs and across machines-worth-of-downloads.

Common uses

# Name it yourself, or pick a directory
rget URL -o ubuntu.iso
rget URL --dir ~/Downloads

# Verify what you got
rget URL --sha256 da1ed7d47c97ed72c9354091628740aa3c40a3c9cd7382871f3cedbd60588234

# Tune the transfer
rget URL --connections 16 --limit 20MiB/s --timeout 30s

# Mirrors of the same file: failed ranges retry against whichever is healthy
rget https://mirror1/f.iso https://mirror2/f.iso --sha256 <digest>

# Authentication and custom headers
rget URL --user alice:secret
rget URL --header 'Authorization: Bearer …' --header 'X-Trace: 1'

Managing downloads

rget list shows the same bar per download, with the status colour-coded so a long list reads at a glance:

ID       FILE                       PROGRESS          STATUS
58a243   linux-6.7.tar.xz           ██████████  100%  complete
9f21bc   dataset.tar                ████░░░░░░   42%  paused
17ef21   model.bin                  ███████░░░   71%  failed

rget config                # where downloads go, and where state lives
rget config --dir ~/ISOs
rget config --reset        # be asked again next time

rget list                  # everything this machine knows about
rget info a82fd1           # validators, ranges, progress, last error
rget resume a82fd1         # continue one, with no other flags needed
rget resume --all          # continue everything that was interrupted
rget forget a82fd1         # drop the metadata; never touches the file
rget forget --all          # drop every download's metadata
rget forget --all --files  # drop metadata and delete the files too

Scripting

--json writes one structured event per line to stdout; human progress always goes to stderr, and turns itself off when stderr is not a terminal.

rget URL --json | jq -c 'select(.event == "download_completed")'

{"event":"download_started","filename":"linux-6.7.tar.xz","total_size":141975800,"connections":8,"parallel":true}
{"event":"download_completed","downloaded":141975800,"elapsed_ms":5012,"average_bps":17196544}

RUST_LOG=debug rget URL explains its decisions: range assignment, retries, resume reasoning and validator mismatches. Credentials are never logged.

How it works

Concern Approach
Parallelism The file is split into byte ranges; workers pwrite straight into the destination at the right offset. No temp files, no merge pass, so a 500 GB download needs 500 GB of disk and finishing costs nothing.
Memory One body chunk per connection, streamed to disk. Peak memory is independent of file size.
Resume Progress lives in SQLite in your platform's data dir (~/.local/share/rget/downloads.db, ~/Library/Application Support/rget/downloads.db). Settings live in the same database, so there is one file to move or delete.
Where files land The system Downloads folder by default — on Linux that is XDG_DOWNLOAD_DIR, which is localised, so it is read rather than guessed.
Safety Before reusing a byte, rget re-validates the remote with ETag/Last-Modified/size and checks the local file is still the same file. If the remote changed, it refuses rather than splicing two versions together.
Slow servers Ranges are subdivided on the fly, so one slow connection does not hold up the tail of a download.
Failure Per-range retries with exponential backoff and jitter, honouring Retry-After. One range failing never discards another's progress.

Durability

The interesting part. write() returning Ok does not mean the bytes survive power loss, but a SQLite commit does — so recording progress before flushing data would let the database claim a range is complete when the file has a hole in it. rget therefore puts a durability barrier between the two:

worker writes ──▶ fdatasync(dest) ──▶ SQLite COMMIT
[claim nothing]   [bytes durable]     [claim only pre-barrier bytes]

Persisted state is always a subset of what is on disk. The worst a crash can cost is one commit interval of re-downloaded bytes — never a corrupt file.

The barrier runs at ~2 Hz rather than per chunk, so it does not cap throughput. Full rationale, including what this does not protect against, is in docs/CRASH_CONSISTENCY.md.

Testing

rget is tested against a purpose-built hostile HTTP server rather than the public internet. It can ignore Range, lie about Content-Length, send malformed Content-Range, hang up mid-body, dribble bytes, change its ETag mid-download, and answer with 429/500/502/503 plus Retry-After.

cargo nextest run                 # unit + integration + property tests
cargo nextest run --run-ignored only  # the soak test: repeated random kills

The resume tests SIGKILL the real binary mid-transfer, restart it, and require the finished file to match the source cryptographically. Range management is property-tested over randomised complete/fail/split/restart sequences, asserting that ranges never overlap, never gap, and never extend past the content length.

Development

Module layout mirrors the concerns above — engine orchestrates, scheduler plans ranges, worker moves bytes, storage persists, resume reconciles, ui renders, and nothing else knows what a terminal is.

Common tasks via the Makefile:

make              # check + build + test
make build        # debug build
make build-release
make install      # install debug binary
make install-release
make run ARGS="--help"
make check        # cargo check + clippy
make fmt          # format
make lint         # fmt check + clippy
make test
make clean
make demo         # install + show --help

Releasing (requires cargo-release and git-cliff):

make release                  # default minor bump
make release LEVEL=patch      # patch bump
make release LEVEL=major      # major bump

The pre-release hook regenerates CHANGELOG.md with git-cliff from your conventional-commit history (grouped into Features, Bug Fixes, etc. per cliff.toml) and commits it alongside the version bump. Pushing the resulting v* tag triggers the release workflow, which builds the multi-platform binaries and publishes a GitHub Release whose notes are generated by git-cliff from the same config.

License

MIT