Install
Requires Rust 1.85+ and ~/.cargo/bin on your PATH.
Published as
rget-cli, installs a command calledrget. The barergetname on crates.io belongs to an unrelated project last published in 2017. Before the first release, install from git instead:cargo install --git https://github.com/cesarferreira/rget
Verify:
# or
Debug install (faster compile, larger binary):
Run without installing:
Quickstart
Give it a URL. It works out the filename, uses as many connections as the server supports, and writes straight into the destination file.
↓ linux-6.7.tar.xz
█████████████████████████▍░░░░░░░░░░░░░░░░░░ 58.0%
78.3 MiB / 135 MiB · 91.5 MiB/s · ETA 1s
8 connections · 18/32 chunks
The bar advances in eighth-of-a-character steps, so it moves smoothly rather
than jumping a whole cell at a time, and the block redraws in place instead of
scrolling. Colour is on when stderr is a terminal and off otherwise — piping, or
setting NO_COLOR, gives you clean text with no escape
codes at all.
The first time you run it, rget asks where downloads should go, prefilled with
your system's Downloads folder:
Where should rget save downloads?
Folder [~/Downloads]:
Press Enter to accept, or type somewhere else. It only asks once. --dir
overrides it for a single download, and rget config --dir <path> changes it
for good. If stdin is not a terminal — a script, a pipeline, --json, --quiet
— it never asks and quietly uses the system Downloads folder, so automation
behaves identically to a terminal.
If anything interrupts it, run the same command again. No --resume flag,
no .part files to clean up:
⟳ Resuming linux-6.7.tar.xz
36.70 MiB of 135 MiB already downloaded
✓ SHA-256 verified
✓ linux-6.7.tar.xz
135 MiB · in 5s · 26.4 MiB/s average
That survives Ctrl+C, SIGKILL, a dropped network, a closed terminal and a
reboot. Progress is kept in a central SQLite database, so it survives across
runs and across machines-worth-of-downloads.
Common uses
# Name it yourself, or pick a directory
# Verify what you got
# Tune the transfer
# Mirrors of the same file: failed ranges retry against whichever is healthy
# Authentication and custom headers
Managing downloads
rget list shows the same bar per download, with the status colour-coded so a
long list reads at a glance:
ID FILE PROGRESS STATUS
58a243 linux-6.7.tar.xz ██████████ 100% complete
9f21bc dataset.tar ████░░░░░░ 42% paused
17ef21 model.bin ███████░░░ 71% failed
Scripting
--json writes one structured event per line to stdout; human progress
always goes to stderr, and turns itself off when stderr is not a terminal.
|
RUST_LOG=debug rget URL explains its decisions: range assignment, retries,
resume reasoning and validator mismatches. Credentials are never logged.
How it works
| Concern | Approach |
|---|---|
| Parallelism | The file is split into byte ranges; workers pwrite straight into the destination at the right offset. No temp files, no merge pass, so a 500 GB download needs 500 GB of disk and finishing costs nothing. |
| Memory | One body chunk per connection, streamed to disk. Peak memory is independent of file size. |
| Resume | Progress lives in SQLite in your platform's data dir (~/.local/share/rget/downloads.db, ~/Library/Application Support/rget/downloads.db). Settings live in the same database, so there is one file to move or delete. |
| Where files land | The system Downloads folder by default — on Linux that is XDG_DOWNLOAD_DIR, which is localised, so it is read rather than guessed. |
| Safety | Before reusing a byte, rget re-validates the remote with ETag/Last-Modified/size and checks the local file is still the same file. If the remote changed, it refuses rather than splicing two versions together. |
| Slow servers | Ranges are subdivided on the fly, so one slow connection does not hold up the tail of a download. |
| Failure | Per-range retries with exponential backoff and jitter, honouring Retry-After. One range failing never discards another's progress. |
Durability
The interesting part. write() returning Ok does not mean the bytes
survive power loss, but a SQLite commit does — so recording progress before
flushing data would let the database claim a range is complete when the file has
a hole in it. rget therefore puts a durability barrier between the two:
worker writes ──▶ fdatasync(dest) ──▶ SQLite COMMIT
[claim nothing] [bytes durable] [claim only pre-barrier bytes]
Persisted state is always a subset of what is on disk. The worst a crash can cost is one commit interval of re-downloaded bytes — never a corrupt file.
The barrier runs at ~2 Hz rather than per chunk, so it does not cap throughput.
Full rationale, including what this does not protect against, is in
docs/CRASH_CONSISTENCY.md.
Testing
rget is tested against a purpose-built hostile HTTP server rather than the
public internet. It can ignore Range, lie about Content-Length, send
malformed Content-Range, hang up mid-body, dribble bytes, change its ETag
mid-download, and answer with 429/500/502/503 plus Retry-After.
The resume tests SIGKILL the real binary mid-transfer, restart it, and require
the finished file to match the source cryptographically. Range management is
property-tested over randomised complete/fail/split/restart sequences, asserting
that ranges never overlap, never gap, and never extend past the content length.
Development
Module layout mirrors the concerns above — engine orchestrates, scheduler
plans ranges, worker moves bytes, storage persists, resume reconciles,
ui renders, and nothing else knows what a terminal is.
Common tasks via the Makefile:
Releasing (requires cargo-release and git-cliff):
The pre-release hook regenerates CHANGELOG.md with git-cliff from your conventional-commit history (grouped into Features, Bug Fixes, etc. per cliff.toml) and commits it alongside the version bump. Pushing the resulting v* tag triggers the release workflow, which builds the multi-platform binaries and publishes a GitHub Release whose notes are generated by git-cliff from the same config.
License
MIT