Module backup 

Continuous backup of a local data directory to object storage, and restore to a point in time.

What is shipped. A background thread (the shipper) uploads, from the shared log (log format 3, [crate::wal]):

• every sealed segment (one the chain has moved past), as soon as the chain leaves it;
• the active segment’s durable prefix, every BackupOptions::upload_interval while it grows;
• every completed checkpoint of every shard, with the part of the run store it names that is not uploaded yet, and the history archives it covers.

Objects are immutable. A log chunk holds whole batches of one segment (<timeline>/log/<segment>/<offset>-<end>.wal); a store chunk holds a byte range of one run-store generation; small chunks are merged into larger ones as they accumulate (eight of one size make one of the next), so a slow log does not leave one object per upload. Nothing on the write path waits for any of this: the shipper reads segment files and run stores that are already durable, and the log only keeps segments the backup has not uploaded yet from being retired.

Restore points. Every BackupOptions::resolution while the log is written, the shipper reads the log’s acknowledged position and the time together; once the log is uploaded through that position, the pair is a restore point: restoring it rebuilds exactly the batches acknowledged at that time. A checkpoint is usable for a restore point when it completed before the position the point names was acknowledged (so it knows nothing the point does not), and a point is restorable when every shard has a usable checkpoint (or the log reaches back to its first record) and the log from the oldest one onward is uploaded without a gap.

Manifest. manifest.json at the root of the target is replaced atomically after each upload. It names a snapshot of the backup’s state (state/<version>.json, immutable) and the events since it (chunks, checkpoints, restore points). An object is uploaded completely before any manifest names it, and deleted only after a manifest that no longer names it is in place, so a crash at any moment leaves a target whose manifest names only complete objects; the next run deletes objects no manifest names and continues from what the manifest says.

Loss window. With upload_interval set (one second by default), at most the log acknowledged since the last upload is not in the backup: about one interval plus the upload’s own time. Without it, at most the active segment (up to the configured segment size). A clean shutdown uploads everything.

Retention. Restore points older than BackupOptions::retention are dropped, and so is every object no remaining point needs. Points older than ten minutes are thinned to one per minute; for points older than an hour one checkpoint per shard per hour is kept (a restore to such a point replays up to an hour of log).

Timelines. A backup continues where the manifest says only if the local log still holds the last batch uploaded, with the same checksum. Otherwise (a directory restored to an earlier point and backed up again to the same target, say) it starts a new timeline, so the two histories never mix. Restore picks the newest restore point at or before the requested time across timelines.

Server-mode and replicated directories are not covered.

Structs

Backup

A running backup (crate::Engine::backup).

BackupConfig

A backup target and its options, set with Config::backup.

BackupOptions

How a backup runs.

BackupStatus

What a running backup has done.

LocalTarget

A directory as a backup target: for tests, or a mounted volume.

Plan

What a restore of one point needs.

Point

A restore point: the log through batch .1, exactly what had been acknowledged at Unix time .0 (milliseconds).

RestoreOptions

What to restore.

RestorePoint

A position in the backup that can be restored: the log through batch seq, exactly what was acknowledged at time_ms.

Restored

What a restore rebuilt.

State

Timeline

One history of one data directory. A directory restored from a backup starts a new one.

Verified

A backup checked object by object.

Traits

Target

Object storage for backups.

Functions

format_time

Formats Unix milliseconds as RFC 3339 in UTC.

load_state

The backup’s manifest and its state, None for an empty target.

open_target

Opens a target from a command-line description: a directory path (or file:///path), or with the s3 feature s3://bucket/prefix (see [crate::backup::s3::S3Target::from_url]).

parse_time

Parses a time for --at: RFC 3339 (2026-11-09T12:00:00Z), Unix milliseconds (1762689600000), or a duration before now (-90s, -15m, -2h, -1d).

restore

Rebuilds dir from target at the newest restore point at or before opts.at_ms: a local data directory (log format 3) that opens and recovers to exactly what had been acknowledged at that time. dir must not exist or be empty.

verify

Reads every object the manifest names and checks it, and which restore points can be restored.