rargz 0.1.2

Fast parallel tar + zstd archiver and extractor with optional chunked format.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100

# rargz

[![Crates.io](https://img.shields.io/crates/v/rargz.svg)](https://crates.io/crates/rargz)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)
[![Rust](https://img.shields.io/badge/Rust-stable-orange.svg)](https://www.rust-lang.org)

`rargz` is a fast parallel tar + zstd archiver. It walks a directory, emits a deterministic tar stream, and compresses chunks concurrently with independent zstd workers.

## Features

- Parallel tar stream generation with independent zstd workers
- Optional progress indicator (auto-disables when stderr is not a TTY)
- Produces standard `.tar.zst` by default (compatible with `tar --zstd` or `zstd -d`)
- Opt-in `.rargz` chunked format with cooperative parallel decompression
- Streaming extraction (`stdin` to filesystem) with automatic format detection
- Metadata inspection via `--list` and `--count`

## Installation

```bash
cargo install rargz
```

From source:

```bash
cargo install --path .
```

## Archiving

Create a standard `.tar.zst` archive and write it to stdout:

```bash
rargz path/to/input > archive.tar.zst
```

Enable the chunked `.rargz` format:

```bash
rargz --format rargz path/to/input > archive.rargz
```

Tune chunk size and thread count:

```bash
rargz --chunk-size 4MiB --jobs 8 path/to/input > archive.tar.zst
```

Disable the progress spinner explicitly when scripting:

```bash
rargz --no-progress path/to/input > archive.tar.zst
```

## Extraction

Extraction reads from stdin (or a path argument) and writes the output to the directory specified with `-o/--output`:

```bash
rargz --extract archive.tar.zst -o ./output
```

Or stream over stdin:

```bash
cat archive.tar.zst | rargz --extract -o ./output
```

`rargz` auto-detects plain `.tar.zst` streams versus chunked `.rargz` streams. Standard archives are decompressed sequentially for compatibility. Chunked archives fan out to the worker pool for parallel decompression and untar.

Progress is enabled by default; disable it with `--no-progress` or by piping stderr.

## Inspecting archives

Scan metadata without unpacking by either piping the archive or passing it as an argument:

```bash
rargz --list archive.tar.zst
rargz --count archive.rargz
```

`--list` prints each entry path (similar to `tar -tf`), while `--count` emits a single total. Both modes skip extracting payloads and work with either format. When no path is supplied, the commands read from stdin (so you can still pipe data in).

## Format details

- `.tar.zst` mode produces a pure tar stream compressed with zstd. Fully compatible with standard tools:
  - `tar --zstd -xf archive.tar.zst`
  - `zstd -d archive.tar.zst | tar -xf -`

- `.rargz` mode adds a small header (`RARGZ\0`, version, chunk size) and length-prefixes each compressed chunk. The extra framing allows `rargz` to decompress different chunks in parallel during extraction or inspection.

## Safety

- Parent directory components (`..`) are removed from archive paths to prevent extraction from writing outside the target tree.
- Any error in the pipeline cancels all worker threads immediately.

## License

MIT. See `LICENSE` for details.