rivet-cli 0.7.7

Rivet: PostgreSQL/MySQL → Parquet/CSV (local, S3, GCS). Crate name rivet-cli; binary rivet.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91

# Local Filesystem Destination

## Config block

```yaml
destination:
  type: local
  path: ./output                    # directory for output files
```

`path` can be absolute (`/data/exports`) or relative to the working directory.

Rivet creates the directory if it does not exist.

## Output filenames

Files are named automatically:

```
{export_name}_{YYYYMMDD}_{HHMMSS}.{format}
```

Examples:
- `users_daily_20260406_120000.parquet`
- `orders_incremental_20260406_120000.csv.zst`

For chunked exports, each chunk appends `_chunk{N}`:
- `orders_chunked_20260406_120000_chunk0.parquet`

## File splitting

For large exports, split output into multiple files:

```yaml
exports:
  - name: big_table
    query: "SELECT * FROM big_table"
    mode: full
    format: parquet
    max_file_size: "256MB"          # split when file exceeds this size
    destination:
      type: local
      path: ./output
```

Parts are named: `big_table_20260406_120000_part000.parquet`, `...part001.parquet`, etc.

Accepted size suffixes: `KB`, `MB`, `GB` (case-insensitive).

## Compression

Compression is applied before writing to disk:

```yaml
exports:
  - name: users
    query: "SELECT * FROM users"
    mode: full
    format: parquet
    compression: zstd               # default for Parquet
    compression_level: 3            # optional: 1 (fast) to 22 (smallest)
    destination:
      type: local
      path: ./output
```

| Format  | Default compression | Options |
|---------|-------------------|---------|
| Parquet | `zstd` | `zstd`, `snappy`, `gzip`, `lz4`, `none` |
| CSV     | `zstd` | `zstd`, `gzip`, `none` |

CSV with compression produces `.csv.zst` or `.csv.gz`.

## Verify

```bash
rivet doctor --config my_export.yaml
```

Output:

```
[OK] Destination './output' — directory writable
```

## List exported files

```bash
rivet state files --config my_export.yaml
rivet state files --config my_export.yaml --export users_daily --last 5
```