Expand description
§sulfite-tools
CLI for S3 built on the sulfite library. Supports listing, single-object ops, and batch ops from CSV.
§Overview
sulfite is a high-level S3 client built on AWS SDK for Rust for even better ease of use, reliability, and bandwidth saturation (>50 Gbps).
The name: SO3^2-, an anion, implying a companion to some other cation (application), is commonly used as a preservative in wines and dried fruits (preserve to S3). It’s S3 with an O in the middle, a play on oxidization.
§Installation
cargo install sulfite-tools§Usage
Global options (can be used with any subcommand):
--region,-r— AWS region (or region of custom endpoint)--endpoint-url,-e— S3 endpoint URL (e.g. for MinIO)--max-retries— Maximum retries per request (default: 3)--retriable-client-status-codes— HTTP status codes to treat as retriable (comma-separated; default: 408,429)--read-timeout— Read timeout in seconds for the HTTP client (default: 60)
§Subcommands
| Command | Description |
|---|---|
list | List objects in a bucket (optional prefix/suffix), output keys to CSV or stdout |
head | Get metadata (HEAD) for one object |
download | Download one object (single request) |
download-multipart | Download one object (multipart transfer) |
upload | Upload one object (single request) |
upload-multipart | Upload one object (multipart transfer) |
delete | Delete one object |
copy | Copy one object from source to destination |
restore | Restore one object from archival storage (e.g. Glacier) |
csv | Run one operation per key from a CSV file (batch) |
§Examples
# List keys in a bucket with prefix, write to CSV
sulfite list -b my-bucket -p my/prefix/ -o keys.csv
# Get metadata (HEAD) for one object
sulfite head -b my-bucket -k path/to/object
# Download an object
sulfite download -b my-bucket -k path/to/object -l local-file
# Upload with multipart (large files)
sulfite upload-multipart -b my-bucket -k path/to/object -l local-file
# Use custom endpoint (e.g. MinIO)
sulfite -e https://minio.example.com list -b my-bucket -p ""§CSV workflow
Use list to write a manifest of keys to a CSV file, then use the csv subcommand to run batch operations (e.g. download all objects into a directory):
# 1. List keys under a prefix and write to a manifest CSV
sulfite list -b my-bucket -p my/prefix/ -o manifest.csv
# 2. Download every key in the manifest to a local directory (keys become paths under that dir)
sulfite csv manifest.csv --has-header download -b my-bucket -p my/prefix/ -l ./downloadedUse --column-idx/-c N if the key column is not the first (0-based index).
CSV skip behavior — For csv download and csv upload, an item is skipped if the destination already exists with the same size and a destination timestamp that is not older than the source. Otherwise the existing file or object is overwritten. This avoids re-transferring unchanged files when re-running a batch.
Multipart activation — In CSV batch mode, download uses a single GET for objects under 1 GB and multipart for ≥ 1 GB. Upload uses single-part for under 1 GB and multipart for ≥ 1 GB. The single-object commands download / upload always use one request; download-multipart / upload-multipart always use multipart.
Archival and small files — When you specify an archival storage class (e.g. --storage-class GLACIER) on csv upload or csv copy, objects under 16 KB are stored as STANDARD instead of the requested class, for efficiency.
Run sulfite --help or sulfite <command> --help for full options.