sheeter-engine 0.11.0

# Sheeter

Convert data files between formats from the command line. **Nine formats, seventy-two routes**, streaming where it matters, auto-detecting where it can.

Formats: `csv`, `tsv`, `json`, `ndjson`, `yaml`, `xml`, `toml`, `xlsx`, `sql`.

## Install

    cargo install sheeter-engine

The installed binary is `sheeter`.

## Usage

    sheeter convert --from <FMT> --to <FMT> [OPTIONS] <input> <output>

`--from` is optional: the format is detected from the file extension, or from content when reading stdin. Use `-` for stdin or stdout.

### The Unix pipeline use case

    # CSV from anywhere, filtered, as NDJSON, into jq
    cat data.csv | sheeter convert --to ndjson --filter "age > 30" - - | jq -c .

    # NDJSON from an API, as a CSV
    curl -s api.example.com/users | sheeter convert --to csv - - | head

    # Auto-detect anything from stdin
    some-tool emit | sheeter convert --to json - -

    # What format is this, anyway?
    sheeter detect mystery.bin

### Options

    --sheet NAME      Select one sheet (XLSX input only)
    --dialect NAME    SQL dialect: generic, postgresql, sqlite, mysql
    --filter EXPR     Filter rows (e.g. "age > 30 and city == 'SP'")
    --select COLS     Keep only these columns (comma-separated)
    --rename PAIRS    Rename columns (e.g. "old=new,foo=bar")
    --limit N         Keep at most N rows
    --sort SPEC       Sort by column(s) (e.g. "age desc,name")
    --dedup           Drop duplicate rows (whole row)
    --dedup-by COLS   Drop duplicates by column(s)

### Examples

    # CSV → JSON, file to file
    sheeter convert --from csv --to json data.csv data.json

    # CSV → CSV with filter, select, rename
    sheeter convert --from csv --to csv \
      --filter "age >= 30" \
      --select "name,age,salary" \
      --rename "salary=wage" \
      data.csv result.csv

    # Sort large CSV in bounded memory (external merge sort)
    sheeter convert --from csv --to csv --sort "age desc" big.csv sorted.csv

    # Multi-sheet XLSX → one CSV per sheet
    sheeter convert --from xlsx --to csv report.xlsx report.csv

    # Inspect any data file
    sheeter inspect --format xlsx report.xlsx

## How it works

Every reader produces a `CanonicalValue`; every writer consumes one. Adding
a new format costs one reader and one writer, not `N-1` pairwise converters.

## Streaming vs buffered

| Input → Output | Memory | Transforms |
|---|---|---|
| CSV/TSV → CSV/TSV (no `--sort`) | Constant | Per row |
| CSV/TSV → CSV/TSV (`--sort`) | Bounded by 50k-row chunk | With external sort |
| CSV/TSV ↔ NDJSON | Constant | Per row |
| NDJSON ↔ NDJSON | Constant | Per row |
| Any other pair | Full document in memory | Via hub round-trip |

`--sort` on non-CSV/TSV paths buffers and applies transforms but not sorting;
full buffered sort arrives in v0.12.0.

## How this compares

| Tool | Formats | Streaming | Auto-detect | Transforms |
|---|---|---|---|---|
| **Sheeter** | 9 | CSV/TSV/NDJSON | Yes | filter, select, rename, limit, sort, dedup |
| `dasel` | 7 | No | No | Limited |
| `yq` | 3 (YAML/JSON/XML) | No | No | jq syntax |
| `miller` | 5 | CSV/TSV/JSON | No | DSL |
| `qsv` | CSV only | Yes | Schema sniff | Rich |
| `dkit` | 6 | No | Yes | Limited |

## Versioning

This project follows [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html).
Changelog follows [Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/).
Commit messages follow [Conventional Commits 1.0.0](https://www.conventionalcommits.org/en/v1.0.0/).