sheeter-engine 1.2.0

Convert, inspect, and transform data files across ten formats with streaming where it matters and auto-detection where it can.
Documentation

Sheeter

Convert, inspect, and transform data files across ten formats — with streaming where it matters and auto-detection where it can.

Sheeter is a conversion-first tool with transforms you can chain. It is not a query language (use jq), not a dataframe (use polars), not a database client (use duckdb). It is what you reach for when you need to move data between formats and apply a quick filter, rename, sort, or dedupe while you are at it.

Formats: csv, tsv, json, ndjson, yaml, xml, toml, xlsx, ods, sql.

Status: feature-complete. v0.12.0 is the final minor release before v1.0.0. The CLI, Format enum, CanonicalValue variants, and convert/convert_workbook signatures are frozen. Remaining work before 1.0.0 is bug fixes and docs.

Install

cargo install sheeter-engine

The installed binary is sheeter.

Usage

sheeter convert --from <FMT> --to <FMT> [OPTIONS] <input> <output>

--from is optional: the format is detected from the file extension, or from content when reading stdin. Use - for stdin or stdout. ZIP-based formats (XLSX, ODS) are detected by inspecting the container.

SQL over files

sheeter query accepts a SELECT statement and resolves the FROM clause against the filesystem. Any format the engine reads can appear in FROM, so a single query can join an XLSX sheet with an NDJSON stream without conversion.

# Group and aggregate across CSV
sheeter query "SELECT country, SUM(revenue) AS total FROM users.csv GROUP BY country ORDER BY total DESC"

# Join XLSX with JSON
sheeter query "SELECT u.name, o.amount FROM users.xlsx u JOIN orders.json o ON u.id = o.user_id"

# Glob multiple files into one table
sheeter query 'SELECT * FROM "logs/*.json" ORDER BY ts DESC LIMIT 100'

The Unix pipeline use case

# CSV from anywhere, filtered, as NDJSON, into jq
cat data.csv | sheeter convert --to ndjson --filter "age > 30" - - | jq -c .

# NDJSON from an API, as a CSV
curl -s api.example.com/users | sheeter convert --to csv - - | head

# Auto-detect anything from stdin (including XLSX and ODS)
some-tool emit | sheeter convert --to json - -

# What format is this, anyway?
sheeter detect mystery.bin

Options

--sheet NAME      Select one sheet (XLSX/ODS input only)
--dialect NAME    SQL dialect: generic, postgresql, sqlite, mysql
--filter EXPR     Filter rows (e.g. "age > 30 and city == 'SP'")
--select COLS     Keep only these columns (comma-separated)
--rename PAIRS    Rename columns (e.g. "old=new,foo=bar")
--limit N         Keep at most N rows
--sort SPEC       Sort by column(s) (e.g. "age desc,name")
--dedup           Drop duplicate rows (whole row)
--dedup-by COLS   Drop duplicates by column(s)

Examples

# CSV → ODS, round-trip
sheeter convert --from csv --to ods data.csv data.ods
sheeter convert --from ods --to csv data.ods back.csv

# Multi-sheet XLSX → one CSV per sheet
sheeter convert --from xlsx --to csv report.xlsx report.csv

# Sort large CSV in bounded memory
sheeter convert --from csv --to csv --sort "age desc" big.csv sorted.csv

# Inspect any data file
sheeter inspect --format xlsx report.xlsx

# Display multiple files as formatted tables
sheeter view data.csv config.tsv report.xlsx

# Merge files into an XLSX workbook (one sheet per file)
sheeter xlsx:merge a.csv b.json c.yaml -o combined.xlsx

# Split every sheet of an XLSX into its own JSON file
sheeter xlsx:frag combined.xlsx

How it works

Every reader produces a CanonicalValue; every writer consumes one. Adding a new format costs one reader and one writer, not N-1 pairwise converters. CanonicalValue::Bytes carries binary columns from Parquet-capable readers (XLSX, future Parquet), round-tripping them through JSON via {"$binary": ...} and through SQL via X'hex'.

Streaming vs buffered

Input → Output Memory Transforms
CSV/TSV → CSV/TSV (no --sort) Constant Per row
CSV/TSV → CSV/TSV (--sort) Bounded by 50k-row chunk With external sort
CSV/TSV ↔ NDJSON Constant Per row
NDJSON ↔ NDJSON Constant Per row
Any other pair Full document in memory Via hub round-trip

How this compares

Tool Formats Streaming Auto-detect (incl. ZIP) Transforms
Sheeter 10 CSV/TSV/NDJSON Yes filter, select, rename, limit, sort, dedup
dasel 7 No No Limited
yq 3 (YAML/JSON/XML) No No jq syntax
miller 5 CSV/TSV/JSON No DSL
qsv CSV only Yes Schema sniff Rich
dkit 6 No Yes Limited

Versioning

This project follows Semantic Versioning 2.0.0. Changelog follows Keep a Changelog 1.1.0. Commit messages follow Conventional Commits 1.0.0.