dkit 0.3.0

Swiss army knife for data format conversion and querying
dkit-0.3.0 is not a library.
Visit the last successful build: dkit-0.9.0

dkit

Swiss army knife for data format conversion and querying.

Convert between JSON, CSV, YAML, TOML, XML, TSV, and MessagePack with a single CLI. Query nested data, compare files, preview as tables, and pipe everything together.

Quick Start

# Install
cargo install dkit

# Convert JSON to YAML
dkit convert data.json --to yaml

# Query nested data
dkit query config.yaml '.database.host'

# Preview CSV as a table
dkit view users.csv --limit 10

Installation

From crates.io

cargo install dkit

From source

git clone https://github.com/syangkkim/dkit.git
cd dkit
cargo install --path .

Supported Formats

Format Extensions Read Write
JSON .json O O
CSV .csv O O
TSV .tsv O O
YAML .yaml, .yml O O
TOML .toml O O
XML .xml O O
MessagePack .msgpack O O

All conversion paths between supported formats are available.

Commands

convert — Format conversion

# Basic conversion
dkit convert data.json --to yaml
dkit convert users.csv --to json
dkit convert config.yaml --to toml
dkit convert config.toml --to json

# Output to file
dkit convert data.json --to csv -o output.csv

# Batch conversion
dkit convert *.csv --to json --outdir ./converted/

# Pipe from stdin
cat data.json | dkit convert --from json --to csv

# Options
dkit convert data.json --to json --compact     # Minified JSON
dkit convert data.tsv --to json --delimiter '\t'  # TSV input
dkit convert data.csv --to json --no-header    # CSV without header

query — Data querying

# Field access
dkit query config.yaml '.database.host'
dkit query config.toml '.server.port'

# Nested path
dkit query data.json '.users[0].name'

# Array iteration
dkit query data.json '.users[].email'

# Negative indexing
dkit query data.json '.items[-1]'

Query syntax:

Syntax Description
.field Object field access
.field.sub Nested field access
.[0] Array index (0-based)
.[-1] Negative index (from end)
.[] Iterate all elements
where .field == value Filter with comparison (==, !=, >, <, >=, <=)
where .field contains "str" Filter with string operators (contains, starts_with, ends_with)
select .field1, .field2 Select specific fields
sort .field / sort .field desc Sort by field (ascending/descending)
limit N Limit number of results
| Pipeline chaining (pass results between operations) 
# Advanced query examples
dkit query data.json '.users[] | where .age > 20 | select .name, .email'
dkit query data.json '.items[] | sort .price desc | limit 5'
dkit query data.json '.users[] | where .name contains "Kim"'

# Output query results in different formats
dkit query data.json '.users[]' --to csv -o users.csv

view — Table preview

# View as table
dkit view users.csv

# Limit rows
dkit view large_data.csv --limit 20

# Navigate nested data
dkit view data.json --path '.users'

# Select columns
dkit view users.csv --columns name,email

stats — Data statistics

# Show overall statistics
dkit stats data.csv

# Navigate to nested data
dkit stats data.json --path .users

# Statistics for a specific column
dkit stats data.csv --column revenue

schema — Data structure inspection

# Show schema as a tree
dkit schema config.yaml
dkit schema data.json

# From stdin
cat data.json | dkit schema - --from json

diff — Compare data files

# Compare same-format files
dkit diff old.json new.json
dkit diff config_dev.yaml config_prod.yaml

# Cross-format comparison
dkit diff data.json data.yaml

# Compare nested path only
dkit diff old.json new.json --path '.database'

# Quiet mode (exit code: 0=same, 1=different)
dkit diff a.json b.json --quiet && echo 'same' || echo 'different'

merge — Combine multiple files

# Merge JSON files
dkit merge a.json b.json --to json

# Merge CSV files and convert to JSON
dkit merge users1.csv users2.csv --to json -o merged.json

# Merge YAML configs
dkit merge config1.yaml config2.yaml --to yaml

Comparison with Existing Tools

Feature dkit jq miller yq
JSON O O O O
CSV/TSV O X O X
YAML O X X O
TOML O X X X
XML O X X O
MessagePack O X X X
Cross-format convert O X Partial Partial
Table output O X O X
Query (where/select/sort) O O O O
Pipeline chaining O O O X
Statistics O X O X
Schema inspection O X X X
File merging O X O X
File diff O X X X
Single binary O O O O

dkit focuses on seamless conversion between all supported formats with a unified query syntax, eliminating the need for separate tools per format.

Building from Source

cargo build              # Build
cargo test               # Run tests
cargo clippy -- -D warnings  # Lint
cargo fmt -- --check     # Format check

Contributing

Contributions are welcome! Please see the GitHub Issues for planned features and known issues.

  1. Fork the repository
  2. Create a feature branch (git checkout -b feat/my-feature)
  3. Commit your changes
  4. Push to the branch and open a Pull Request

Please ensure cargo test and cargo clippy -- -D warnings pass before submitting.

License

MIT