toon-format-rs 0.1.0

# TOON Format - Token-Oriented Object Notation

[![Crates.io](https://img.shields.io/crates/v/toon-format-rs)](https://crates.io/crates/toon-format-rs)
[![Docs.rs](https://docs.rs/toon-format-rs/badge.svg)](https://docs.rs/toon-format-rs)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

A Rust implementation of the **TOON (Token-Oriented Object Notation)** format.

TOON is a compact, human-readable serialization format designed specifically to minimize tokens when sending structured data to Large Language Models (LLMs). It achieves **30-60% token reduction** compared to JSON for uniform arrays while maintaining full lossless round-trip capability.

## Features

- 🚀 **Compact**: YAML-like indentation + CSV-like tabular arrays
- 🔄 **Lossless**: JSON → TOON → JSON without data loss
- 📊 **Schema-aware**: Explicit array lengths `[N]` and field lists `{fields}`
- 🧠 **LLM-optimized**: Designed specifically for LLM prompts
- ⚡ **Fast**: Zero-copy parsing with minimal allocations
- 🔧 **Flexible**: Multiple delimiters (comma, tab, pipe)

## Quick Start

```rust
use toon_format_rs::{parse, to_string, Value};

// Parse TOON string
let input = r#"id: 123
name: Alice
active: true"#;

let value = parse(input).unwrap();
assert_eq!(value.get("id").unwrap().as_i64(), Some(123));

// Serialize to TOON
let output = to_string(&value).unwrap();
assert!(output.contains("id: 123"));
```

## Examples

### Simple Object

**TOON:**
```toon
id: 123
name: Alice
active: true
```

**Equivalent JSON:**
```json
{"id": 123, "name": "Alice", "active": true}
```

### Nested Object

**TOON:**
```toon
user:
  id: 1
  name: Alice
```

### Inline Array

**TOON:**
```toon
tags[3]: foo,bar,baz
```

**Equivalent JSON:**
```json
{"tags": ["foo", "bar", "baz"]}
```

### Tabular Array (Major Token Savings!)

**TOON:**
```toon
users[2]{id,name,role}:
  1,Alice,admin
  2,Bob,user
```

**Equivalent JSON:**
```json
{"users": [
  {"id": 1, "name": "Alice", "role": "admin"},
  {"id": 2, "name": "Bob", "role": "user"}
]}
```

### Complex Document

**TOON:**
```toon
context:
  task: Our favorite hikes together
  location: Boulder
  season: spring_2025
friends[3]: ana,luis,sam
hikes[3]{id,name,distanceKm,elevationGain,companion,wasSunny}:
  1,Blue Lake Trail,7.5,320,ana,true
  2,Ridge Overlook,9.2,540,luis,false
  3,Wildflower Loop,5.1,180,sam,true
```

## JSON Interoperability

```rust
use toon_format_rs::{json_to_toon, toon_to_json};

// Convert JSON to TOON
let json = r#"{"id":123,"name":"Alice"}"#;
let toon = json_to_toon(json).unwrap();

// Convert TOON back to JSON
let json_back = toon_to_json(&toon).unwrap();
```

## Installation

Add to your `Cargo.toml`:

```toml
[dependencies]
toon-format-rs = "0.1"
```

## Why TOON?

| Format | Tokens | Relative |
|--------|--------|----------|
| JSON | 4,587 | 100% |
| YAML | 3,749 | 82% |
| JSON Compact | 3,104 | 68% |
| **TOON** | **2,759** | **60%** |

*Benchmark: 100 uniform employee records*

## Specification

This implementation follows the [TOON Specification v3.0](https://github.com/toon-format/spec).

Key features:
- **Indentation-based objects**: No braces, YAML-like nesting
- **Explicit array headers**: `key[N]:` for inline arrays, `key[N]{fields}:` for tabular arrays
- **Minimal quoting**: Strings only quoted when necessary
- **Multiple delimiters**: Comma `,`, tab `\t`, or pipe `|`
- **Canonical numbers**: No scientific notation, no trailing zeros

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## Acknowledgments

- [TOON Format Organization](https://github.com/toon-format) for the specification
- Inspired by JSON, YAML, and CSV formats