Crate serdeio 

A lightweight Rust library for seamless serialization/deserialization of Serde-compatible structs across JSON, JSON Lines, CSV, YAML, MessagePack, and TOML formats.

Install

cargo add serdeio

SerdeIO supports JSON and JSON Lines formats by default. Additional formats can be enabled with feature flags:

# For CSV support
cargo add serdeio --features csv

# For YAML support
cargo add serdeio --features yaml

# For MessagePack support
cargo add serdeio --features messagepack

# For TOML support
cargo add serdeio --features toml

# For both CSV and YAML
cargo add serdeio --features csv,yaml

Supported Formats

Format	Extensions	Single Record	Multiple Records	Feature Flag
JSON	.json	✓	✓	(default)
JSON Lines	.jsonl, .jsl	✗	✓	(default)
CSV	.csv	✗	✓	csv
YAML	.yaml, .yml	✓	✓	yaml
MessagePack	.msgpack, .mpack, .mpk	✓	✓	messagepack
TOML	.toml	✓	✗	toml

Features

• Lightweight: Minimal dependencies with feature-gated optional formats
• Auto-detection: File format automatically determined from extensions (case-insensitive)
• Iterator support: Efficient streaming writes without collecting into vectors
• Serde-compatible: Works with any struct that implements Serde traits
• Flexible: Supports both single records and collections across formats

Performance

• Uses BufReader/BufWriter internally for optimal I/O performance
• Iterator-based writing enables memory-efficient streaming without allocations
• Format detection and parsing optimized for common use cases

API Overview

SerdeIO provides 8 main functions for reading and writing data:

Reader-based functions:

• read_record_from_reader<T>(reader, data_format) - Read a single record from any Read implementation
• read_records_from_reader<T>(reader, data_format) - Read multiple records as Vec<T> from any Read
• write_record_to_writer<T>(writer, record, data_format) - Write a single record to any Write
• write_records_to_writer<T>(writer, records, data_format) - Write multiple records using an iterator

File-based functions:

• read_record_from_file<T>(path, data_format?) - Read a single record, auto-detecting format from file extension
• read_records_from_file<T>(path, data_format?) - Read multiple records, auto-detecting format from file extension
• write_record_to_file<T>(path, record, data_format?) - Write a single record, auto-detecting format from file extension
• write_records_to_file<T>(path, records, data_format?) - Write multiple records, accepts any iterator or collection

Note: Some formats like CSV and JSON Lines only support multiple records (Vec<T>). File-based functions accept an optional DataFormat override; if not provided or set to Auto, the format is inferred from the file extension.

Examples

Reading and Writing Multiple Records (Common Use Case)

This example reads a JSON file containing multiple user records and converts it to JSON Lines format:

use anyhow::{Context, Result as AnyResult};
use serde::{Deserialize, Serialize};
use serdeio::{read_records_from_file, write_records_to_writer, DataFormat};

#[derive(Debug, Deserialize, Serialize)]
struct User {
    id: u32,
    name: String,
    age: u8,
    items: Vec<String>,
}

pub fn main() -> AnyResult<()> {
    let input_file_path = "examples/users.json";

    // Read JSON file to memory (format auto-detected from .json extension)
    let users: Vec<User> = read_records_from_file(input_file_path, DataFormat::Auto)
        .context("Failed to read records from file")?;

    // Write to stdout in JSON Lines format
    let writer = std::io::stdout();
    write_records_to_writer(writer, users.iter(), DataFormat::JsonLines)?;

    Ok(())
}

API Reference

For complete API documentation, visit docs.rs/serdeio.

Key types:

• DataFormat - Enum for specifying data formats
• Error - Comprehensive error type with format-specific variants

Contributing

Contributions are welcome! Please:

• Run tests with cargo test
• Format code with cargo fmt
• Check with cargo clippy
• Follow the existing code style and patterns

License

SerdeIO is licensed under the MIT License.

Enums

DataFormat

Supported data formats for serialization and deserialization.

Error

Functions

read_record_from_file

Reads a single record from a file and deserializes it into the specified type.

read_record_from_reader

Reads a single record from a reader and deserializes it into the specified type.

read_records_from_file

Reads multiple records from a file and deserializes them into a vector of the specified type.

read_records_from_reader

Reads multiple records from a reader and deserializes them into a vector of the specified type.

write_record_to_file

Writes a single record to a file in the data format inferred from the file extension.

write_record_to_writer

Writes a single record to a writer in the specified data format.

write_records_to_file

Writes multiple records to a file in the data format inferred from the file extension.

write_records_to_writer

Writes multiple records to a writer in the specified data format.