babbel 0.3.0

Fast, modular, multi-format serialization and document processing toolkit for Rust
Documentation

babbel

CI License: MIT Rust Edition

The master facade crate for the Babbel multi-format serialization and document processing ecosystem. It provides unified ergonomics, re-exports all domain engines (JSON, TOML, YAML, XML, Bencode, CSV, TSV, INI, JSON Lines), provides dynamic format resolution via FormatRegistry, and powers open-ended, $O(N)$ cross-format conversion pipelines.


Features

  • Unified Prelude: Access all formats and text processing primitives with a single dependency.
  • Polyglot Parsing: Parse JSON, TOML, YAML, XML, Bencode, CSV/TSV, and INI into idiomatic typed representations.
  • Universal Cross-Conversion: Convert documents between 9 arbitrary formats without manual intermediate representations:
    • JSON $\leftrightarrow$ TOML $\leftrightarrow$ YAML $\leftrightarrow$ XML $\leftrightarrow$ Bencode
    • CSV / TSV $\leftrightarrow$ JSON $\leftrightarrow$ TOML $\leftrightarrow$ YAML
    • INI / .env $\leftrightarrow$ JSON $\leftrightarrow$ TOML $\leftrightarrow$ YAML
    • JSON Lines $\leftrightarrow$ JSON $\leftrightarrow$ TOML $\leftrightarrow$ CSV
  • Engine-Driven Pipelines (convert_format): Open-ended pipelines accepting any FormatEngine (OCP & DIP).
  • Dynamic Format Discovery: Look up format engines dynamically by format ID, MIME type, or file extension via FormatRegistry.
  • Zero-Allocation Embedded Module (babbel::embedded):
    • Stack allocation: StackBuffer<const N>, MemoryTracker, EmbeddedLimits, CompactError.
    • Zero-allocation destinations: SliceDestination and ArrayVecDestination.
    • Zero-allocation streaming pull parsers: JsonPullParser, TomlPullParser, XmlPullParser, CsvPullParser, and IniPullParser.
  • First-Class Text Processing:
    • RFC 4180 CSV & TSV parsing/emission with automatic delimiter sniffing and scalar type inference.
    • Section-based INI ([section]), Java .properties, and .env parsing.
    • Line-delimited JSON (.jsonl/.ndjson) streaming reader and writer.
    • Document frontmatter extraction (split_frontmatter for YAML --- and TOML +++).
  • Extensible Streaming I/O: Stream to and from files or memory buffers powered by babbel_core.
  • Fine-Grained Feature Flags: Enable only the formats your application requires.

Installation

Add babbel to your Cargo.toml:

[dependencies]
babbel = "0.3.0"

Or as a workspace path dependency:

[dependencies]
babbel = { path = "crates/babbel" }

Feature Flags

By default, all formats and the conversion pipeline are enabled (["std", "json", "toml", "yaml", "xml", "bencode", "msgpack", "cbor", "bson", "convert", "serde"]). To minimize binary size, enable only what you need:

Feature Description
std (default) Enables standard library I/O and OS integration.
alloc Enables heap allocation without full std (no_std environments).
json (default) Re-exports babbel_json (babbel::json, JsonEngine, pull parser).
toml (default) Re-exports babbel_toml (babbel::toml, TomlEngine, pull parser).
yaml (default) Re-exports babbel_yaml (babbel::yaml, YamlEngine).
xml (default) Re-exports babbel_xml (babbel::xml, XmlEngine, pull parser).
bencode (default) Re-exports babbel_bencode (babbel::bencode, BencodeEngine).
msgpack (default) Re-exports babbel_msgpack (babbel::msgpack, MsgPackEngine).
cbor (default) Re-exports babbel_cbor (babbel::cbor, CborEngine).
bson (default) Re-exports babbel_bson (babbel::bson, BsonEngine).
convert (default) Enables the babbel::convert cross-format conversion pipeline across all 12 formats.
serde (default) Enables universal serde serializing and deserializing for Value.

Quickstart

1. Document Parsing & Navigation

use babbel::json;
use babbel::toml;
use babbel::yaml;
use babbel::xml;
use babbel::bencode;
use babbel::{parse_csv, parse_ini, split_frontmatter, CsvOptions, IniOptions};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // 1. Parse JSON
    let json_doc = json::from_str(r#"{"service": "babbel", "port": 8080}"#)?;
    assert_eq!(json_doc.get("service").and_then(|n| n.as_str()), Some("babbel"));

    // 2. Parse TOML
    let toml_doc = toml::parse_string("service = \"babbel\"\nport = 8080\n")?;
    assert_eq!(toml_doc.get("service").and_then(|n| n.as_str()), Some("babbel"));

    // 3. Parse YAML
    let yaml_doc = yaml::parse_string("service: babbel\nport: 8080\n")?;
    assert_eq!(yaml_doc.get("service").and_then(|n| n.as_str()), Some("babbel"));

    // 4. Parse XML
    let xml_doc = xml::parse("<service port=\"8080\">babbel</service>")?;
    assert_eq!(xml_doc.get_root_element_name(), Some("service"));

    // 5. Parse Bencode
    let bencode_node = bencode::parse_bytes(b"d7:service6:babbel6:status6:activee")?;

    // 6. Parse CSV with type inference
    let csv_val = parse_csv("id,name\n1,Alice\n", &CsvOptions::default())?;
    assert_eq!(csv_val.as_array().unwrap().len(), 1);

    // 7. Parse INI with sections
    let ini_val = parse_ini("[app]\nname = Babbel\n", &IniOptions::default())?;
    assert!(ini_val.get("app").is_some());

    // 8. Split Markdown Frontmatter
    let doc = "---\ntitle: Guide\n---\n# Content";
    let parsed = split_frontmatter(doc);
    assert_eq!(parsed.content, "# Content");

    Ok(())
}

2. Universal Cross-Format Conversions

use babbel::convert::{convert_format, ConversionOptions};
use babbel_json::JsonEngine;
use babbel_toml::TomlEngine;
use babbel_yaml::YamlEngine;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Engine Pipeline: JSON <-> TOML
    let toml_output = convert_format(
        r#"{"title": "Babbel", "version": 1}"#,
        &JsonEngine,
        &TomlEngine,
        &ConversionOptions::pretty(),
    )?;

    // TOML <-> YAML
    let yaml_output = babbel::convert::toml_to_yaml(&toml_output)?;

    // TOML <-> Bencode
    let bencode_bytes = babbel::convert::toml_to_bencode(&toml_output)?;

    // CSV <-> JSON
    let csv_data = "id,name\n1,Alice\n2,Bob\n";
    let json_arr = babbel::convert::csv_to_json(csv_data)?;
    let csv_back = babbel::convert::json_to_csv(&json_arr)?;

    Ok(())
}

3. Dynamic Format Registry

use babbel::{default_registry, convert::convert_format, convert::ConversionOptions};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let registry = default_registry();
    if let (Some(from), Some(to)) = (registry.get_by_id("json"), registry.get_by_id("toml")) {
        let toml_doc = convert_format(
            r#"{"status": "ok"}"#,
            from.as_ref(),
            to.as_ref(),
            &ConversionOptions::default(),
        )?;
        println!("{}", toml_doc);
    }
    Ok(())
}

4. Zero-Allocation Streaming Pull Parsers

use babbel::embedded::{JsonPullParser, JsonPullEvent, JsonScalar};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut parser = JsonPullParser::new(r#"{"temp": 24.5, "sensor": "DHT22"}"#);
    while let Some(event) = parser.next_event()? {
        if let JsonPullEvent::Scalar(JsonScalar::Float(val)) = event {
            println!("Sensor value: {}", val);
        }
    }
    Ok(())
}

Sub-Libraries

Crate Link Description
babbel_core crates/babbel_core Core I/O traits, FormatEngine, universal Value AST, RFC 4180 CSV/TSV, INI/.env, frontmatter, BOM detection, and embedded primitives.
babbel_json crates/json RFC 6901 JSON Pointer, RFC 7396 Merge Patch, JSON Lines (.jsonl/.ndjson) streaming, and zero-allocation JsonPullParser.
babbel_toml crates/toml Strict TOML v1.1.0 parser/emitter with micro-allocations, typed RFC 3339 datetimes, and TomlPullParser.
babbel_yaml crates/yaml YAML 1.2 specification compliance, anchors, aliases, and custom tags.
babbel_xml crates/xml XML DOM, C14N Canonical XML, DTD validation, XSD schema, XPath 1.0, and XmlPullParser.
babbel_bencode crates/bencode Zero-copy borrowed parsing and streaming for BitTorrent Bencode.
babbel_msgpack crates/msgpack Fast, binary-safe MessagePack parser, serializer, and FormatEngine.
babbel_cbor crates/cbor RFC 8949 CBOR parser, serializer, indefinite streaming, and FormatEngine.
babbel_bson crates/bson BSON (Binary JSON) parser, serializer, document model, and FormatEngine.

Examples

Run any of the root examples using cargo:

# Full 8-format conversion pipeline (JSON -> TOML -> YAML -> XML -> Bencode -> MsgPack -> CBOR -> BSON)
cargo run --package babbel --example babbel_all_formats_roundtrip

# Binary formats size and byte structure comparison (JSON vs Bencode vs BSON vs CBOR vs MsgPack)
cargo run --package babbel --example binary_formats_comparison

Documentation

See the Documentation Hub for full workspace guides:


License

Licensed under the MIT License.