SPUD (Structured Payload of Unintelligible Data)

SPUD is a custom binary file format for efficient storage and transmission of structured data. It uses type tags and field name interning for compactness. This implementation is written in Rust.

Features

• Compact Binary Representation: Binary encoding for all supported data types.
• Supported Data Types: null, bool, signed/unsigned integers (i8–i64, u8–u64), f32, f64, String, and raw binary blobs (Vec<u8>).
• Field Name Interning: Field names are mapped to unique IDs to reduce redundancy.
• Versioning: Files start with a version string for compatibility.
• Simple Structure: Version header, field name map, data payload, and EOF marker ([0xDE, 0xAD, 0xBE, 0xEF]).
• Serde Integration: Serialize/deserialize Rust structs via serde.

File Structure

A .spud file consists of:

1. Version String: UTF-8 bytes.
2. Field Name Map: Sequence of (length, field_name_bytes, id) entries, ending with 0x01.
3. Data Payload: Sequence of (field_id, type_tag, value_bytes) entries. Strings/blobs include length. Arrays/objects are delimited by start/end tags.
4. EOF Marker: 0xDE, 0xAD, 0xBE, 0xEF.

Usage

Encoding (Writing a SPUD file)

You can build SPUD files manually or by serializing Rust structs with serde.

Manual Usage

```rust
use spud::spud_builder::SpudBuilder;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut builder = SpudBuilder::new();

    builder.object(|obj| {
        obj.field("name", "Example Object")?;
        obj.field("version", 1u8)?;
        obj.field("enabled", true)?;
        obj.field("description", Option::<String>::None)?;
        obj.field("value", 123.45f64)?;
        obj.field("raw_data", vec![0x01, 0x02, 0x03, 0x04])?;

        Ok(())
    })?;

    builder.build_file("output_dir", "my_spud_data")?;

    Ok(())
}

Serde Usage (TODO)

use spud::spud_builder::SpudBuilder;
use serde::Serialize;

#[derive(Serialize)]
struct MyData {
    name: String,
    version: u8,
    enabled: bool,
    description: Option<String>,
    value: f64,
    raw_data: Vec<u8>,
}

let data = MyData {
    name: "Example Object".to_string(),
    version: 1,
    enabled: true,
    description: None,
    value: 123.45,
    raw_data: vec![0x01, 0x02, 0x03, 0x04],
};

let mut builder = SpudBuilder::from_serde(&data);
builder.build_file("output_dir", "my_spud_data").unwrap();

Decoding (Reading a SPUD file)

You can decode SPUD files manually or deserialize them into Rust structs with serde.

Manual Usage

use spud::spud_decoder::SpudDecoder;

let mut decoder = SpudDecoder::new_from_path("output_dir/my_spud_data.spud").unwrap();
let data = decoder.decode().unwrap();
println!("{:?}", data);

Serde Usage

use spud::spud_decoder::SpudDecoder;
use serde::Deserialize;

#[derive(Deserialize, Debug)]
struct MyData {
    name: String,
    version: u8,
    enabled: bool,
    description: Option<String>,
    value: f64,
    raw_data: Vec<u8>,
}

let mut decoder = SpudDecoder::new_from_path("output_dir/my_spud_data.spud").unwrap();
let data: MyData = decoder.deserialize().unwrap();
println!("{:?}", data);

Roadmap / TODO

• Parallelism
• Support for nested objects
• serde integration for SpudBuilder and SpudDecoder
• spud!{} macro
• CLI tool for inspecting and converting .spud files

Known Issues

Some minor bugs may exist; please report any issues.

Contributing

Contributions are welcome! Open an issue or submit a pull request.

README.md partially generated by AI