ZeroProto

ZeroProto is a zero-copy binary serialization format designed for high-performance Rust applications. It provides schema-based code generation with compile-time type safety and runtime performance that rivals hand-optimized protocols.

Features

• Zero-Copy Deserialization - Read data directly from buffers without allocation
• Schema-Based Code Generation - Define messages in .zp schema files
• Type-Safe API - Generated Rust code with compile-time guarantees
• High Performance - Optimized for speed with minimal overhead
• Memory Safe - No unsafe code in public APIs
• no_std Support - Works in embedded environments
• Cross-Platform - Little-endian format for consistency
• Rich Type System - Primitives, strings, bytes, vectors, and nested messages

Quick Start

Installation

Add ZeroProto to your Cargo.toml:

[dependencies]

zeroproto = "0.2.0"



[build-dependencies]

zeroproto-compiler = "0.2.0"

Define a Schema

Create a schemas/user.zp file:

message User {
    id: u64;
    username: string;
    email: string;
    age: u8;
    friends: [u64];
    profile: Profile;
}

message Profile {
    bio: string;
    avatar_url: string;
    settings: UserSettings;
}

message UserSettings {
    theme: Theme;
    notifications_enabled: bool;
    max_friends: u32;
}

enum Theme {
    Light = 0;
    Dark = 1;
    Auto = 2;
}

Generate Code

Create a build.rs file:

fn main() -> Result<(), Box<dyn std::error::Error>> {
    zeroproto_compiler::build()?;
    Ok(())
}

Use Generated Types

mod generated;
use generated::user::*;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create a user
    let mut builder = UserBuilder::new();
    builder.set_id(12345);
    builder.set_username("alice");
    builder.set_email("alice@example.com");
    builder.set_age(25);
    
    let mut friends = vec![1001, 1002, 1003];
    builder.set_friends(&friends);
    
    let mut profile_builder = ProfileBuilder::new();
    profile_builder.set_bio("Software developer");
    profile_builder.set_avatar_url("https://example.com/avatar.jpg");
    
    let mut settings_builder = UserSettingsBuilder::new();
    settings_builder.set_theme(Theme::Dark);
    settings_builder.set_notifications_enabled(true);
    settings_builder.set_max_friends(500);
    
    let settings_data = settings_builder.finish();
    profile_builder.set_settings(&settings_data);
    
    let profile_data = profile_builder.finish();
    builder.set_profile(&profile_data);
    
    let user_data = builder.finish();
    
    // Read the user (zero-copy!)
    let user = UserReader::from_slice(&user_data)?;
    println!("User: {}", user.username());
    println!("Email: {}", user.email());
    println!("Age: {}", user.age());
    
    // Access nested data
    let profile = user.profile()?;
    println!("Bio: {}", profile.bio());
    
    let settings = profile.settings()?;
    println!("Theme: {:?}", settings.theme());
    println!("Notifications: {}", settings.notifications_enabled());
    
    // Iterate over friends
    let friends_reader = user.friends()?;
    for friend_id in friends_reader.iter() {
        println!("Friend ID: {}", friend_id?);
    }
    
    Ok(())
}

Documentation

• API Documentation
• Binary Format Specification
• Schema Language Guide
• Performance Benchmarks
• Migration Guide

Architecture

ZeroProto consists of several crates:

• zeroproto - Core runtime library with readers and builders
• zeroproto-compiler - Schema compiler and code generator with hand-written recursive descent parser
• zeroproto-macros - Procedural macros for derive support
• zeroproto-cli - Command-line interface for development

Compiler Pipeline

The compilation process follows these steps:

1. Parsing - Hand-written recursive descent parser parses .zp schema files
2. Validation - AST validation ensures schema correctness and type safety
3. IR Generation - Abstract Syntax Tree is lowered to Intermediate Representation
4. Code Generation - Rust code is generated from IR using proc_macro2 and quote

Binary Format

+----------------------+---------------------------+
| Field Count (u16)    | Field Table (N entries)   |
+----------------------+---------------------------+
| Offset-to-Field-0    | Offset-to-Field-1 ...     |
+----------------------+---------------------------+
| Payload Section                                  |
+--------------------------------------------------+

Each field entry contains:

• Type ID (1 byte): Primitive type identifier
• Offset (4 bytes): Absolute offset to field data

CLI Usage

Compile Schemas

# Compile a single schema file

zeroproto compile schemas/user.zp --output src/generated


# Compile all schemas in a directory

zeroproto compile schemas/ --output src/generated


# Watch for changes and recompile

zeroproto watch schemas/ --output src/generated


# Validate schemas without generating code

zeroproto check schemas/


# Initialize a new project

zeroproto init my-project

Project Templates

# Create a new ZeroProto project

zeroproto init my-project --current-dir


# This creates:

# - Cargo.toml with dependencies

# - build.rs for compilation

# - schemas/ directory

# - src/main.rs with example

# - README.md with setup instructions

Performance

ZeroProto is designed for maximum performance:

Benchmarks performed on Intel i7-9700K, Rust 1.75, message size ~100 bytes

Zero-Copy Benefits

• No Allocation - Deserialization doesn't allocate memory
• No Copying - Data is read directly from input buffer
• Cache Friendly - Sequential memory access patterns
• Predictable Performance - Consistent timing regardless of data size

Testing

Run the test suite:

# Run all tests

cargo test


# Run with coverage

cargo tarpaulin --out Html


# Run benchmarks

cargo bench


# Check formatting

cargo fmt --check


# Run lints

cargo clippy -- -D warnings

Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Setup

# Clone the repository

git clone https://github.com/zeroproto/zeroproto.git

cd zeroproto


# Install development dependencies

cargo install cargo-watch cargo-typst


# Run tests

cargo test


# Run benchmarks

cargo bench

Code Style

• Use rustfmt for formatting
• Follow the official Rust style guide
• Add documentation for all public APIs
• Include examples in documentation
• Write tests for new functionality

Roadmap

Version 0.2.0 (Planned)

• Schema evolution support
• Custom field attributes
• Enum variant values
• Default field values
• Optional fields
• Oneof fields
• Map types
• Schema validation improvements

Version 0.3.0 (Planned)

• Compression support
• Streaming serialization
• Async I/O support
• Reflection API
• Schema registry integration
• Protocol adapters (HTTP, gRPC)
• Language bindings (C++, Python, Go)

Long-term Goals

• WASM support
• Database integration
• Message routing
• Distributed systems support
• Real-time synchronization
• Cloud-native features

License

ZeroProto is licensed under the Apache License 2.0 or MIT License, at your choice.

See LICENSE-APACHE and LICENSE-MIT for details.

Acknowledgments

ZeroProto is inspired by existing serialization formats:

• Protocol Buffers - Schema evolution concepts
• FlatBuffers - Zero-copy design
• Cap'n Proto - Performance optimizations
• MessagePack - Compact binary format

Thank you to their creators and communities for paving the way!

Support

• Discord: Join our Discord - Chat with the community and get help

ZeroProto - Fast, Safe, Zero-Copy Serialization for Rust

Made with ❤️ by the ZeroProto community