Crate protocrap 

Protocrap

A small, efficient, and flexible protobuf implementation for Rust.

Overview

Protocrap takes a different approach than other protobuf libraries. Instead of generating parsing and serialization code for each message type, it uses a single table-driven implementation. Code generation produces only struct definitions with accessors and static lookup tables.

This design yields:

• Small binaries: No code duplication across message types
• Fast compilation: No macro expansion or monomorphization explosion
• Flexible memory: Arena allocation with custom allocator support
• Universal streaming: Push-based API works with sync and async

Quick Start

Code Generation

Generate Rust code from .proto files using protocrap-codegen:

# Create descriptor set with protoc
protoc --include_imports --descriptor_set_out=types.bin my_types.proto

# Generate Rust code
protocrap-codegen types.bin src/types.pc.rs

Include the generated code in your crate:

use protocrap;
include!("types.pc.rs");

Encoding Messages

use protocrap::{ProtobufRef, ProtobufMut, arena::Arena};
use protocrap::google::protobuf::FileDescriptorProto;
use allocator_api2::alloc::Global;

let mut arena = Arena::new(&Global);
let mut msg = FileDescriptorProto::ProtoType::default();
msg.set_name("example.proto", &mut arena);
msg.set_package("my.package", &mut arena);

// Encode to a Vec<u8>
let bytes = msg.encode_vec::<32>().unwrap();

// Or encode to a fixed buffer
let mut buffer = [0u8; 1024];
let encoded = msg.encode_flat::<32>(&mut buffer).unwrap();

The const generic (::<32>) specifies the maximum message nesting depth.

Decoding Messages

use protocrap::{ProtobufRef, ProtobufMut, arena::Arena};
use protocrap::google::protobuf::FileDescriptorProto;
use allocator_api2::alloc::Global;

// First encode a message to get some bytes
let mut arena = Arena::new(&Global);
let mut original = FileDescriptorProto::ProtoType::default();
original.set_name("example.proto", &mut arena);
let bytes = original.encode_vec::<32>().unwrap();

// Decode from a byte slice
let mut decoded = FileDescriptorProto::ProtoType::default();
decoded.decode_flat::<32>(&mut arena, &bytes);
assert_eq!(decoded.name(), "example.proto");

Runtime Reflection

Inspect messages dynamically without compile-time knowledge of the schema:

use protocrap::{ProtobufRef, arena::Arena};
use protocrap::google::protobuf::{FileDescriptorProto, DescriptorProto};
use protocrap::descriptor_pool::DescriptorPool;
use allocator_api2::alloc::Global;

// Build descriptor pool from the library's own file descriptor
let mut pool = DescriptorPool::new(&Global);
let file_desc = FileDescriptorProto::ProtoType::file_descriptor();
pool.add_file(file_desc);

// Encode a real DescriptorProto (the descriptor for DescriptorProto itself)
let descriptor = DescriptorProto::ProtoType::descriptor_proto();
let bytes = descriptor.encode_vec::<32>().unwrap();

// Decode dynamically using the pool
let mut arena = Arena::new(&Global);
let msg = pool.decode_message(
    "google.protobuf.DescriptorProto",
    &bytes,
    &mut arena,
).unwrap();

// Access fields dynamically
for field in msg.descriptor().field() {
    if let Some(value) = msg.get_field(field.as_ref()) {
        println!("{}: {:?}", field.name(), value);
    }
}

Architecture

Arena Allocation

All variable-sized data (strings, bytes, repeated fields, sub-messages) is allocated in an arena::Arena. This provides:

• Speed: Allocation is a pointer bump in the common case
• Bulk deallocation: Drop the arena to free all messages at once
• Custom allocators: Pass any &dyn Allocator to control memory placement

use protocrap::arena::Arena;
use allocator_api2::alloc::Global;

let mut arena = Arena::new(&Global);
// All allocations during decode/set operations use this arena
// When arena drops, all memory is freed

Push-Based Streaming

The parser uses a push model: you provide data chunks, it returns updated state. This signature (state, buffer) -> updated_state enables:

• Single implementation for sync and async
• No callback traits or complex lifetime requirements
• Works in embedded, WASM, and any async runtime

Generated Code Structure

For each protobuf message, the codegen produces a module containing:

• ProtoType: The message struct with #[repr(C)] layout
• Accessor methods following protobuf conventions

Field accessors follow this pattern:

Proto Type	Getter	Setter	Other
Scalar	field() -> T	set_field(T)	has_field(), clear_field()
String/Bytes	field() -> &str/&[u8]	set_field(&str, &mut Arena)	has_field(), clear_field()
Message	field() -> Option<&M>	field_mut() -> &mut M	has_field(), clear_field()
Repeated	field() -> &[T]	field_mut() -> &mut RepeatedField<T>	add_field(...)

Modules

• arena: Arena allocator for message data
• containers: Collection types (containers::RepeatedField, containers::String, containers::Bytes)
• reflection: Runtime message inspection and dynamic decoding
• TypedMessage: Wrapper for repeated message elements

Feature Flags

• std (default): Enables std::io integration, Vec-based encoding
• serde_support (default): Enables serde serialization via reflection
• nightly: Use nightly Rust features for slightly better codegen (branch hints)

For no_std environments, disable default features:

[dependencies]
protocrap = { version = "0.1", default-features = false }

Restrictions

Protocrap is designed for “sane” schemas:

• Up to 256 optional fields per message
• Struct sizes up to 64KB
• Field numbers 1-2047 (1 or 2 byte wire tags)
• Field numbers should be mostly consecutive

The following are intentionally unsupported:

• Unknown fields: Discarded during decoding (no round-trip preservation)
• Extensions: Proto2 extensions are silently dropped
• Maps: Decoded as repeated key-value pairs
• Proto3 zero-value omission: All set fields are serialized

Modules

arena

Arena allocator for protobuf message data.

containers

Collection types for protobuf messages.

descriptor_pool

Dynamic message pool for runtime schema handling.

google

proto_json

Proto JSON serialization support.

reflection

Runtime reflection for protobuf messages.

serde

tests

Structs

TypedMessage

A typed non-null message pointer for repeated fields. Implements Deref<Target=T> so &[TypedMessage<T>] can be used like &[&T].

Enums

Error

Errors that can occur during protobuf encoding/decoding operations.

Traits

Allocator

An implementation of Allocator can allocate, grow, shrink, and deallocate arbitrary blocks of data described via Layout.

ProtobufMut

Mutable protobuf operations (decode, deserialize).

ProtobufRef

Read-only protobuf operations (encode, serialize, inspect).