Crate protocrap 

Protocrap

A small, efficient, and flexible protobuf implementation for Rust.

Overview

Protocrap takes a different approach than other protobuf libraries. Instead of generating parsing and serialization code for each message type, it uses a single table-driven implementation. Code generation produces only struct definitions with accessors and static lookup tables.

This design yields:

• Small binaries: No code duplication across message types
• Fast compilation: No macro expansion or monomorphization explosion
• Flexible memory: Arena allocation with custom allocator support
• Universal streaming: Push-based API works with sync and async

Quick Start

Code Generation

Generate Rust code from .proto files using protocrap-codegen:

# Create descriptor set with protoc
protoc --include_imports --descriptor_set_out=types.bin my_types.proto

# Generate Rust code
protocrap-codegen types.bin src/types.pc.rs

Include the generated code in your crate:

use protocrap;
include!("types.pc.rs");

Encoding Messages

use protocrap::{ProtobufRef, ProtobufMut, arena::Arena};
use protocrap::google::protobuf::FileDescriptorProto;
use allocator_api2::alloc::Global;

let mut arena = Arena::new(&Global);
let mut msg = FileDescriptorProto::ProtoType::default();
msg.set_name("example.proto", &mut arena);
msg.set_package("my.package", &mut arena);

// Encode to a Vec<u8>
let bytes = msg.encode_vec::<32>().unwrap();

// Or encode to a fixed buffer
let mut buffer = [0u8; 1024];
let encoded = msg.encode_flat::<32>(&mut buffer).unwrap();

The const generic (::<32>) specifies the maximum message nesting depth.

Decoding Messages

use protocrap::{ProtobufRef, ProtobufMut, arena::Arena};
use protocrap::google::protobuf::FileDescriptorProto;
use allocator_api2::alloc::Global;

// First encode a message to get some bytes
let mut arena = Arena::new(&Global);
let mut original = FileDescriptorProto::ProtoType::default();
original.set_name("example.proto", &mut arena);
let bytes = original.encode_vec::<32>().unwrap();

// Decode from a byte slice
let mut decoded = FileDescriptorProto::ProtoType::default();
decoded.decode_flat::<32>(&mut arena, &bytes);
assert_eq!(decoded.name(), "example.proto");

Runtime Reflection

Inspect messages dynamically without compile-time knowledge of the schema:

use protocrap::{Protobuf, ProtobufRef, arena::Arena};
use protocrap::google::protobuf::{FileDescriptorProto, DescriptorProto};
use protocrap::descriptor_pool::DescriptorPool;
use allocator_api2::alloc::Global;

// Build descriptor pool from the library's own file descriptor
let mut pool = DescriptorPool::new(&Global);
let file_desc = FileDescriptorProto::ProtoType::file_descriptor();
pool.add_file(file_desc);

// Encode a real DescriptorProto (the descriptor for DescriptorProto itself)
let descriptor = DescriptorProto::ProtoType::descriptor_proto();
let bytes = descriptor.encode_vec::<32>().unwrap();

// Decode dynamically using the pool
let mut arena = Arena::new(&Global);
let msg = pool.decode_message(
    "google.protobuf.DescriptorProto",
    &bytes,
    &mut arena,
).unwrap();

// Access fields dynamically
for field in msg.descriptor().field() {
    if let Some(value) = msg.get_field(field.as_ref()) {
        println!("{}: {:?}", field.name(), value);
    }
}

Architecture

Arena Allocation

All variable-sized data (strings, bytes, repeated fields, sub-messages) is allocated in an arena::Arena. This provides:

• Speed: Allocation is a pointer bump in the common case
• Bulk deallocation: Drop the arena to free all messages at once
• Custom allocators: Pass any &dyn Allocator to control memory placement

use protocrap::arena::Arena;
use allocator_api2::alloc::Global;

let mut arena = Arena::new(&Global);
// All allocations during decode/set operations use this arena
// When arena drops, all memory is freed

Push-Based Streaming

The parser uses a push model: you provide data chunks, it returns updated state. This signature (state, buffer) -> updated_state enables:

• Single implementation for sync and async
• No callback traits or complex lifetime requirements
• Works in embedded, WASM, and any async runtime

Generated Code Structure

For each protobuf message, the codegen produces a module containing:

• ProtoType: The message struct with #[repr(C)] layout
• Accessor methods following protobuf conventions

Field accessors follow this pattern:

Proto Type	Getter	Setter	Other
Scalar	field() -> T	set_field(T)	has_field(), clear_field()
String/Bytes	field() -> &str/&[u8]	set_field(&str, &mut Arena)	has_field(), clear_field()
Message	field() -> Option<&M>	field_mut() -> &mut M	has_field(), clear_field()
Repeated	field() -> &[T]	field_mut() -> &mut RepeatedField<T>	add_field(...)

Modules

• arena: Arena allocator for message data
• base: Core message wrapper types (base::TypedMessage, base::OptionalMessage)
• containers: Collection types (containers::RepeatedField, containers::String, containers::Bytes)
• reflection: Runtime message inspection and dynamic decoding

Feature Flags

• std (default): Enables std::io integration, Vec-based encoding
• serde_support (default): Enables serde serialization via reflection
• nightly: Use nightly Rust features for slightly better codegen (branch hints)

For no_std environments, disable default features:

[dependencies]
protocrap = { version = "0.1", default-features = false }

Restrictions

Protocrap is designed for “sane” schemas:

• Up to 256 optional fields per message
• Struct sizes up to 64KB
• Field numbers 1-2047 (1 or 2 byte wire tags)
• Field numbers should be mostly consecutive

The following are intentionally unsupported:

• Unknown fields: Discarded during decoding (no round-trip preservation)
• Extensions: Proto2 extensions are silently dropped
• Maps: Decoded as repeated key-value pairs
• Proto3 zero-value omission: All set fields are serialized

Modules

arena

Arena allocator for protobuf message data.

base

Core message wrapper types.

containers

Collection types for protobuf messages.

descriptor_pool

google

proto_json

Proto JSON serialization support.

reflection

serde

tests

Enums

Error

Traits

Allocator

An implementation of Allocator can allocate, grow, shrink, and deallocate arbitrary blocks of data described via Layout.

Protobuf

ProtobufMut

Mutable protobuf operations (decode, deserialize). Extends ProtobufRef with mutation capabilities.

ProtobufRef

Read-only protobuf operations (encode, serialize, inspect). The lifetime parameter 'pool refers to the descriptor/table pool lifetime.

Functions

as_object

as_object_mut