memlink-protocol 0.1.0

Binary protocol definitions for MemLink IPC - fixed headers, magic constants, and error types
Documentation

memlink-protocol

Binary protocol definitions for MemLink IPC

Overview

memlink-protocol provides the core protocol definitions for MemLink IPC, including fixed 32-byte message headers, version negotiation, feature flags, and MessagePack serialization.

Key Features:

  • 📦 Fixed 32-byte message headers with packed representation
  • 🔢 Magic numbers and version constants for protocol identification
  • 🏷️ Type aliases for consistent sizing across platforms
  • ❌ Comprehensive error types for all failure modes
  • 🔄 Version negotiation with feature flag support
  • ⚡ Zero-copy message parsing from shared memory
  • 🎯 Arena-backed slices for efficient memory management
  • 🔐 Stream handles for large payload transfers

Features

Feature Description
std Enable std library support (for threading tests)
shm Enable integration with memlink-shm crate

Quick Start

use memlink_protocol::{MessageHeader, MessageType, Request, Response};
use memlink_protocol::msgpack::MessagePackSerializer;
use memlink_protocol::serializer::Serializer;

// Create a request header
let header = MessageHeader::new(
    MessageType::Request,
    1,           // request_id
    42,          // module_id
    0x12345678,  // method_hash
    256,         // payload_len
);

// Validate the header
assert!(header.validate().is_ok());

// Convert to bytes for transmission
let bytes = header.as_bytes();

// Parse from bytes
let parsed = MessageHeader::from_bytes(&bytes)?;

// Create and serialize a request
let request = Request::new(
    1,
    memlink_protocol::Priority::Normal,
    "calculator",
    "add",
    vec![1, 2, 3, 4],
);

let bytes = MessagePackSerializer.serialize_request(&request)?;
let parsed = MessagePackSerializer.deserialize_request(&bytes)?;

📚 More examples

Wire Format

All multi-byte fields use big-endian byte order for network transmission:

Offset Size Field Type Description
0 4 magic u32 Magic number (0x4D4C4E4B)
4 1 version u8 Protocol version
5 1 msg_type u8 Message type
6 2 features u16 Feature flags (big-endian)
8 8 request_id u64 Request identifier
16 8 module_id u64 Module identifier
24 4 method_hash u32 Method name hash
28 4 payload_len u32 Payload size in bytes

Total: 32 bytes (exactly)

Protocol Constants

Constant Value Description
MEMLINK_MAGIC 0x4D4C4E4B "MLNK" in ASCII, big-endian
PROTOCOL_VERSION 1 Current protocol version
HEADER_SIZE 32 Header size in bytes
MAX_PAYLOAD_SIZE 64 MiB Maximum payload (67,108,864 bytes)
CONTROL_REGION_SIZE 4 KiB SHM control region (4,096 bytes)

Type Aliases

Type Alias Description
RequestId u64 Unique identifier for request/response correlation
ModuleId u64 Identifier for modules or services
MethodHash u32 FNV-1a hash of method name for dispatch
TraceId u128 Distributed tracing identifier
SpanId u64 Span identifier within a trace

Message Types

Type Value Description
Request 0 Initiates an operation (expects response)
Response 1 Completes an operation
Error 2 Indicates failure
StreamHandle 3 Stream handle for streaming operations
HealthCheck 16 Health check request/response
LoadModule 32 Load a new module
UnloadModule 33 Unload a module
Stats 48 Statistics request/response
Event 96 Event notification

Status Codes

Status Value Description
Success 0 Operation completed successfully
ModuleNotFound 1 The specified module was not found
MethodNotFound 2 The specified method was not found
ExecutionError 3 Execution failed with an error
Timeout 4 Operation timed out
QuotaExceeded 5 Resource quota exceeded
BackpressureRejection 6 Request rejected due to backpressure

Feature Flags

Feature Bit Description
STREAMING 0 Chunked transfer support
BATCHING 1 Batch message grouping
PRIORITY_DEGRADATION 2 Priority-based fallback

Version Negotiation

use memlink_protocol::{negotiate_version, validate_version, V1_0, V1_1, V1_2};

// Negotiate between client and server versions
let result = negotiate_version(&V1_2, &V1_0);
assert!(result.is_ok());
assert_eq!(result.unwrap().minor, 0); // Negotiates to lowest common

// Validate version support
assert!(validate_version(&V1_0).is_ok());
assert!(validate_version(&V1_2).is_ok());

Performance

Benchmark results from cargo bench -p memlink-protocol:

Serialization Performance

Operation Time Throughput
Request serialize 1.07 µs 4.59 MiB/s
Request deserialize 1.11 µs 4.44 MiB/s
Response serialize 1.17 µs 4.16 MiB/s
Response deserialize 491 ns 10.2 MiB/s
Error serialize 1.00 µs 20.8 MiB/s
Error deserialize 477 ns 43.7 MiB/s

Large Payload Performance

Payload Serialize Deserialize
1 KB 18.2 µs (55.5 MiB/s) 12.0 µs (83.3 MiB/s)
10 KB 109 µs (89.4 MiB/s) 121 µs (81.0 MiB/s)
100 KB 1.30 ms (75.4 MiB/s) 1.55 ms (62.9 MiB/s)

Header Operations

Operation Time
as_bytes() 46 ns
from_bytes() 37 ns
validate() <1 ns

📊 Run benchmarks

Error Handling

use memlink_protocol::ProtocolError;

match header.validate() {
    Ok(()) => { /* valid header */ }
    Err(ProtocolError::InvalidMagic(magic)) => { /* wrong magic */ }
    Err(ProtocolError::UnsupportedVersion(ver)) => { /* unsupported version */ }
    Err(ProtocolError::PayloadTooLarge(size, max)) => { /* payload too big */ }
    Err(ProtocolError::SerializationFailed(msg)) => { /* serialization error */ }
    Err(ProtocolError::BufferOverflow { required, available }) => { /* buffer full */ }
}

Error Recovery

Error Type Recoverable Description
Timeout ✅ Yes Retry with backoff
BufferOverflow ✅ Yes Apply backpressure
QuotaExceeded ✅ Yes Wait for resources
InvalidMagic ❌ No Data corruption
UnsupportedVersion ❌ No Protocol mismatch

Design Principles

  • Binary-only: No JSON or text-based protocols for maximum performance
  • Fixed header size: Exactly 32 bytes for predictable memory layout
  • Platform-independent: Fixed-width types (u32, u64) and big-endian wire format
  • Explicit packing: #[repr(C, packed)] to prevent padding
  • Safe conversion: No std::mem::transmute for byte conversion

Safety

This crate follows strict safety guidelines:

  • ✅ No unsafe code in core implementation
  • ✅ No transmute for byte conversion (uses explicit to_be_bytes)
  • ✅ Compile-time size assertions for header structure
  • ✅ Comprehensive validation before parsing
  • ✅ Bounds checking on all buffer operations

Development

Prerequisites

  • Rust 1.70 or later

Building

# Build the crate

cargo build -p memlink-protocol


# Build with SHM integration

cargo build -p memlink-protocol --features shm

Testing

# Run all tests

cargo test -p memlink-protocol --features std


# Run integration tests

cargo test -p memlink-protocol --features std --test integration


# Run doc tests

cargo test -p memlink-protocol --doc

Benchmarks

# Run all benchmarks

cargo bench -p memlink-protocol --features std


# Run specific benchmark

cargo bench -p memlink-protocol --features std -- serialization

Code Quality

# Format code

cargo fmt -p memlink-protocol


# Run clippy

cargo clippy -p memlink-protocol --features std -- -D warnings

Examples

# Run the basic example

cargo run -p memlink-protocol --example basic --features std

Related Crates

Crate Description
memlink-shm Shared memory IPC library
memlink-runtime Dynamic module loading
memlink-msdk Module SDK
memlink-msdk-macros Proc macros

License

Apache-2.0 license.

See LICENSE-APACHE for the full license text.

Contributing

Contributions are welcome! Please follow the project guidelines:

  1. Add tests for new functionality
  2. Ensure cargo clippy passes with no warnings
  3. Update documentation for API changes
  4. Follow the existing code style

memlink-protocol - Fast, reliable binary protocol for Rust IPC