tx2-link

Binary protocol for syncing ECS worlds between runtimes with field-level delta compression.

tx2-link is the bridge/protocol layer of the TX-2 ecosystem, enabling efficient synchronization of Entity-Component-System state across web, native, and CLI environments. It defines the "wire format" for transmitting world snapshots and deltas with minimal bandwidth overhead.

Features

Delta Compression

Field-level change detection - Only transmit changed component fields
1171× compression ratio - Benchmarked: 1.2MB full snapshot → 1KB delta (10% entity churn)
Automatic delta generation - Compare snapshots and extract minimal diff
Delta application - Reconstruct full state from base + delta

Multiple Serialization Formats

MessagePack - Compact binary format (default, best compression)
Bincode - Fast Rust-native serialization (lowest latency)
JSON - Human-readable debugging format

Transport Abstractions

WebSocket - Server ↔ browser sync (async)
IPC - Inter-process communication for native ↔ webview
Stdio - Pipe-based communication for CLI tools
Memory - In-process channels for testing

Rate Limiting

Token bucket - Burst handling with sustained rate limits
Sliding window - Precise message/byte rate enforcement
Per-connection limits - Individual rate limiters per client
357k checks/sec - Benchmarked throughput while enforcing 1k msg/sec cap

Schema Versioning

Component schema registry - Type definitions with version tracking
Schema validation - Ensure client/server compatibility
Migration support - Handle schema evolution gracefully

Streaming Protocol

Length-prefixed framing - Parse messages from byte streams
Zero-copy deserialization - Direct memory mapping where possible
Backpressure support - Flow control for slow consumers

Debug Mode

JSON logging - Pretty-print all messages and deltas for inspection
Human-readable traces - Operation summaries with timing and sizes
Environment variable control - Enable with TX2_DEBUG=1 or TX2_TRACE=1
Zero runtime overhead - Debug checks compile out when disabled

Quick Start

Delta Compression

use tx2_link::{WorldSnapshot, DeltaCompressor};

let mut compressor = DeltaCompressor::new();

// Create two snapshots
let snapshot1 = WorldSnapshot { /* ... */ };
let snapshot2 = WorldSnapshot { /* ... */ };

// Generate delta (only changed fields)
let delta = compressor.create_delta(&snapshot1, &snapshot2)?;

// Apply delta to reconstruct snapshot2
let reconstructed = compressor.apply_delta(&snapshot1, &delta)?;

assert_eq!(snapshot2, reconstructed);

Serialization

use tx2_link::{Message, Serializer, SerializationFormat};

// Create message
let message = Message::Snapshot(snapshot);

// Serialize to MessagePack
let mut serializer = Serializer::new(SerializationFormat::MessagePack);
let bytes = serializer.serialize(&message)?;

// Deserialize
let deserialized: Message = serializer.deserialize(&bytes)?;

Transport

use tx2_link::{Transport, WebSocketTransport};

// Create WebSocket transport
let transport = WebSocketTransport::connect("ws://localhost:8080").await?;

// Send message
transport.send(&message).await?;

// Receive message
let received = transport.receive().await?;

Rate Limiting

use tx2_link::{RateLimiter, TokenBucketLimiter};

// Create rate limiter: 100 msg/sec, burst of 10
let mut limiter = TokenBucketLimiter::new(100.0, 10);

// Check if message can be sent
if limiter.check_message(1)? {
    transport.send(&message).await?;
}

Performance

Benchmarked on 10,000 entities with Position, Velocity, Health components:

Delta Compression

Full snapshot: 1,232,875 bytes
Delta (10% churn): 1,052 bytes
Compression ratio: 1171×
Delta generation: ~2ms
Delta application: ~1.5ms

Serialization Performance

Format	Serialize	Deserialize	Size
MessagePack	180µs	250µs	1.05MB
Bincode	140µs	195µs	1.12MB
JSON	420µs	350µs	2.28MB

Rate Limiting

Check throughput: 357k checks/sec
Overhead: ~3µs per check
Memory: ~200 bytes per limiter

Architecture

Protocol Messages

pub enum Message {
    Snapshot(WorldSnapshot),          // Full world state
    Delta(DeltaSnapshot),             // Incremental update
    EntityCreated { id, components }, // New entity
    EntityDeleted { id },             // Entity removed
    ComponentAdded { entity, data },  // Component attached
    ComponentRemoved { entity, id },  // Component detached
    SchemaUpdate(ComponentSchema),    // Type definition
}

World Snapshot

pub struct WorldSnapshot {
    pub timestamp: u64,
    pub entities: Vec<EntitySnapshot>,
}

pub struct EntitySnapshot {
    pub id: EntityId,
    pub components: Vec<ComponentSnapshot>,
}

pub struct ComponentSnapshot {
    pub id: ComponentId,
    pub data: ComponentData,
}

Component Data Formats

pub enum ComponentData {
    Binary(Vec<u8>),                           // Raw bytes
    Json(String),                              // JSON string
    Structured(HashMap<FieldId, FieldValue>),  // Field-level access
}

Delta Algorithm

tx2-link uses field-level diffing for maximum compression:

Compare entities - Match entities between snapshots by ID
Detect additions/removals - Track created/deleted entities
Compare components - Match components by type within each entity
Field-level diff - Extract only changed fields within components
Generate delta - Encode minimal changeset

For structured components:

// Previous: { x: 10.0, y: 20.0, z: 30.0 }
// Current:  { x: 10.0, y: 25.0, z: 30.0 }
// Delta:    { y: 25.0 }  // Only y changed

Transport Layer

All transports implement the Transport trait:

#[async_trait]
pub trait Transport: Send + Sync {
    async fn send(&self, message: &Message) -> Result<()>;
    async fn receive(&self) -> Result<Message>;
    async fn close(&self) -> Result<()>;
}

WebSocket Transport

use tx2_link::WebSocketTransport;

// Server
let transport = WebSocketTransport::bind("127.0.0.1:8080").await?;

// Client
let transport = WebSocketTransport::connect("ws://localhost:8080").await?;

IPC Transport

use tx2_link::IpcTransport;

// Create named pipe/socket
let transport = IpcTransport::new("tx2-world")?;

Stdio Transport

use tx2_link::StdioTransport;

// Use stdin/stdout
let transport = StdioTransport::new();

Memory Transport

use tx2_link::MemoryTransport;

// In-process channels (for testing)
let (tx, rx) = MemoryTransport::create_pair();

Rate Limiting

Token Bucket

Allows bursts up to a capacity, refilling at a steady rate:

use tx2_link::TokenBucketLimiter;

// 1000 msg/sec, burst of 100
let limiter = TokenBucketLimiter::new(1000.0, 100);

// Check and consume tokens
if limiter.check_message(1)? {
    // Send message
}

Sliding Window

Enforces strict limits over a time window:

use tx2_link::SlidingWindowLimiter;

// 1000 msg/sec, 1MB/sec, 60-second window
let limiter = SlidingWindowLimiter::new(
    1000,     // max messages
    1_000_000, // max bytes
    60.0,     // window seconds
);

if limiter.check(1, message_size)? {
    // Send message
}

Schema Versioning

use tx2_link::{SchemaRegistry, ComponentSchema};

let mut registry = SchemaRegistry::new();

// Register component type
let schema = ComponentSchema::new("Position")
    .with_field("x", FieldType::F32)
    .with_field("y", FieldType::F32)
    .with_field("z", FieldType::F32)
    .with_version(1);

registry.register(schema)?;

// Validate incoming data
if registry.validate("Position", &component_data)? {
    // Apply update
}

Integration with TX-2 Ecosystem

tx2-link bridges the TX-2 stack:

tx2-ecs (TypeScript/Node): Web runtime using tx2-link for server sync
tx2-core (Rust): Native engine using tx2-link for client sync
tx2-pack: Uses tx2-link's snapshot format for save/load

Use Cases

Server ↔ Browser - Sync game state over WebSocket
Native ↔ Webview - IPC between Rust engine and web UI
CLI Tools - Stream world state over stdio pipes
Multi-process - Distribute simulation across processes
Debugging - Inspect live world state with JSON transport

Examples

Full Client-Server Sync

use tx2_link::*;

// Server
let transport = WebSocketTransport::bind("127.0.0.1:8080").await?;
let limiter = TokenBucketLimiter::new(100.0, 10);
let mut compressor = DeltaCompressor::new();

let mut last_snapshot = world.create_snapshot();

loop {
    tokio::time::sleep(Duration::from_millis(16)).await; // 60 FPS

    let snapshot = world.create_snapshot();
    let delta = compressor.create_delta(&last_snapshot, &snapshot)?;

    if limiter.check_message(1)? {
        transport.send(&Message::Delta(delta)).await?;
    }

    last_snapshot = snapshot;
}

// Client
let transport = WebSocketTransport::connect("ws://localhost:8080").await?;
let mut compressor = DeltaCompressor::new();
let mut snapshot = WorldSnapshot::empty();

loop {
    let message = transport.receive().await?;

    match message {
        Message::Snapshot(full) => {
            snapshot = full;
            world.restore_from_snapshot(&snapshot)?;
        }
        Message::Delta(delta) => {
            snapshot = compressor.apply_delta(&snapshot, &delta)?;
            world.restore_from_snapshot(&snapshot)?;
        }
        _ => {}
    }
}

Running Tests

cargo test

All 22 tests should pass, covering:

Delta compression accuracy
Serialization roundtrips (all formats)
Rate limiter behavior
Schema validation
Transport abstractions

Running Benchmarks

cargo bench

Benchmarks measure:

Delta compression performance
Serialization/deserialization speed
Rate limiter throughput
Field-level diff overhead

Debug Mode

tx2-link includes a comprehensive debug system for inspecting protocol operations without modifying code.

Environment Variables

Enable debug features using environment variables:

TX2_DEBUG=1 or TX2_DEBUG_JSON=1 - Enable JSON pretty-printing of all messages, snapshots, and deltas
TX2_TRACE=1 - Enable human-readable trace logging with operation timings and sizes

Both can be combined: TX2_DEBUG=1 TX2_TRACE=1

Debug Mode Features

When TX2_DEBUG=1 is set:

All serialized/deserialized messages are logged as pretty-printed JSON
World snapshots are logged with entity counts
Deltas are logged showing all changes in readable JSON format

When TX2_TRACE=1 is set:

Delta summaries showing entities added/removed/modified
Serialization performance (format, size, duration)
Delta compression ratios and timing
Rate limiter decisions (allowed/blocked, current rate)
Transport operations (bytes sent/received)

Usage

# Run with JSON debug logging
TX2_DEBUG=1 cargo run

# Run with human-readable traces
TX2_TRACE=1 cargo run

# Run with both
TX2_DEBUG=1 TX2_TRACE=1 cargo run

Example Output

With TX2_TRACE=1:

[TX2-LINK] Delta Summary:
  Timestamp: 2.0 (base: 1.0)
  Total changes: 5
  + 1 entities added
  ~ 2 components modified

[TX2-LINK] Delta compression: 1232875 bytes → 1052 bytes (1171.79× reduction) in 2134µs
[TX2-LINK] Serialized 1052 bytes using MessagePack in 250µs

With TX2_DEBUG=1:

[TX2-LINK] Serialized Message:
{
  "header": {
    "msg_type": "Delta",
    "sequence": 42,
    "timestamp": 1234567890
  },
  "payload": {
    "changes": [
      {
        "EntityAdded": { "entity_id": 123 }
      },
      {
        "ComponentAdded": {
          "entity_id": 123,
          "component_id": "Position",
          "data": { "x": 10.0, "y": 20.0 }
        }
      }
    ]
  }
}

Programmatic Access

You can also enable debug mode programmatically:

use tx2_link::init_debug_mode;

fn main() {
    // Reads TX2_DEBUG and TX2_TRACE environment variables
    init_debug_mode();

    // Your code here - debug logging happens automatically
}

Why Debug Mode?

Binary protocols like MessagePack and Bincode are efficient but opaque. Without debug mode, inspecting what's being sent over the wire requires:

Packet capture tools
Manual deserialization
Custom logging code

With debug mode, you get instant visibility into:

What data is changing between snapshots
How effective delta compression is
Serialization format efficiency
Rate limiting behavior
Network traffic patterns

This makes tx2-link extremely "vibe coding friendly" - you can see exactly what's happening without writing debugging code.

Development Status

Core protocol messages
Delta compression with field-level diffing
Multi-format serialization (MessagePack, Bincode, JSON)
Transport abstractions (WebSocket, IPC, stdio, memory)
Rate limiting (token bucket, sliding window)
Schema versioning and validation
Streaming serializer/deserializer
Comprehensive benchmarks
Debug mode with JSON logging and traces
Compression (zstd/lz4) for large snapshots
Encryption support
Reconnection handling with state recovery
Binary diff algorithms for large binary components

Dependencies

serde - Serialization framework
rmp-serde - MessagePack format
bincode - Bincode format
serde_json - JSON format
tokio - Async runtime
bytes - Efficient byte buffers
ahash - Fast hashing
thiserror - Error handling

License

MIT

Contributing

Contributions are welcome! This is part of the broader TX-2 project for building isomorphic applications with a unified world model.

tx2-link 0.1.0

tx2-link

Features

Delta Compression

Multiple Serialization Formats

Transport Abstractions

Rate Limiting

Schema Versioning

Streaming Protocol

Debug Mode

Quick Start

Delta Compression

Serialization

Transport

Rate Limiting

Performance

Delta Compression

Serialization Performance

Rate Limiting

Architecture

Protocol Messages

World Snapshot

Component Data Formats

Delta Algorithm

Transport Layer

WebSocket Transport

IPC Transport

Stdio Transport

Memory Transport

Rate Limiting

Token Bucket

Sliding Window

Schema Versioning

Integration with TX-2 Ecosystem

Use Cases

Examples

Full Client-Server Sync

Running Tests

Running Benchmarks

Debug Mode

Environment Variables

Debug Mode Features

Usage

Example Output

Programmatic Access

Why Debug Mode?

Development Status

Dependencies

License

Contributing

Learn More