# Witnet Wire Protocol
[](https://crates.io/crates/witnet)
**Witnet** is a lightweight, high-performance HTTP-like TCP server framework built explicitly for ultra-fast transfers of "Execution Witnesses" between Execution Clients and Provers.
It strips away all unnecessary application-layer overhead (like HTTP/2, gRPC, or QUIC framing) in favor of a zero-buffering, strictly 9-byte framed socket connection. It achieves **blazing fast throughput (>13.4 GB/s)** on local loopback, crushing standard Unix Domain Socket (IPC) latency limitations.
Designed as the transport protocol for the Ethereum Engine API, Witnet implements connection-level JWT authentication following the same conventions used by Geth and other execution clients.
## Core Features
1. **HTTP-Like Developer Experience:** You structure your applications using `Router`, `Request`, and `Response` paradigms common in frameworks like `Actix` or `Axum`.
2. **JWT Authentication:** Connection-level authentication using HS256 JWTs, matching the [Ethereum EngineAPI auth spec](https://github.com/ethereum/execution-apis/blob/main/src/engine/authentication.md). Supports hex-encoded 256-bit shared secrets, `iat` ±60s validation, and secret loading from file.
3. **Zero-Buffering Pipelines:** The `Request::into_body_stream()` returns a `Body` struct mapped directly to the active `tokio::net::TcpStream`. No `BytesMut` string-buffering takes place behind the scenes.
4. **Protection Limits:** Hard-coded `MAX_PAYLOAD_SIZE` (5GB) and `MAX_AUTH_PAYLOAD_SIZE` (8KB) drop connections immediately if payload allocations exceed bounds, avoiding DoS vulnerabilities.
5. **Massive Throughput Enhancements:** Implements `TCP_NODELAY` and `tokio_util::io::ReaderStream` 64KB capacities under the hood to completely eliminate kernel allocation thrashing, beating manual loop-scripts.
## Installation
Add `witnet` to your `Cargo.toml`:
```toml
[dependencies]
witnet = "0.1.2"
```
## The Protocol Specification
A single message frame consists of:
* **1-byte Message Type identifier**
* **8-byte Length Prefix** (Big-Endian `u64`)
* **Raw Payload bytes**
As soon as the 9-byte header is parsed, the socket halts, yielding control to your `Router` handler logic exactly before payload ingress.
## Standard Usage
### Server Configuration
```rust
use witnet::{Server, Router, Request, Response, Body, JwtSecret};
use futures::StreamExt;
use std::time::Duration;
const WITNESS_BY_NUMBER: u8 = 0x01;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// Load JWT secret from file (or generate one for dev)
let jwt_secret = JwtSecret::from_file("jwt.hex")
.unwrap_or_else(|_| JwtSecret::random());
let app = Router::new()
.route(WITNESS_BY_NUMBER, handle_by_number);
Server::bind("127.0.0.1:8080")
.with_jwt_secret(jwt_secret) // Enable JWT authentication
.with_handshake_timeout(Duration::from_secs(5)) // Optional: defaults to 10s
.with_tcp_nodelay(true) // Optional: defaults to false
.serve(app)
.await?;
Ok(())
}
async fn handle_by_number(req: Request) -> Response {
println!("Incoming Request with payload length: {}", req.len());
let body_stream = req.into_body_stream();
// Stream the payload dynamically directly out of the socket half
if let Body::Stream(_, mut stream) = body_stream {
while let Some(Ok(chunk)) = stream.next().await {
// Process the payload without loading gigabytes into RAM
}
}
// Always respond
Response::new(WITNESS_BY_NUMBER, "Processed Successfully")
}
```
### State Extraction (Axum-style)
Witnet provides a `State` extractor wrapper heavily inspired by Axum, allowing you to pass `Arc`/shared state objects directly into your handler functions!
```rust
use witnet::{Server, Router, Request, Response, State};
use std::sync::Arc;
#[derive(Clone)]
struct AppState {
db_pool: Arc<String>,
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let state = AppState {
db_pool: Arc::new("postgres://user:pass@localhost".to_string()),
};
let app = Router::new()
.route(0x01, handle_with_state)
.with_state(state); // Bakes the state into the handlers
Server::bind("127.0.0.1:8081").serve(app).await?;
Ok(())
}
// Extract state gracefully using `State(state)`
async fn handle_with_state(req: Request, State(state): State<AppState>) -> Response {
println!("Type: {} | Config: {}", req.msg_type, state.db_pool);
Response::new(0x01, "Success")
}
```
### Client Configuration
```rust
use witnet::{Client, Request, Body, JwtSecret};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// Use the same shared secret as the server
let jwt_secret = JwtSecret::from_file("jwt.hex")?;
// Connect and authenticate in one step
let client = Client::connect_with_auth("127.0.0.1:8080", &jwt_secret).await?;
let massive_mock_data = vec![0x11; 10 * 1024 * 1024];
let req = Request::builder(0x01, massive_mock_data);
let resp = client.send(req).await?;
println!("Response Type: {}", resp.msg_type);
Ok(())
}
```
## JWT Authentication
Witnet implements connection-level JWT authentication following the [Ethereum EngineAPI authentication spec](https://github.com/ethereum/execution-apis/blob/main/src/engine/authentication.md), matching the behaviour of Geth's `--authrpc.jwtsecret`.
### How It Works
| Algorithm | **HS256** (HMAC + SHA-256) |
| Secret | 256-bit hex-encoded key (same as Geth's `jwt.hex`) |
| Required claim | `iat` — must be within **±60 seconds** of server time |
| Auth flow | First message on each TCP connection must be `0x00` (auth) |
### Generating a Secret
```bash
# Generate a jwt.hex file (same method used for Geth/Ethereum clients)
### Loading Secrets
```rust
use witnet::JwtSecret;
// From a hex file (production — same as Geth's --authrpc.jwtsecret)
let secret = JwtSecret::from_file("jwt.hex")?;
// From a hex string
let secret = JwtSecret::from_hex("0xabcdef...")?;
// Random ephemeral secret (dev/testing)
let secret = JwtSecret::random();
```
### Manual Authentication
If you need more control over the auth handshake:
```rust
use witnet::{Client, JwtSecret};
let jwt_secret = JwtSecret::from_file("jwt.hex")?;
// Connect without auto-auth
let mut client = Client::connect("127.0.0.1:8080").await?;
// Generate and send the JWT token manually
let token = jwt_secret.generate_token()?;
client.authenticate(&token).await?;
// Now send data messages...
```
## Benchmarks
Benchmarked against an Apple Silicon M3 Max running on the `127.0.0.1` macOS loopback layer. Because of the `ReaderStream` 64KB scaling, `witnet` averages speeds drastically higher than manual socket polling scripts.
| **8 MB** | **2538 MB/s** |
| **20 MB** | **3481 MB/s** |
| **100 MB** | **4195 MB/s** |
| **300 MB** | **8774 MB/s** |
| **500 MB** | **13.4 GB/s** |
*Note: You can replicate the testing parameters yourself by running `cargo run --release --example dummy_witness`.*