http-pack

Binary framing for HTTP requests/responses so they can be relayed as a stream-decodable payload.

The format is designed to carry HTTP/1.1, H2, or H3 messages after TLS termination, then rebuild HTTP/1.1 requests or responses on the receiving side.

Format (v1)

• Magic: HPK1
• Format version: u8
• Kind: u8 (1 = request, 2 = response)
• HTTP version: u8 (1 = HTTP/1.1, 2 = H2, 3 = H3)
• Request fields (length-delimited with varint):
  • method
  • scheme (empty = none)
  • authority (empty = none)
  • path
• Response fields:
  • status (u16, network order)
• Headers:
  • header count (varint)
  • name length + bytes
  • value length + bytes
• Body:
  • body length (varint)
  • body bytes

Lengths are unsigned LEB128 varints so the payload can be decoded from a stream.

Usage

use bytes::Bytes;
use http::Request;
use http_pack::{decode, encode_request, Decoder, PackedMessage};

let req = Request::builder()
    .method("POST")
    .uri("https://example.com/ingest")
    .body(Bytes::from_static(b"hello"))
    .unwrap();

let payload = encode_request(&req).unwrap();
let decoded = decode(&payload).unwrap();

if let PackedMessage::Request(packed) = decoded {
    let http1 = packed.to_http1_bytes().unwrap();
    // send http1 bytes to an HTTP/1.1 backend
}

The HTTP/1.1 reconstruction helpers drop transfer-encoding and add a content-length when missing so the resulting request/response is valid in HTTP/1.1 form.

Streaming frames

For streaming relays, use the stream frame format (HPKS). Each frame is a small binary payload that can be signed and packetized independently:

• Headers frame: carries the request/response line + headers.
• Body frame: carries a chunk of body bytes.
• End frame: marks the end of the body.

On the receiving side, Http1StreamRebuilder converts these frames into HTTP/1.1 bytes using transfer-encoding: chunked so the body can be forwarded without buffering.

use http_pack::stream::{StreamFrame, StreamHeaders, StreamRequestHeaders, StreamBody, StreamEnd, Http1StreamRebuilder};

let headers = StreamHeaders::Request(StreamRequestHeaders {
    stream_id: 1,
    version: http_pack::HttpVersion::Http11,
    method: b"POST".to_vec(),
    scheme: None,
    authority: Some(b"example.com".to_vec()),
    path: b"/upload".to_vec(),
    headers: vec![],
});

let mut rebuilder = Http1StreamRebuilder::new();
let header_bytes = rebuilder.push_frame(StreamFrame::Headers(headers))?;
let body_bytes = rebuilder.push_frame(StreamFrame::Body(StreamBody { stream_id: 1, data: Bytes::from_static(b"hi") }))?;
let end_bytes = rebuilder.push_frame(StreamFrame::End(StreamEnd { stream_id: 1 }))?;

Streaming encode helpers

• HTTP/1.1 raw: h1::H1StreamDecoder emits StreamFrame values as bytes arrive.
• HTTP/2 (or any http_body::Body): stream::body::encode_request/encode_response emits frames via a callback.
• HTTP/3: stream::h3::encode_server_request/encode_client_response emits frames from h3::RequestStream.

These helpers let you stream the body without buffering it in memory.

HTTP/1.1 raw decoder

Enable h1 to parse raw HTTP/1.1 bytes into PackedRequest/PackedResponse values. This parser expects either content-length or transfer-encoding: chunked when a body is present.

http-pack = { path = "../http-pack", features = ["h1"] }

use http_pack::h1::{decode_request, H1MessageKind, H1Decoder};

let bytes = b"GET /hello HTTP/1.1\r\nHost: example.com\r\n\r\n";
let (req, _consumed) = decode_request(bytes).unwrap().unwrap();

let mut decoder = H1Decoder::new(H1MessageKind::Request);
decoder.push(bytes);
let msg = decoder.try_decode().unwrap();

Body collection for HTTP/1.1 and HTTP/2

Enable body to collect http_body::Body payloads (hyper requests/responses, h2 responses, etc.) into PackedRequest/PackedResponse values.

http-pack = { path = "../http-pack", features = ["body"] }

use http_pack::body::pack_request;

let packed = pack_request(req).await?;

HTTP/3 stream adapter

Enable h3 to collect data frames from h3::server::RequestStream or h3::client::RequestStream and build packed messages.

http-pack = { path = "../http-pack", features = ["h3"] }

use http_pack::h3::{pack_server_request, pack_client_response};

let packed_req = pack_server_request(req, &mut stream).await?;
let packed_resp = pack_client_response(resp, &mut stream).await?;

message-packetizer integration

http-pack always ships a HttpPackMessage wrapper that implements message_packetizer::SignableMessage. You encode an HTTP request/response into a packed payload, wrap it in HttpPackMessage, then sign and stream packets using message-packetizer.

use http_pack::packetizer::HttpPackMessage;
use message_packetizer::MessageSigner;

let msg = HttpPackMessage::from_request(&req).unwrap();
let mut signer = MessageSigner::new(&signing_key)?;
let signed = signer.sign(&msg)?;

for packet in signed.to_packets() {
    // stream packet bytes over SRT/UDP/etc
}

For streaming relays, use HttpPackStreamMessage and send each frame independently:

use http_pack::packetizer::HttpPackStreamMessage;
use http_pack::stream::StreamFrame;
use message_packetizer::MessageSigner;

let msg = HttpPackStreamMessage::from_frame(&frame);
let mut signer = MessageSigner::new(&signing_key)?;
let signed = signer.sign(&msg)?;
for packet in signed.to_packets() {
    // send packet bytes
}

Or use the convenience adapters to emit HttpPackStreamMessage directly (requires body feature):

use http_pack::packetizer::stream;

stream::encode_request(req, stream_id, |msg| {
    let signed = signer.sign(&msg)?;
    for packet in signed.to_packets() {
        // send packet bytes
    }
    Ok(())
}).await?;

Testing

cargo test

The test suite includes:

• Core HPK1 encode/decode roundtrips
• Real HTTP/2 connections via h2
• Real HTTP/3 connections via quinn + h3
• Byte-for-byte body fidelity across all protocols (all 256 byte values, null bytes, CRLF sequences, 1MB bodies)

License

MIT