bytes-handoff

bytes-handoff is a small Rust crate for moving owned byte buffers across async I/O boundaries.

It does not replace AsyncRead or AsyncWrite. It layers on top of them so protocol code can:

• read bytes as soon as they arrive
• keep nonblocking reads in safe, owned mutable buffer state
• peek at incomplete input without committing
• preserve unconsumed tails across parser or mode boundaries
• split complete prefixes into Bytes for cheap cross-task handoff
• submit owned Bytes writes to an async writer without borrowing memory until the socket finishes
• bound queued writes by item count and byte count

The zero-copy claim here is intentionally scoped: BytesMut/Bytes avoid extra application-level copies after socket ingress. They do not make TCP kernel-to-userspace reads zero-copy.

The main use case is content-routed streaming I/O: many independent connections, each with bytes arriving in arbitrary fragments, where routing decisions depend on the stream content. In that setting the read buffer is protocol state, not temporary scratch memory.

Use plain read(&mut [u8]) when bytes are consumed immediately in the same function and then discarded. Use bytes-handoff when the read buffer itself is part of the protocol state: partial frames must survive, complete prefixes need owned handoff, or a parser may switch into a raw tunnel without losing already read bytes.

Examples

The repository includes small runnable examples:

• line_protocol: incremental line parsing with a partial tail.
• length_prefix: length-prefixed frames where the header arrives before the full payload.
• content_routing: inspect buffered bytes, route complete safe prefixes, then switch to a raw tunnel while preserving already-read tail bytes.
• write_handoff: submit owned bytes to an async writer and await completion.

Run one with:

cargo run --example content_routing

Read Handoff

use bytes::Bytes;
use bytes_handoff::HandoffBuffer;

/// Reads available bytes, splits one complete line, and hands it off.
async fn read_one_line<R>(reader: &mut R) -> Result<(), Box<dyn std::error::Error>>
where
    R: tokio::io::AsyncRead + Unpin,
{
    let mut buffer = HandoffBuffer::new(64 * 1024);

    buffer.read_available(reader).await?;

    if let Some(newline) = buffer.peek().iter().position(|b| *b == b'\n') {
        let line = buffer.split_prefix(newline + 1)?;
        send_to_worker(line);
    }

    Ok(())
}

/// Sends an owned byte slice to sync or async protocol work.
fn send_to_worker(_: Bytes) {}

Write Handoff

use bytes::Bytes;
use bytes_handoff::{WriteHandoff, WriteHandoffConfig};

/// Submits owned bytes to an async writer without blocking the producer.
fn submit_owned_write<W>(writer: W) -> Result<(), Box<dyn std::error::Error>>
where
    W: tokio::io::AsyncWrite + Unpin + Send + 'static,
{
    let handoff = WriteHandoff::spawn(writer, WriteHandoffConfig::new(1024, 8 * 1024 * 1024));

    handoff.try_write_fire_and_forget(Bytes::from_static(b"owned bytes"))?;

    // Use `try_write` or `write` instead when the producer needs a completion
    // ticket for this chunk.
    Ok(())
}

Benchmarks

The repository includes Criterion benchmarks for the main operations this crate adds around async I/O:

• incremental reads into persistent state
• complete-frame splitting into owned Bytes
• preserving an already-read tail when parser mode changes
• bounded owned write submission and completion tracking
• byte-budget backpressure

Run them with:

cargo bench

For a defensible local baseline:

./bench/run-criterion-baseline.sh

The baseline script uses --sample-size 100 --measurement-time 5 by default. On Linux, set TASKSET_CORES=2-5 to pin the run with taskset and PERF_STAT=1 to collect cycles, instructions, cache misses, context switches, and CPU migrations alongside Criterion. For release-grade numbers, run on the same kernel and CPU family as deployment, with the CPU governor and turbo policy fixed outside the script.

For an end-to-end stream harness rather than a Criterion microbenchmark:

./bench/run-stream-harness.sh --scenario fragmented --runs 5
./bench/run-stream-harness.sh --scenario coalesced --runs 5
./bench/run-stream-harness.sh --transport tcp --scenario fragmented --runs 5
./bench/run-stream-matrix.sh
./bench/run-stream-harness.sh --scenario fragmented --completion fire_and_forget --runs 5

The harness drives complete client/proxy/sink streams through fragmented input, content-routed prefixes, tunnel handoff, and WriteHandoff output. It writes machine-readable run artifacts under bench/results/, including throughput, latency percentiles, context switches, CPU cost, and peak RSS. Use --transport tcp for a localhost TCP service/sink harness; see bench/README.md.

Latest TCP Harness Results

These are end-to-end localhost TCP harness results from an Ubuntu 24.04 server (adam), not Criterion microbenchmarks. The driver/client and sink processes were pinned to a different CPU set than the proxy service so the service-side CPU cost can be read separately from load generation.

Run shape:

• transport: localhost TCP
• worker threads: 16
• concurrent connections: 128
• route frames per connection: 64
• route frame payload: 63 bytes
• tunnel payload per connection: 1 MiB
• read reserve: 16 KiB
• handoff write pending budget: 32 KiB default (2 * read_reserve)
• completion mode: fire-and-forget
• runs: 2 per point, 10 second target duration

The implementations are:

• handoff: HandoffBuffer plus WriteHandoff; this is the crate path.
• manual_vec: a hand-written persistent Vec<u8> parser with direct writes; this is the practical read(&mut [u8]) comparison.
• raw_copy: unparsed async copy; this is a lower bound and does less protocol work than handoff.

Coalesced input, where client writes arrive in 16 KiB chunks:

Fragmented input, where client writes arrive in 64 byte chunks:

Interpretation:

• In the coalesced TCP workload, handoff is about 8% behind the practical manual Vec<u8> baseline on throughput and uses about 23% more proxy-service CPU. That is the current cost of the safer buffer lifecycle, owned prefix handoff, bounded write queue, and mode-switch tail preservation.
• In the fragmented 64 byte workload, throughput is dominated by tiny socket operations. All implementations cluster around 104 MiB/s; the difference is mostly service CPU cost.
• raw_copy is useful as a lower bound, but it is not a semantic replacement: it does not parse route frames, preserve parser state, or hand off owned prefixes.

What The Read Benchmarks Measure

The read benchmarks are split by workload so the output does not imply one single headline throughput number.

The raw discard lower bound should be much faster than the framed workloads. That does not mean HandoffBuffer is slow at the job it is meant to do; it means the raw benchmark does less work. For wrapper overhead, compare read_owned_lines/bytesmut_split with read_owned_lines/handoff_buffer.

What The Write Benchmarks Measure

The write benchmarks measure owned Bytes submission into one async writer, batched drain behavior, optional completion notification, and backpressure. They are not raw socket throughput benchmarks.

Treat all benchmark numbers as directional, not universal. They use in-memory readers/writers; real sockets add scheduler, kernel, and network effects. The benchmark exists to make the tradeoff explicit: raw slice reads are best for immediate consumption, while bytes-handoff targets safe mutable buffering, prefix ownership, tail preservation, and bounded async write handoff.