The model
Five layers, each usable on its own:
- Framing.
Framingnames the wire format; aDecodercuts message bodies out of a byte stream (sans-IO: you read, it buffers at most one message, and gives memory back after a large one) andFraming::encodeframes them back. - Messages.
decodereads a body intoMessages (Request,Notification,Response), a batch, or anInvalidthat says how to answer it.paramsandresultstay borrowedRawslices of the body until a handler parses them. - Correlation.
Pendingmaps our outstanding request ids to waiters, with deadlines and a capacity; every operation isO(log n). - Endpoint. An
Endpointroutes the peer's calls to aRouter's handlers and settles answers to our own requests, sent through aPeerfrom any thread.serveis the one-call server;Serverserves a listener. - Output. A
Sinktakes each outgoing message.Queuedwrites from its own thread and anoutboxfrom an async task, both behind bounded limits, so a slow peer blocks no sender while the connection is open. Closing waits for the queue to drain only after releasing the endpoint's lock, and for at mostQueueLimits::close_timeout.
What it guarantees, and how each guarantee is checked:
| Guarantee | How it is held |
|---|---|
Every request gets exactly one response; no valid notification gets one (a malformed or oversized message is answered under null, as the specification requires). |
A property test sends random batches mixing requests (answered at once, later, by a panicking handler, for a missing method, or cancelled), notifications, malformed elements, and stray responses, and checks the reply holds each request id exactly once with the right code. |
Malformed messages get the specification's codes (-32700, -32600, -32601, -32602, -32603), under their own id when it can be read. |
Every example in section 7 of the JSON-RPC 2.0 specification is a test, verbatim; a property test checks envelope classification against a reference built on a full parse. |
| Every message shape survives encoding and decoding, through either framing. | Property tests over random requests, notifications, results, and errors (any id, any string, any JSON), fed through the decoder in random chunk sizes. |
| No input makes it panic, memory stays bounded, and the framing does not depend on how the input is split. | Fuzz targets for both framings, message and batch decoding, and a live endpoint (fuzz/), whose checks also run in cargo test on random, JSON-shaped, and mutated inputs; the decoder's buffer is checked against its documented bound after every step. |
| A peer that stops reading stalls no sender, and closing over it holds up no other thread. | Tests block the writer and check that every handle still sends, that the connection closes once the queue's byte, message, or no-progress limit is reached, that a slow but steady reader is not cut off, and that while a close waits on a stalled writer a Responder on another thread returns at once. |
| Deadlines fire on their own. | Tests let a request time out with no traffic and no waiter and check the cancel notification goes out. |
| A handler panic is contained. | Tests panic in request, deferred, and notification handlers, the interceptor, batches, the event hook, the sink, and the writer, and check the answers and that the session goes on (or, for the output, closes cleanly). |
| Cancellation races resolve to one outcome. | Tests race worker answers against peer cancellations, and our cancellations against the peer's answers, hundreds of times each. |
| No quadratic paths. | Tripwire tests: 4 MiB bodies and lines fed a byte at a time, 100,000 messages through one endpoint, a full batch of cancelled deferred requests, 50,000 forged answers while 50,000 requests are held. |
Installation
[]
= "0.3"
= "0.2" # to build and read JSON values
The framing, message, and correlation layers without the standard library:
[]
= { = "0.3", = false }
| Feature | Default | Adds |
|---|---|---|
std |
yes | The endpoint, its handles, serve, Queued, outbox, progress helpers. |
serde |
no | Typed params and results: Router::request_typed, Params::deserialize, Peer::request_typed, Call::wait_typed (with serde and serde_json). |
futures-io |
no | The async adapter: Endpoint::run_async, and Outbox as AsyncRead/AsyncBufRead with write_to. Runtime-agnostic. |
tracing |
no | Spans for every call and events for what the endpoint does on its own. Off, the instrumentation compiles to nothing. |
net |
no | Server: an accept loop over TCP and, on Unix, Unix domain sockets. |
Quick start
A server on stdin and stdout is one call. This one takes newline-delimited messages, as MCP's stdio transport does; LSP uses Framing::ContentLength:
use json;
use ;
serve writes through a Queued sink: a writer thread does the writes, so a client that stops reading does not block a handler's sends. The connection closes once the queue is full (64 MiB by default), or at the first send after the writer has written nothing for 60 seconds. When the input ends, serve waits for the queue to drain, for at most 10 seconds (QueueLimits::close_timeout).
State, and calls in the other direction
An Endpoint carries state for its handlers and sends requests of its own. Answers are matched to calls by id; a Call can be waited on from any thread but the endpoint's, polled, or .awaited:
use mpsc;
use json;
use ;
Long work, answered later and cancellable
A deferred handler gets a Responder it can keep or move to a worker. If the peer cancels ($/cancelRequest in LSP, notifications/cancelled in MCP), the responder's token is set and the request is answered as the protocol says; a late answer is refused:
use mpsc;
use ;
Protocol presets
Config::lsp() and Config::mcp() set the cancellation spelling, the initialize gate, and the limits each protocol wants (MCP turns batches off, as its 2025-06-18 revision removed them, and serves ping before initialize). Limits::network() and Limits::strict() cut every budget for peers across a network. Progress sends MCP's notifications/progress and WorkDone LSP's $/progress for the token a request carries:
use ;
Async, and the lower layers
Endpoint::next_read, commit, and finish are run taken apart: the endpoint says where to read, any reader (blocking, non-blocking, or .awaited on any runtime) fills it, and the endpoint handles what arrived. With the futures-io feature, Endpoint::run_async and Outbox::write_to do this over AsyncRead and AsyncWrite. Below the endpoint, the decoder and decode work on their own:
use ;
Typed params through serde, the async adapter, the TCP server, and the interceptor have examples in docs/API.md.
Examples
Runnable programs in examples/:
| Example | What it shows |
|---|---|
stdio_echo |
A server on stdin and stdout, newline framing (MCP) or --content-length (LSP): echo '{"jsonrpc":"2.0","id":1,"method":"ping"}' | cargo run --example stdio_echo |
pipe_pair |
A client and a server over an in-memory pipe, each on its own thread: concurrent requests correlated by id, a deferred answer with progress notifications, and a cancelled request. |
tcp_server |
A Server on a TCP port with per-connection state, network limits, an interceptor, and a clean stop (--features net). |
Performance
Decoding is one pass of jsonx-lib's pull reader over each body: the envelope members are read as raw slices, so params and result are validated once and never copied or decoded unless a handler asks. Method names and string ids borrow from the body when they have no escapes, and deferred requests are looked up without copying their ids. Framed frames messages into a reused buffer and writes once per turn, however many messages a handler sends; Queued writes each batch of queued messages with one write_all and one flush. The decoder reads straight into its own buffer, searches for line ends eight bytes at a time, resumes each search where the last one stopped, and grows geometrically while a large message arrives.
Measured with the benchmarks in benches/, x86_64 Windows, Rust 1.99, release profile, on LSP-shaped textDocument/hover requests (about 160 bytes each). The machine was under heavy background load from other work, and repeated runs there differed by up to 40%, so these are lower bounds to compare trends with, not the crate's speed on a quiet machine. Across back-to-back runs of 0.2.0 and 0.3.0 there, neither was consistently faster: the differences between the versions were smaller than those between repeated runs.
| Benchmark | What it measures | Windows |
|---|---|---|
framing/decode_100k_ContentLength |
Cut 100,000 Content-Length frames out of one 19 MiB stream. | ~2.7 GiB/s (6.8 ms) |
framing/decode_100k_Newline |
The same with newline framing. | ~6.1 GiB/s (2.7 ms) |
framing/encode_100k_* |
Frame 100,000 bodies (Content-Length / newline). | ~1.8 / ~1.4 GiB/s |
messages/decode_100k_requests |
Decode 100,000 requests: envelope, id, method, params validated. |
~1.6 M/s (~610 ns each) |
messages/decode_batch_1024 |
Decode a full batch of 1,024 requests. | ~460 µs |
messages/decode_and_write_100k_responses |
Decode 100,000 responses and write each back. | ~1.6 M/s |
endpoint/handle_100k_requests |
The whole server path, 100,000 times: decode, route, parse params, build a result, encode, frame, write. |
~420 K/s (~2.4 µs each) |
endpoint/run_100k_framed_requests |
The same, read from one Content-Length stream by run, written by Framed. |
~455 K/s |
endpoint/run_100k_queued |
The same through a Queued sink: a writer thread does the writes, and the time includes waiting for it to finish. |
~355 K/s |
pending/issue_then_settle_100k_out_of_order |
Issue 100,000 requests with deadlines, settle them in scrambled order. | ~1.2 M pairs/s |
The queued path costs a hand-off to the writer thread per batch; it buys senders that do not wait on the peer while the connection is open. Use Framed for a peer that always reads (a local pipe), Queued for anything that might stall.
Criterion writes per-benchmark reports to target/criterion/. Numbers vary by CPU and load; use the trend across runs, not a single absolute.
Design notes
- Exactly one response. A deferred request is tracked until answered; a
Responderdropped without answering (a worker returned early, a panic unwound through it) answersINTERNAL_ERRORitself. A cancelled request is answered by the endpoint (-32800for LSP) or not at all (MCP), and a late answer from the worker returnsError::AlreadyAnsweredinstead of reaching the wire. - Ids. Request ids are strings or integers within
i64.null, fractions, and other values are refused as invalid requests, as LSP and MCP require. A request whose id is still being answered by a deferred handler is refused under anullid, so the only response carrying that id is the real answer (a deliberate departure from the letter of JSON-RPC 2.0, which keepsnullfor ids that could not be read). Our own ids are positive integers counted from 1; answers to ids we are not waiting on are reported and dropped. - Malformed messages. A message with a
methodis answered under its own id when that id is valid and appears once; anything unreadable is answered undernull, as the specification requires. A broken response is never answered (an error under its id would read as an answer to one of the peer's requests); it ends the call it names instead. - Budgets.
Limitscaps message bytes (64 MiB), header bytes (8 KiB, the terminating blank line included), header lines (32), batch size (1024, or 0 for no batches), the length an oversized body may announce and still be skipped, and every JSON budget ofjsonx-lib(depth 128 by default). An oversized message is skipped as it arrives, never buffered, and answered; a broken frame ends the connection, since no reliable boundary is left. The defaults suit a local peer;Limits::network()andLimits::strict()suit remote ones, since a handler that parsesparamsbuilds a value tree that can be several times the message's size. - Output. A
Sinkruns under the endpoint's lock.Framedwrites in the sending thread, which is fine for a peer that always reads; withQueuedandoutboxa send does not wait on the peer: messages wait in a bounded queue (bytes, messages, and a no-progress write timeout), and a peer that stops reading closes the connection instead of freezing it. Closing takes the lock only to close the queue; the closing thread then waits for the drain without it, for at mostQueueLimits::close_timeout(10 s), and other threads see the connection closed at once. Messages this side sends before the gate opens are held, withinConfig::max_held_bytes. - Calls. Dropping a
Callbefore its outcome cancels the request: its pending slot is freed and the peer is told, as a dropped future should (aselect!that took another branch).Callis#[must_use]for that reason, andCall::detachkeeps a request outstanding without a waiter. - Deadlines. A request's deadline fires on its own: a timer thread, started with the first deadline and stopped when the connection closes, ends the call with
TimedOutand tells the peer to cancel, even when nothing else happens.Config::with_timer(false)leaves them to the caller. - Panics. Handler, interceptor, and event-hook panics are caught; so are panics in a
Sinkor aQueuedwriter, which close the connection as an I/O error. This needspanic = "unwind", Rust's default. An I/O error a handler forwards with?reaches the peer as "the transport failed", without the operating system's text. - Threads. Handlers run on the endpoint's thread, one at a time, in arrival order.
Peer,Call,Responder, andCancellerareSendand work from any thread. The endpoint's lock is held to encode a message and hand it to the sink, never while a handler or hook runs; withQueuedoroutbox, never during a write either. A message sent from a worker goes out at once, even while a handler runs. - Observability.
Endpoint::stats()andPeer::stats()give the connection's counters (traffic both ways, errors, timeouts, evictions, cancellations, panics) and gauges (pending, in flight, held). Thetracingfeature adds a span per call and events at documented levels. They record methods, ids, sizes, error codes, error messages, and panic messages, neverparams, results, or errordataas such. Text the peer chose is clipped and escaped; an error message may quote part of what the peer sent (aserdeerror names the value that did not fit), clipped to 256 bytes. - Serving.
Servercloses a connection that leaves a message unfinished formessage_timeout(30 s per message, counted from the read that brought its first bytes, so a client streaming messages back to back is never cut off; also the time a new connection has to send its first message) or stays idle foridle_timeout(300 s: no traffic either way and nothing in flight, so a client waiting on a long answer is kept, while a request of ours counts only within its timeout, so a client that never answers one cannot hold the connection; long-lived LSP or MCP sessions may wantwith_idle_timeout(None)), so silent or trickling sockets cannot hold itsmax_connectionsslots. Its defaults are for untrusted peers:Limits::network()(4 MiB messages) and an 8 MiB, 4,096-message queue per connection. Running out of descriptors or memory atacceptpauses the loop (10 ms, doubling to 1 s) instead of ending it.
Testing
tests/properties.rs holds the crate to independent references: a classifier built on a full jsonx-lib parse decides what every random envelope should decode to, and random batches check one response per request id. tests/fuzz_harness.rs runs the fuzz targets' checks on random, JSON-shaped, and mutated bytes in every cargo test. tests/endpoint.rs runs section 7 of the JSON-RPC 2.0 specification verbatim, tests/framing.rs runs the framings against malformed and hostile streams, tests/session.rs covers cancellation, timeouts, the gate, a client and server over a pipe, and races, tests/output.rs stalls the peer, and tests/adversarial.rs holds regression tests for the 0.2 review's findings; the 0.3 audits' regressions sit with the behaviour they cover (tests/output.rs, tests/server.rs, tests/session.rs, tests/typed.rs, tests/tracing.rs). Every rust example in this README and in docs/API.md is compiled and run as a doctest.
Cross-platform support
CI builds and tests on:
- Linux (x86_64), and 32-bit Linux (i686)
- macOS (Apple Silicon)
- Windows (x86_64)
and builds (without running) the no_std layers for bare metal (thumbv7em-none-eabihf).
The crate uses only std's threads, channels, clocks, and sockets, and no platform-specific code beyond Unix domain sockets in Server::serve_unix and how Server waits for a socket to become readable on Windows (a short timed peek, since shutting a socket down there does not wake a blocked read). Other targets (aarch64 Linux, x86_64 macOS) are expected to work but are not exercised by CI.
Contributing
See REPS.md for the engineering standards every change is held to, dev/DIRECTIVES.md for the definition of done, and dev/ROADMAP.md for what comes next. Before a PR: cargo fmt --all, cargo clippy --all-targets --all-features -- -D warnings, and cargo test --all-features must be clean.