flatwire (Rust)
Streaming JSON serialization that keeps memory flat and time linear. Stream
large collections element-by-element instead of materializing the whole payload,
so peak memory is bounded by the largest single element — not the collection
size. Built on serde_json; wire format is plain JSON.
Part of the cross-language flatwire project (identical API in Python, Node, .NET, Rust, Go, and Java).
Install
Usage
use ;
use Cursor;
// Whole value
let bytes = encode?;
let value = decode?;
// Stream a large collection — flat memory
let mut out = Vecnew;
encode_array?;
for element in decode_array
decode_array(reader).with_max_depth(200) bounds how deeply a single element may
nest before the iterator rejects the input (DoS guard); with_max_depth(0)
disables it.
API
| Function | Description |
|---|---|
encode(&value) |
value → Vec<u8> |
decode(&bytes) |
bytes → serde_json::Value |
encode_to(&value, &mut writer) |
stream a single value out |
decode_from(reader) |
read a single value |
encode_array(items, &mut writer) |
stream a large collection |
decode_array(reader) |
lazy Iterator over a large array |
Formats
Beyond JSON (default), the streaming array pair also speaks XML, binary MessagePack, and binary CBOR — same flat memory:
encode_array?;
for row in decode_array
encode_array?;
for row in decode_array
encode_array?;
for row in decode_array
MessagePack and CBOR are byte-identical across all six flatwire languages (see the conformance matrix).
Checked streams
Partial-stream failure semantics: wrap a streamed array in an envelope whose terminal status is written last, so the consumer distinguishes clean completion, an in-band producer error after N rows, and truncation.
use ;
encode_checked_array?; // writes ...,"complete":true} last
for item in decode_checked_array
The envelope is plain JSON, so a checked stream written in any flatwire language decodes in every other. See docs/FAILURE.md.
License
Apache-2.0 — see the repository.
Benchmarks
See the live benchmark dashboard and the cross-language summary.
Changelog
See CHANGELOG.md.