structio

High-performance JSON and BEVE for Rust structs. No dependencies, no proc-macros required, no intermediate representation.

Values are read straight into your types and written straight out of them. There is no token stream and no intermediate document model: a field's bytes are converted exactly once, into the member that will hold them.

Built in the spirit of Glaze.

Installation

[dependencies]
structio = "0.8"

Rust 2024 edition, MSRV 1.96. For #[derive(Structio)], enable the derive feature:

structio = { version = "0.8", features = ["derive"] }

Quickstart

use structio::{from_beve, from_str, to_beve, to_string};

#[derive(Default, Debug, PartialEq)]
struct Config {
    name: String,
    port: u16,
    hosts: Vec<String>,
}

structio::object!(Config { name, port, hosts });

fn main() -> Result<(), structio::Error> {
    let text = r#"{"name":"api","port":8080,"hosts":["a","b"]}"#;

    // JSON, straight into the struct.
    let config: Config = from_str(text)?;
    assert_eq!(config.port, 8080);
    assert_eq!(to_string(&config), text);

    // The same schema, as BEVE binary. No second declaration.
    let bytes = to_beve(&config);
    let same: Config = from_beve(&bytes)?;
    assert_eq!(same, config);

    Ok(())
}

That is the whole API surface most work needs: declare the schema once, then convert. This example is examples/quickstart.rs, and a test fails if the two stop matching.

With the derive feature, #[derive(Structio)] on the struct takes the place of the object! line and expands to the same declaration. The derive has its attributes.

Is this the right library for you?

serde and serde_json are the right default for most Rust projects, and this does not try to replace them. They have a vast ecosystem, support dozens of formats, and offer field attributes this crate has no answer to.

Reach for structio when one of these matters more:

Do not reach for it if most of your documents have no shape you know at compile time (there is a Value tree for those, but the crate is built around declared types), you need a format other than JSON or BEVE, or you rely on serde attributes that have no counterpart here yet, such as flatten, untagged, adjacent tagging or skip_serializing_if. See what it does not do.

Highlights

• No dependencies. Standard library only.
• No proc-macros required. object!, array!, transparent!, unit_enum! and tagged_enum! are macro_rules! macros. #[derive(Structio)], behind the derive feature, is a front end that expands to them; see the derive.
• A declaration is checked against its type. A field added to the struct and forgotten in the declaration is a build error naming the field, not a member that quietly stops being written. .. at the end says an omission is deliberate.
• One schema, both formats. The field list and its hash table are declared once and shared; only the bytes differ.
• Objects, arrays, or the field alone. object! declares a struct by key. array! declares one by position, for tuple structs and for types like a coordinate whose field names carry nothing. transparent! writes a one-field newtype as that field, so UserId(7) is 7.
• Renaming without breaking old documents. A field or variant can answer to more than one name: object!(Settings { timeout | "timeout_ms" }) reads either key and writes timeout, so documents already written under the old spelling still read.
• One direction when that is all you need. write_only declares the write half alone, so a type that is only ever emitted needs no Read impls and no Default for its fields.
• Required members, one at a time. A field marked #[required] has to be in the document; the rest keep their defaults when it is quiet. Mixed schemas are most schemas, so this is the type's business rather than a reader policy.
• Enums by name, not by index. unit_enum! writes a variant as its name; tagged_enum! writes one carrying a value as a one-member object keyed by that name; tagged_enum!(.. as tag "kind") puts that name inside the payload's object instead, the convention most JSON APIs use. Adding or reordering variants does not change what a document already means, and an internal tag does not have to be the object's first member.
• Keys are hashed at compile time. The macro picks the cheapest perfect hash that fits your key set, from a single byte comparison up to a full key hash.
• Reads reuse what you already own. Parsing into an existing value refills its buffers instead of reallocating them, so a loop over records of the same shape settles into no allocation at all.
• JSON that goes through untouched. A json::Raw field holds one value as the text that spelled it, so a body a gateway forwards keeps the key order, the number spellings and the escapes it arrived with. Reading borrows the span out of the document and writing copies it back; under Pretty it is laid out again at the depth it actually sits at, rather than left as an unindented blob.
• One field out of a BEVE document. from_beve_at(&bytes, "/servers/1/port") walks the headers in front of the value and decodes nothing else, and validate_beve checks a document is well formed without decoding any of it.
• Both formats stream, both ways. Documents and Feed hand out one value at a time from a reader or from chunks pushed at you, in JSON and in BEVE, so a file too large to hold costs one record rather than the file. A BEVE typed array streams element by element too.
• Arrays a reader can point at. to_beve_aligned writes BEVE's aligned typed and complex arrays, padding each numeric payload onto its own element width so the block can be borrowed rather than copied. A Cow<'de, [f64]> field takes that borrow where the document allows it and copies where it does not. The same document either way, and every reader here takes both forms.
• A tree when there is no type. Value holds a document nothing declares, in either format, and is walked by key, index or JSON Pointer. Its objects keep their member order, and the map beneath them, OrderedMap, works as a declared field too.
• A BEVE document you have no type for. beve_to_json(&bytes) rewrites it as JSON in one walk, with no tree and no schema, which is the answer to "what is actually in this file".
• Complex numbers and matrices. Complex<T> and Matrix<T> cover BEVE's two data-carrying extensions, in both formats. A Vec<Complex<f64>> is one header and one block, so it moves in a single copy exactly as a Vec<f64> does.
• Errors carry a byte offset, and Error::display_with(input) renders it with a line, column, and caret. A missing key also names itself, the offset there being able to point only at the object that lacks it, and Error::key_in(input) reads an unknown key's name back out of the document.

API

The crate root carries the JSON entry points unqualified and the BEVE ones with the format in the name. Inside the modules the format is dropped, so structio::to_beve and structio::beve::to_vec are the same function.

JSON

BEVE

read_into and write_into are the ones to reach for in a loop. read_into is also the way in for a type with no meaningful zero value, and a placeholder to read over does not have to be public. It drops the Default a returning function needs only for the T you hand it, though: whatever the read has to construct still needs one, which is a growing Vec's new elements, a map's values, an Option's payload and an enum variant's payload. A field the read only ever fills in place does not, so Box<T> and [T; N] hold a type with no Default where Vec<T> cannot.

A generic function that parses a T out of a buffer it owns bounds it by ReadOwned, which is Default plus a read that borrows nothing from the input.

Without a declared type

Between the formats

Documentation

cargo doc --open builds the API reference.

Performance in one paragraph

Against Glaze on identical documents, reading is 82-117% and writing 65-163% depending on the type, with reading level on the representative mixed document. Writing integers and booleans is faster than Glaze; strings and floats are below it. Output is byte-identical to Glaze's on every benchmark document, floats included. Figures, methodology, and an important caveat about how sensitive the benchmark is to code layout: docs/performance.md.

No comparison against serde_json has been run, so please do not infer one.

What it does not do

Statically known types first. A document you have no type for reads into Value, a plain tree with the accessors a tree needs, meant for a shape nothing declares and something walks by path: the register map a device publishes, a setting stored under a key some plugin chose. It is a destination, not a stage every read passes through, and anything you could declare a type for is better read into that type. A body you forward rather than look at is a different problem, and a tree is the wrong answer to it: Value keeps an object's member order but respells its numbers, decodes its escapes and lays the tokens out afresh, so what comes out is a different document than the one that arrived. json::Raw is the field that carries one through unchanged. beve_to_json turns any BEVE document into JSON without either.

No json_to_beve. BEVE prefixes every container with its count and JSON gives that up only at the end, so the reverse of the above is not the same one-pass walk. JSON with a schema is from_str::<T> and then to_beve.

Two formats. Of the formats Glaze supports, JSON and BEVE are here; CSV, TOML, and the rest are not.

Few options. Options covers indentation, keeping an array on the line it began on, leaving null members out, refusing a key nothing claims, requiring every key the schema declares, and reading JSONC comments, which is where Glaze's glz::opts starts. Requiring some of the keys is a property of the schema rather than a policy: mark those fields #[required].

Not every serde attribute. Keys can be renamed one at a time or by a case rule and given aliases, fields can be marked #[required] or left out, a field can name an adapter, a newtype can be transparent, a type can be write-only, and an enum can be tagged externally or internally. There is no flatten or untagged, and neither is planned. Adjacent tagging, skip_serializing_if, per-field defaults and one-direction fields are planned as later stages of the derive. The derive's table maps each serde attribute to its counterpart.

Foreign types need an adapter or a wrapper. Rust's orphan rule means you cannot describe a type from another crate the way you can specialize glz::meta for any C++ type. A field can name an adapter that says how its type is read and written, which keeps the type out of your API; a newtype is still the answer when the foreign type has no Default, or when it appears in many structs, and declared with transparent! it adds nothing to the document.

No depth limit on writing. Reading refuses a document nested past 256 levels, because its depth is the sender's choice. A value being written is the program's own, so its depth is the program's to bound, as it already is for dropping that value; one deep enough overflows the stack. docs/design.md has why.

Status

Version 0.8.0. The API is not yet frozen and the version number should be taken at face value. What changes between releases is in CHANGELOG.md.

What that does not mean is untested. See docs/correctness.md for the fuzzing, the exhaustive f32 sweep, the Miri coverage of every unsafe block, and the byte-for-byte cross-check of the BEVE encoding against an independent implementation.

License

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Third-party code

src/num/zmij.rs is a port of zmij by Victor Zverovich, used for shortest round-trip float formatting. The algorithm, its lookup-table seeds, and its SWAR digit split are his work, and it carries his copyright under the terms in LICENSE-THIRD-PARTY. Nothing else in the crate is derived from another project.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.