Crate structio

Expand description

Structs in, structs out, in the spirit of Glaze.

Values are read straight into your types and written straight out of them. There is no token stream and no intermediate document model: a field’s bytes are converted exactly once, into the member that will hold them.

#[derive(Default, PartialEq, Debug)]
struct Config {
    name: String,
    port: u16,
    hosts: Vec<String>,
}

structio::object!(Config { name, port, hosts });

let json = r#"{"name":"api","port":8080,"hosts":["a","b"]}"#;
let config: Config = structio::from_str(json).unwrap();
assert_eq!(config.port, 8080);
assert_eq!(structio::to_string(&config), json);

§Two formats, one schema

object! declares a struct’s field list once. Both formats read and write against it, so the same type round-trips through either without a second declaration:

json is JSON text.
beve is BEVE, a tagged binary format that keeps JSON’s shape and self-description while storing numbers as numbers and numeric arrays as contiguous blocks.

let sample = Sample { id: 7, values: vec![1.5, 2.5, 3.5] };

let text = structio::to_string(&sample);
let binary = structio::to_beve(&sample);

assert_eq!(structio::from_str::<Sample>(&text).unwrap(), sample);
assert_eq!(structio::from_beve::<Sample>(&binary).unwrap(), sample);

§Objects and arrays

object! encodes a struct by key. array! encodes it by position, for a type whose field names carry nothing a reader does not already have:

#[derive(Default, PartialEq, Debug)]
struct Vec3 { x: f64, y: f64, z: f64 }
structio::array!(Vec3 [x, y, z]);

let v = Vec3 { x: 1.0, y: 2.0, z: 3.0 };
assert_eq!(structio::to_string(&v), "[1,2,3]");

Nothing is hashed and no keys go on the wire, at the cost of a schema that cannot change shape: an array of the wrong length is an error, where an object can be read with a policy that steps over a field it does not recognize.

transparent! is the third reading of a struct: a one-field wrapper written as that field alone, for the newtype that exists to keep two u64s apart in Rust and means nothing to either format.

#[derive(Default, PartialEq, Debug)]
struct UserId(u64);
structio::transparent!(UserId { 0 });

assert_eq!(structio::to_string(&UserId(7)), "7");

§Enums

An enum’s schema is its variant names, and they go on the wire as names rather than as positions. A variant carrying nothing is written as its name; a variant carrying a value is written as an object of one member keyed by that name, and is marked (_) in the declaration.

#[derive(Default, PartialEq, Debug)]
enum Shape {
    #[default]
    Empty,
    Sides(u32),
}
structio::tagged_enum!(Shape { Empty, Sides(_) });

assert_eq!(structio::to_string(&Shape::Empty), "\"Empty\"");
assert_eq!(structio::to_string(&Shape::Sides(3)), r#"{"Sides":3}"#);
assert_eq!(structio::from_str::<Shape>(r#"{"Sides":3}"#).unwrap(), Shape::Sides(3));

unit_enum! is the same declaration for an enum whose variants all carry nothing, and will not compile if one of them does, so its wire form is a plain string and stays one.

A variant carrying nothing reads back from either form, but one carrying a value has only the object form, and the two macros’ pages say what each error means. docs/enums.md is the long form.

The crate root re-exports the JSON entry points unqualified, because JSON is what most callers want first. The BEVE ones carry the format in the name at the root (to_beve, from_beve) and drop it inside the module, so beve::to_vec and to_beve are the same function.

§Design

No dependencies. Standard library only.
No proc-macros required. object!, array! and tagged_enum! are macro_rules! macros, so there is no proc-macro crate to build and link for the host before your code can, and nothing extra when cross-compiling. It buys a smaller dependency graph rather than a faster build: expanding a declaration costs about what a derive costs. The derive feature adds #[derive(Structio)], a dependency-free front end that expands to the same macros, for a type you own and would rather not restate.
A declaration is checked against its type. Naming a field the struct does not have has always been an error; leaving one out is now one too, naming the field. End a declaration with .. where the omission is deliberate.
Keys are hashed at compile time. KeyMap::build runs during const evaluation and picks the cheapest perfect hash that fits your key set, from a single byte comparison up to a full key hash. Both formats look keys up in that one table.
Reads reuse what you already own. Parsing into an existing value refills its buffers instead of reallocating them.

§What it does not do

This is for statically known types. Reading a document you have no type for is a different matter, and Value is the tree for that case: a shape nothing declares and something walks by path, the register map a device publishes, a setting stored under a key some plugin chose. It is a destination like any other, not a stage every read passes through, and a value you could have declared a type for is better read into that type. A body you forward rather than look at is a different problem again, and a tree is the wrong answer to it: json::Raw carries one through as the text that spelled it. See value for what it holds and what it costs.

A BEVE document can still be looked into without being decoded whole. from_beve_at reads the one value a JSON Pointer names and steps over everything else, and validate_beve checks a document is well formed without decoding any of it. What comes back from the first is still a type you declared.

beve_to_json hands back a BEVE document’s contents without a type and without a tree: it rewrites the whole document as JSON in a single walk. See transcode for what survives the trip and what does not.

§Complex numbers and matrices

BEVE’s core covers what JSON covers; its extensions cover what scientific data needs on top of that. Complex and Matrix are those two, as ordinary types that work in both formats:

use structio::{Complex, Matrix, MatrixLayout};

let signal = vec![Complex::new(1.0f64, 2.0), Complex::new(3.0, -4.0)];
let bytes = structio::to_beve(&signal);
assert_eq!(structio::from_beve::<Vec<Complex<f64>>>(&bytes).unwrap(), signal);

let m = Matrix::new(MatrixLayout::RowMajor, vec![2, 3], (0..6i32).collect()).unwrap();
assert_eq!(
    structio::to_string(&m),
    r#"{"layout":"layout_right","extents":[2,3],"value":[0,1,2,3,4,5]}"#
);
assert_eq!(structio::from_beve::<Matrix<i32>>(&structio::to_beve(&m)).unwrap(), m);

A run of complex numbers is one header and one block, so a Vec<Complex<f64>> moves in a single copy exactly as a Vec<f64> does, and a Matrix<Complex<f64>> stores its data that way with no case of its own. See ext.

§Options

Whether the JSON is indented, whether a member that would be null is written at all, and whether a key nothing claims is refused, are decided at compile time by a policy type. Every entry point has a _with twin that takes one, and the plain one is Standard:

use structio::{Pretty, SkipNull, to_string, to_string_with};

let server = Server { port: 8080, tls: None };

assert_eq!(to_string(&server), r#"{"port":8080,"tls":null}"#);
assert_eq!(
    to_string_with::<Pretty, _>(&server),
    "{\n  \"port\": 8080,\n  \"tls\": null\n}"
);
assert_eq!(to_string_with::<SkipNull, _>(&server), r#"{"port":8080}"#);

Indentation puts every element of an array on a line of its own, which is not what a run of numbers wants. PrettyInlineArrays keeps each array on the line it began on and indents everything else:

use structio::{PrettyInlineArrays, to_string_with};

let sample = Sample { id: 7, values: vec![1.5, 2.5, 3.5] };

assert_eq!(
    to_string_with::<PrettyInlineArrays, _>(&sample),
    "{\n  \"id\": 7,\n  \"values\": [1.5, 2.5, 3.5]\n}"
);

Text that is already JSON takes the same policy from the other side. prettify lays out a document that did not come from a Write impl, and emits its whitespace through the same writer, so the result is what writing the same data would have produced:

use structio::{PrettyInlineArrays, json::prettify_with, prettify};

assert_eq!(prettify(r#"{"a":[1,2]}"#).unwrap(), "{\n  \"a\": [\n    1,\n    2\n  ]\n}");
assert_eq!(
    prettify_with::<PrettyInlineArrays>(r#"{"a":[1,2]}"#).unwrap(),
    "{\n  \"a\": [1, 2]\n}"
);

minify goes the other way, and has no layout to agree with: it copies the document through and drops the whitespace between its tokens. That needs nothing but the strings located, so it neither reads a value nor checks a bracket, which is what makes it the fastest thing here.

assert_eq!(structio::minify("{\n  \"a\": [1, 2]\n}").unwrap(), r#"{"a":[1,2]}"#);

Reading takes a policy the same way. It has three settings, and only one of them is on by default: a key that no field claims is an ErrorCode::UnknownKey, which catches a typo or the wrong document rather than passing over it in silence. Ask for SkipUnknown to read a subset of a larger document. The other way round, a declared field the document leaves out is simply left as the destination had it, since reading is into a value that already exists. Mark a field #[required] in the declaration and its absence is an ErrorCode::MissingKey under every policy, which is what a mixed schema wants; RequireKeys says the same of every field at once.

use structio::{ErrorCode, SkipUnknown, from_str, from_str_with};

let doc = r#"{"port":8080,"debug":true}"#;

assert_eq!(from_str::<Server>(doc).unwrap_err().code, ErrorCode::UnknownKey);
assert_eq!(from_str_with::<SkipUnknown, Server>(doc).unwrap().port, 8080);

use structio::{ErrorCode, RequireKeys, from_str, from_str_with};

let doc = r#"{"port":8080}"#;

assert_eq!(from_str::<Server>(doc).unwrap().host, "");
let e = from_str_with::<RequireKeys, Server>(doc).unwrap_err();
assert_eq!(e.code, ErrorCode::MissingKey);
// A member that is not there has no position of its own, so the offset
// names the object and the error names the member.
assert_eq!(e.key, Some("host"));

The third is AllowComments, which reads JSONC: // and /* */ wherever whitespace is allowed, for the documents people edit by hand.

The setting you do not ask for costs nothing: a compact writer emits no indentation code at all. Combinations are your own unit struct and an impl, since every constant on Options has a default. See options.

§Streaming

Documents that do not fit, or have not fully arrived, go through json::stream and beve::stream: to_writer drains into an std::io::Write, Documents pulls a sequence of values out of an std::io::Read, and Feed takes chunks pushed at it. The BEVE counterparts are beve::to_writer, beve::Documents and beve::Feed, and they hand out the elements of a typed array one at a time as readily as whole records. beve_to_json_writer drains a transcode into a sink the same way.

Re-exports§

pub use error::Error;
pub use error::ErrorCode;
pub use error::Result;
pub use error::StreamError;
pub use error::StreamResult;
pub use ext::Complex;
pub use ext::Matrix;
pub use ext::MatrixLayout;
pub use ext::MatrixRef;
pub use keymap::KeyMap;
pub use map::OrderedMap;
pub use options::AllowComments;
pub use options::Options;
pub use options::Pretty;
pub use options::PrettyInlineArrays;
pub use options::RequireKeys;
pub use options::SkipNull;
pub use options::SkipUnknown;
pub use options::Standard;
pub use value::Number;
pub use value::Object;
pub use value::Value;
pub use value::from_value;
pub use value::from_value_with;
pub use value::to_value;
pub use json::Documents;
pub use json::Feed;
pub use json::Mode;
pub use json::append;
pub use json::append_with;
pub use json::from_reader;
pub use json::from_reader_with;
pub use json::from_slice;
pub use json::from_slice_with;
pub use json::from_str;
pub use json::from_str_with;
pub use json::minify;
pub use json::minify_into;
pub use json::minify_into_with;
pub use json::minify_with;
pub use json::prettify;
pub use json::prettify_into;
pub use json::prettify_into_with;
pub use json::prettify_with;
pub use json::read_into;
pub use json::read_into_with;
pub use json::to_string;
pub use json::to_string_with;
pub use json::to_vec;
pub use json::to_vec_with;
pub use json::to_writer;
pub use json::to_writer_buffered;
pub use json::to_writer_buffered_with;
pub use json::to_writer_with;
pub use json::write_into;
pub use json::write_into_with;
pub use beve::append as append_beve;
pub use beve::append_aligned as append_beve_aligned;
pub use beve::append_aligned_with as append_beve_aligned_with;
pub use beve::append_with as append_beve_with;
pub use beve::from_reader as from_beve_reader;
pub use beve::from_reader_array as from_beve_reader_array;
pub use beve::from_reader_with as from_beve_reader_with;
pub use beve::from_slice as from_beve;
pub use beve::from_slice_at as from_beve_at;
pub use beve::from_slice_at_with as from_beve_at_with;
pub use beve::from_slice_with as from_beve_with;
pub use beve::read_array_into as read_beve_array_into;
pub use beve::read_into as read_beve_into;
pub use beve::read_into_at as read_beve_into_at;
pub use beve::read_into_at_with as read_beve_into_at_with;
pub use beve::read_into_with as read_beve_into_with;
pub use beve::size as beve_size;
pub use beve::size_after as beve_size_after;
pub use beve::size_after_with as beve_size_after_with;
pub use beve::size_aligned as beve_size_aligned;
pub use beve::size_aligned_after as beve_size_aligned_after;
pub use beve::size_aligned_after_with as beve_size_aligned_after_with;
pub use beve::size_aligned_with as beve_size_aligned_with;
pub use beve::size_with as beve_size_with;
pub use beve::slice_ref as beve_slice_ref;
pub use beve::to_vec as to_beve;
pub use beve::to_vec_aligned as to_beve_aligned;
pub use beve::to_vec_aligned_with as to_beve_aligned_with;
pub use beve::to_vec_with as to_beve_with;
pub use beve::to_writer as to_beve_writer;
pub use beve::to_writer_buffered as to_beve_writer_buffered;
pub use beve::to_writer_buffered_with as to_beve_writer_buffered_with;
pub use beve::to_writer_with as to_beve_writer_with;
pub use beve::validate as validate_beve;
pub use beve::validate_reader as validate_beve_reader;
pub use beve::write_into as write_beve_into;
pub use beve::write_into_with as write_beve_into_with;
pub use transcode::beve_to_json;
pub use transcode::beve_to_json_into;
pub use transcode::beve_to_json_into_with;
pub use transcode::beve_to_json_with;
pub use transcode::beve_to_json_writer;
pub use transcode::beve_to_json_writer_buffered;
pub use transcode::beve_to_json_writer_buffered_with;
pub use transcode::beve_to_json_writer_with;

Modules§

beve: BEVE: the same structs, as tagged binary.
case: Compile-time case conversion for keys and variant names.
error: Error types.
ext: The two BEVE extensions that carry data, as ordinary Rust types.
json: JSON: text in, text out.
keymap: Compile-time perfect hashing of object keys.
map: An insertion-ordered map with String keys.
options: Compile-time options, for reading and for writing.
transcode: Between the formats, without a type in the middle.
value: A value whose shape is not known at compile time.

Macros§

array: Declare a struct’s schema as a positional array, for every format.
beve_array: Declare a struct’s schema as a positional array, for BEVE alone.
beve_object: Declare a struct’s schema for BEVE alone.
beve_tagged_enum: Declare an enum for BEVE alone.
beve_transparent: Declare a one-field struct as that field alone, for BEVE only.
json_array: Declare a struct’s schema as a positional array, for JSON alone.
json_object: Declare a struct’s schema for JSON alone.
json_tagged_enum: Declare an enum for JSON alone.
json_transparent: Declare a one-field struct as that field alone, for JSON only.
object: Declare a struct’s schema, for every format.
tagged_enum: Declare an enum, for every format.
transparent: Declare a one-field struct as that field alone.
unit_enum: Declare an enum whose variants carry nothing, for every format.
value: Build a Value from JSON-shaped syntax.

Structs§

Same: The identity adapter: read and write this position the way the type itself would.

Traits§

Elements: The length of a struct encoded as a positional array.
Keys: The key schema of a struct, and its compile-time perfect hash.
ReadOwned: Convenience bound for a function that parses a T out of a document it owns: readable in every format this crate supports, from any input, and constructible.
ReadWrite: Convenience bound for generic containers: readable and writable in every format this crate supports, from any input.
Variants: The variant names of an enum, and their compile-time perfect hash.
Write: Convenience bound for generic containers that are only ever written: writable in every format this crate supports.

Functions§

assert_tag_not_a_field: Refuse an internally tagged declaration whose tag is also a field of the variant’s payload.

Derive Macros§

Structio: Declare a type’s schema from its definition: #[derive(Structio)].

Crate structio

Crate structio Copy item path

§Two formats, one schema

§Objects and arrays

§Enums

§Design

§What it does not do

§Complex numbers and matrices

§Options

§Streaming

Re-exports§

Modules§

Macros§

Structs§

Traits§

Functions§

Derive Macros§

Crate structio