# miniconf: serialize/deserialize/access reflection for trees
[](https://crates.io/crates/miniconf)
[](https://docs.rs/miniconf)
[](https://matrix.to/#/#quartiq:matrix.org)
[](https://github.com/quartiq/miniconf/actions)
`miniconf` enables lightweight (`no_std`/no alloc) serialization, deserialization,
and access within a tree of heretogeneous types by keys.
## Example
See below for an example showing some of the features of the `Tree*` traits.
See also the documentation and doctests of the [`TreeKey`] trait for a detailed description.
Note that the example below focuses on JSON and slash-separated paths while in fact
any `serde` backend (or `dyn Any` trait objects) and many different `Keys`/`Transcode`
providers are supported.
```rust
use serde::{Deserialize, Serialize};
use miniconf::{Error, json, JsonPath, Traversal, Tree, TreeKey, Path, Packed, Node, Leaf, Metadata};
#[derive(Deserialize, Serialize, Default, Tree)]
pub struct Inner {
a: Leaf<i32>,
b: Leaf<i32>,
}
#[derive(Deserialize, Serialize, Default, Tree)]
pub enum Either {
#[default]
Bad,
Good,
A(Leaf<i32>),
B(Inner),
C([Inner; 2]),
}
#[derive(Tree, Default)]
pub struct Settings {
foo: Leaf<bool>,
enum_: Leaf<Either>,
struct_: Leaf<Inner>,
array: Leaf<[i32; 2]>,
option: Leaf<Option<i32>>,
#[tree(skip)]
#[allow(unused)]
skipped: (),
struct_tree: Inner,
enum_tree: Either,
array_tree: [Leaf<i32>; 2],
array_tree2: [Inner; 2],
tuple_tree: (Leaf<i32>, Inner),
option_tree: Option<Leaf<i32>>,
option_tree2: Option<Inner>,
array_option_tree: [Option<Inner>; 2],
}
let mut settings = Settings::default();
// Access nodes by field name
json::set(&mut settings,"/foo", b"true")?;
assert_eq!(*settings.foo, true);
json::set(&mut settings, "/enum_", br#""Good""#)?;
json::set(&mut settings, "/struct_", br#"{"a": 3, "b": 3}"#)?;
json::set(&mut settings, "/array", b"[6, 6]")?;
json::set(&mut settings, "/option", b"12")?;
json::set(&mut settings, "/option", b"null")?;
// Nodes inside containers
// ... by field name in a struct
json::set(&mut settings, "/struct_tree/a", b"4")?;
// ... or by index in an array
json::set(&mut settings, "/array_tree/0", b"7")?;
// ... or by index and then struct field name
json::set(&mut settings, "/array_tree2/0/a", b"11")?;
// ... or by hierarchical index
json::set_by_key(&mut settings, [8, 0, 1], b"8")?;
// ... or by packed index
let (packed, node): (Packed, _) = Settings::transcode([8, 1, 0]).unwrap();
assert_eq!(packed.into_lsb().get(), 0b1_1000_1_0);
assert_eq!(node, Node::leaf(3));
json::set_by_key(&mut settings, packed, b"9")?;
// ... or by JSON path
json::set_by_key(&mut settings, &JsonPath(".array_tree2[1].b"), b"10")?;
// Hiding paths by setting an Option to `None` at runtime
assert_eq!(json::set(&mut settings, "/option_tree", b"13"), Err(Traversal::Absent(1).into()));
settings.option_tree = Some(0.into());
json::set(&mut settings, "/option_tree", b"13")?;
// Hiding a path and descending into the inner `Tree`
settings.option_tree2 = Some(Inner::default());
json::set(&mut settings, "/option_tree2/a", b"14")?;
// Hiding items of an array of `Tree`s
settings.array_option_tree[1] = Some(Inner::default());
json::set(&mut settings, "/array_option_tree/1/a", b"15")?;
let mut buf = [0; 16];
// Serializing nodes by path
let len = json::get(&settings, "/struct_", &mut buf).unwrap();
assert_eq!(&buf[..len], br#"{"a":3,"b":3}"#);
// Tree metadata
let meta: Metadata = Settings::traverse_all().unwrap();
assert!(meta.max_depth <= 6);
assert!(meta.max_length("/") <= 32);
// Iterating over all leaf paths
for path in Settings::nodes::<Path<heapless::String<32>, '/'>, 6>() {
let (path, node) = path.unwrap();
assert!(node.is_leaf());
// Serialize each
match json::get(&settings, &path, &mut buf) {
// Full round-trip: deserialize and set again
Ok(len) => { json::set(&mut settings, &path, &buf[..len])?; }
// Some Options are `None`, some enum variants are absent
Err(Error::Traversal(Traversal::Absent(_))) => {}
e => { e.unwrap(); }
}
}
# Ok::<(), Error<serde_json_core::de::Error>>(())
```
## Settings management
One possible use of `miniconf` is a backend for run-time settings management in embedded devices.
It was originally designed to work with JSON ([`serde_json_core`](https://docs.rs/serde-json-core))
payloads over MQTT ([`minimq`](https://docs.rs/minimq)) and provides a MQTT settings management
client in the `miniconf_mqtt` crate and a Python reference implementation to interact with it.
Miniconf is agnostic of the `serde` backend/format, key type/format, and transport/protocol.
## Formats
`miniconf` can be used with any `serde::Serializer`/`serde::Deserializer` backend, and key format.
Explicit support for `/` as the path hierarchy separator and JSON (`serde_json_core`) is implemented.
Support for the `postcard` wire format with any `postcard` flavor and
any [`Keys`] type is implemented. Combined with the [`Packed`] key representation, this is a very
space-efficient serde-by-key API.
Blanket implementations are provided for all
`TreeSerialize`+`TreeDeserialize` types for all formats.
## Transport
`miniconf` is also protocol-agnostic. Any means that can receive or emit serialized key-value data
can be used to access nodes by path.
The `MqttClient` in the `miniconf_mqtt` crate implements settings management over the [MQTT
protocol](https://mqtt.org) with JSON payloads. A Python reference library is provided that
interfaces with it. This example discovers the unique prefix of an application listening to messages
under the topic `quartiq/application/12345` and set its `/foo` setting to `true`.
```sh
python -m miniconf -d quartiq/application/+ /foo=true
```
## Derive macros
For structs `miniconf` offers derive macros for [`macro@TreeKey`], [`macro@TreeSerialize`], [`macro@TreeDeserialize`], and [`macro@TreeAny`].
The macros implements the [`TreeKey`], [`TreeSerialize`], [`TreeDeserialize`], and [`TreeAny`] traits.
Fields/variants that form internal nodes (non-leaf) need to implement the respective `Tree{Key,Serialize,Deserialize,Any}` trait.
Leaf fields/items need to support the respective [`serde`] (and the desired `serde::Serializer`/`serde::Deserializer`
backend) or [`core::any`] trait.
Structs, enums, arrays, Options, and many other containers can then be cascaded to construct more complex trees.
See also the [`TreeKey`] trait documentation for details.
## Keys and paths
Lookup into the tree is done using a [`Keys`] implementation. A blanket implementation through [`IntoKeys`]
is provided for `IntoIterator`s over [`Key`] items. The [`Key`] lookup capability is implemented
for `usize` indices and `&str` names.
Path iteration is supported with arbitrary separator `char`s between names.
Very compact hierarchical indices encodings can be obtained from the [`Packed`] structure.
It implements [`Keys`].
## Limitations
* `enum`: The derive macros don't support enums with record (named fields) variants or tuple variants with more than one (non-skip) field. Only unit, newtype and skipped variants are supported. Without the derive macros, any `enum` is still however usable as a `Leaf` node. Note also that netwype variants with a single inline tuple are supported.
* The derive macros only support flattening in non-ambiguous situations (single field structs and single variant enums, both modulo skipped fields/variants and unit variants).
## Features
* `json-core`: Enable helper functions for serializing from and
into json slices (using the `serde_json_core` crate).
* `postcard`: Enable helper functions for serializing from and
into the postcard compact binary format (using the `postcard` crate).
* `derive`: Enable the derive macros in `miniconf_derive`. Enabled by default.
## Reflection
`miniconf` enables certain kinds of reflective access to heterogeneous trees.
Let's compare it to [`bevy_reflect`](https://crates.io/crates/bevy_reflect)
which is a comprehensive and mature reflection crate:
`bevy_reflect` is thoroughly `std` while `miniconf` aims at `no_std`.
`bevy_reflect` uses its `Reflect` trait to operate on and pass nodes as trait objects.
`miniconf` uses serialized data or `Any` to access leaf nodes and pure "code" to traverse through internal nodes.
The `Tree*` traits like `Reflect` thus give access to nodes but unlike `Reflect` they are all decidedly not object-safe
and can not be used as trait objects. This allows `miniconf` to support non-`'static` borrowed data
(only for `TreeAny` the leaf nodes need to be `'static`)
while `bevy_reflect` requires `'static` for `Reflect` types.
`miniconf`supports at least the following reflection features mentioned in the `bevy_reflect` README:
* ➕ Derive the traits: `miniconf` has `Tree*` derive macros and blanket implementations for arrays and Options.
Leaf nodes just need some impls of `Serialize/Deserialize/Any` where desired.
* ➕ Interact with fields using their names
* ➖ "Patch" your types with new values: `miniconf` only supports limited changes to the tree structure at runtime
(`Option` and custom accessors) while `bevy_reflect` has powerful dynamic typing tools.
* ➕ Look up nested fields using "path strings": In addition to a superset of JSON path style
"path strings" `miniconf` supports hierarchical indices and bit-packed ordered keys.
* ➕ Iterate over struct fields: `miniconf` Supports recursive iteration over node keys.
* ➕ Automatically serialize and deserialize via Serde without explicit serde impls:
`miniconf` supports automatic serializing/deserializing into key-value pairs without an explicit container serde impls.
* ➕ Trait "reflection": Together with [`crosstrait`](https://crates.io/crates/crosstrait) supports building the
type registry and enables casting from `dyn Any` returned by `TreeAny` to other desired trait objects.
Together with [`erased-serde`](https://crates.io/crates/erased-serde) it can be used to implement node serialization/deserialization
using `miniconf`'s `TreeAny` without using `TreeSerialize`/`TreeDeserialize` similar to `bevy_reflect`.
Some tangential crates:
* [`serde-reflection`](https://crates.io/crates/serde-reflection): extract schemata from serde impls
* [`typetag`](https://crates.io/crates/typetag): "derive serde for trait objects" (local traits and impls)
* [`deflect`](https://crates.io/crates/deflect): reflection on trait objects using adjacent DWARF debug info as the type registry
* [`intertrait`](https://crates.io/crates/intertrait): inspiration and source of ideas for `crosstrait`
## Functional Programming, Polymorphism
The type-heterogeneity of `miniconf` also borders on functional programming features. For that crates like the
following may also be relevant:
* [`frunk`](https://crates.io/crates/frunk)
* [`lens-rs`](https://crates.io/crates/lens-rs)/[`rovv`](https://crates.io/crates/rovv)