Expand description
Substrait: Cross-Language Serialization for Relational Algebra
Serialization and deserialization
This crate provides generated types to serialize and deserialize Substrait data.
Protobuf
Protobuf serialization and deserialization are provided via prost in the proto module.
Example
Serialize and deserialize a plan
use prost::Message;
use substrait::proto::Plan;
let plan = Plan::default();
// Serialize the plan
let encoded = plan.encode_to_vec();
// Deserialize the buffer to a Plan
let decoded = Plan::decode(encoded.as_slice())?;
assert_eq!(plan, decoded);Serde support
There are two (non-default) features available that provide derived Deserialize and Serialize implementations for the generated types.
Note that these features are mutually exclusive. The main difference between
those implementations are the field name case convention and field value
encoding. The examples below show how the minor_number field name in
Version matches the Protobuf field name with the serde
feature whereas it expects a lower camel case minorNumber field name with
the pbjson feature enabled. Please refer to the Protobuf JSON Mapping
documentation for more details.
serde
This adds #[serde(Deserialize, Serialize)] to all generated Protobuf
types. In addition, to match Protobuf defaults for missing optional data,
this adds [serde(default)] to all messages.
Example
Deserialize a plan version using the serde feature
use substrait::proto::Version;
let version_json = r#"{
"minor_number": 21
}"#;
let version = serde_json::from_str::<Version>(version_json)?;
assert_eq!(
version,
Version {
minor_number: 21,
..Default::default()
}
);pbjson
This generates serde implementation that match the Protobuf JSON Mapping via pbjson.
Example
Deserialize a plan version using the pbjson feature
use substrait::proto::Version;
let version_json = r#"{
"minorNumber": 21
}"#;
let version = serde_json::from_str::<Version>(version_json)?;
assert_eq!(
version,
Version {
minor_number: 21,
..Default::default()
}
);Text
Substrait defines a YAML schema for extensions. Types with serialization and deserialization support for these are provided via typify in the text module.
Example
Read a simple extension
use substrait::text::simple_extensions::SimpleExtensions;
let simple_extension_yaml = r#"
%YAML 1.2
---
scalar_functions:
-
name: "add"
description: "Add two values."
impls:
- args:
- name: x
value: i8
- name: y
value: i8
options:
overflow:
values: [ SILENT, SATURATE, ERROR ]
return: i8
"#;
let simple_extension = serde_yaml::from_str::<SimpleExtensions>(simple_extension_yaml)?;
assert_eq!(simple_extension.scalar_functions.len(), 1);
assert_eq!(simple_extension.scalar_functions[0].name, "add");