fluvio-jolt 0.2.0

Overview

JSON to JSON transformation where the "specification" for the transform is itself a JSON document.

Port of Java Jolt library written in Rust.

Usage Example

Add fluvio-jolt crate to your Cargo.toml file:

[dependencies]
fluvio-jolt = { version = "0.1"}

Then, for example, if you want to repack your JSON record, you can do the following:

use serde_json::{json, Value};
use fluvio_jolt::{transform, TransformSpec};

let input: Value = serde_json::from_str(r#"
    {
        "id": 1,
        "name": "John Smith",
        "account": {
            "id": 1000,
            "type": "Checking"
        }
    }
"#).unwrap();

let spec: TransformSpec =
serde_json::from_str(r#"[
    {
      "operation": "shift",
      "spec": {
        "name": "data.name",
        "account": "data.account"
      }
    }
  ]"#).unwrap();

let output = transform(input, &spec);

assert_eq!(output, json!({
    "data" : {
      "name": "John Smith",
      "account": {
        "id": 1000,
        "type": "Checking"
      }
    }
}));

Supported Operations

1. shift: copy data from the input tree and put it the output tree
2. default: apply default values to the tree
3. remove: remove data from the tree

Specification

Composes a list of operation specifications. Each operation has its own DSL (Domain Specific Language) in order to facilitate its narrow job.

use fluvio_jolt::TransformSpec;

let spec: TransformSpec =
serde_json::from_str(r#"[
    {
      "operation": "shift",
      "spec": {
        "name": "data.name",
        "account": "data.account"
      }
    }
  ]"#).unwrap();

Shift operation

Specifies where the data from the input JSON should be placed in the output JSON, or in other words, how the input JSON/data should be shifted around to make the output JSON/data.

At a base level, a single shift operation is a mapping from an input path to an output path, similar to the mv command in Unix, mv /var/data /var/backup/data.

The input path is a JSON tree structure, and the output path is flattened "dot notation" path notation.

For example, given this simple input JSON:

A simple spec could be constructed by copying that input, and modifying it to supply an output path for each piece of data:

would produce the following output JSON:

Wildcards

The shift specification on the keys level supports wildcards and conditions:
1. * - match everything
2. name1|name2|nameN - match any of the specified names

& lookup allows referencing the values captured by the * or |.
It allows for specs to be more compact. For example, for this input:

to get the output:

the spec with wildcards would be:

If you want only id and name in the output, the spec is:

& wildcard also allows to dereference any level of the path of given node:

for the input:

will produce:

Default operation

Applies default values if the value is not present in the input JSON.

For example, given this simple input JSON:

with the following specification for default operation:

the output JSON will be:

As you can see, the field mobile remains not affected while the code has a default '+1' value.

Remove operation

Removes content from the input JSON. The spec structure matches the input JSON structure. The value of fields is ignored.

For example, given this simple input JSON:

you can remove the country by the following specification for remove operation:

the output JSON will be:

Contributing

If you'd like to contribute to the project, please read our Contributing guide.

License

This project is licensed under the Apache license.