Module chewdata::step::generator

Generate an empty crate::DataResult that you can enrich with the crate::step::Transformer.

It possible to duplicate input data and enrich them after.

Actions

1 - Get a crate::Context from the input queue.
2 - Extract the crate::DataResult from the crate::Context.
3 - Clone the current crate::Context or it create a new one if empty.
4 - Push the crate::Context into the output queue.
5 - Go to the step 3 n times.
6 - Go to step 1 until the input queue is not empty.

Configuration

key	alias	Description	Default Value	Possible Values
type	-	Required in order to use generator step	generator	generator / g
name	alias	Name step	null	Auto generate alphanumeric value
data_type	data	Type of data used for the transformation. skip other data type	ok	ok / err
dataset_size	batch	Stack size limit before to push data into the resource though the connector	1000	unsigned number

Examples

[
    {
        "type": "generator",
        "name": "my_generator",
        "data_type": "ok",
        "dataset_size": 1000,
    },
    {
        "type": "transformer",
        "actions": [
            {
                "field":"firstname",
                "pattern": "{{ fake_first_name() }}"
            },
            {
                "field":"lastname",
                "pattern": "{{ fake_last_name() }}"
            },
            {
                "field":"city",
                "pattern": "{{ fake_city() }}"
            },
            {
                "field":"password",
                "pattern": "{{ fake_password(min = 5, max = 10) }}"
            },
            {
                "field":"color",
                "pattern": "{{ fake_color_hex() }}"
            }
        ]
    },
    {
        "type": "writer"
    }
]

No input.

output:

[
    {"firstname": "my firstname", "lastname": "my lastname", "city": "my city", "password": "my password", "color": "my color"},
    ...
]

Structs

• Generator