Expand description
Write data into the crate::document through the crate::connector.
§Actions
1 - Get a crate::Context from the input queue.
2 - Extract the crate::DataResult from the crate::Context.
2 - Write data in the crate::document though the crate::connector.
5 - Clone the current crate::Context.
6 - Push the crate::Context into the output queue.
7 - Go to step 1 until the input queue is not empty.
§Configuration
| key | alias | Description | Default Value | Possible Values |
|---|---|---|---|---|
| type | - | Required in order to use writer step. | writer | writer / write / w |
| connector_tyoe | conn / connector | Connector type to use in order to read a resource. | io | See crate::connector |
| document_tyoe | doc / document | Document type to use in order to manipulate the resource. | json | See crate::document |
| name | alias | Name step. | null | Auto generate alphanumeric value |
| data_type | data | Data type read for writing. skip other data type. | ok | ok / err |
| concurrency_limit | - | Limit of steps to run in concurrence. | 1 | unsigned number |
| dataset_limit | batch | Stack size limit before to push data into the resource though the connector. | 1000 | unsigned number |
§Examples
[
...
{
"type": "writer",
"name": "write_a",
"connector": {
"type": "io"
},
"document": {
"type": "json"
},
"data": "ok",
"concurrency_limit": 1,
"dataset_limit": 1000
},
{
"type": "writer",
"name": "write_b",
"connector": {
"type": "local",
"path": "./data/my_data.{{ metadata.mime_subtype }}"
},
"document": {
"type": "json"
}
}
...
]