Skip to main content

Module stream_executor

Module stream_executor 

Source
Expand description

Contains the logic for executing Stream pipelines on their own Tokio tasks with zero-cost Instruments options for gathering stats, logging and so on.
Four executors are provided to attend to different Stream output item types:

  1. Items that are non-futures & non-fallible. For instance, Stream::Item = String
  2. Items that are futures, but are non-fallible: Stream::Item = Future<Output=DataType> – DataType may be, for instance, String
  3. Items that are non-futures, but are fallible: Stream::Item = Result<DataType, Box<dyn std::error::Error>>
  4. Items that are fallible futures: Stream::Item = Future<Output=Result<DataType, Box<dyn std::error::Error>>> – allowing futures to time out

Apart from handling the specific return types, logging & filling in the available metrics, all executors do the same things:

  1. Register start & finish metrics
  2. Log start (info), items (trace) and finish (warn)

Specific executors do additional work:

  1. Streams that returns a Result: logs & counts untreated-errors (errors that made it all the way through the pipeline and ended up in the executor), calling the provided on_err_callback();
  2. Streams that returns a Future: they will, optionally, compute the time for each future to complete and, also optionally, register a time out for each future to be completed – cancelling the Future if it exceeds the time budget.

Structs§

AtomicExecutorStatus
A wrapper around ExecutorStatus which can be safely shared between threads.
StreamExecutor
See Instruments

Enums§

ExecutorStatus
will derive AtomicExecutorStatus

Traits§

StreamExecutorStats
The trait to be passed along after the executor has been started