shepherd-rs 0.2.0

Shepherd is a resilient, non-blocking orchestrator that persistently transforms and delivers data—built for remote, compute-heavy workloads.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

//! # Shepherd Framework
//!
//! This crate provides a general framework for writing schedulers that
//! transform input data into output data by invoking workers.
//!
//! ## Features
//! - **Flexible Worker Integration**: Workers can be implemented as separate
//!   processes, remote services, or via Apache Kafka.
//! - **Data Transformation**: Input data is transformed into output data using
//!   configurable pipelines.
//! - **Output Handling**: Output data can be sent to remote services, written
//!   to files, or stored in databases.
//!
//! ## Modules
//! - `config`: Handles composite configurations for the shepherd.
//! - `consumer`: Manages consumption attempts and results.
//! - `database`: Provides database abstractions for storing transformation
//!   data.
//! - `emitter`: Handles emission of transformation requests.
//! - `instance`: Represents an shepherd instance.
//! - `instrumentation`: Provides instrumentation for monitoring and debugging.
//! - `processor`: Processes transformation requests and manages attempts.
//! - `test`: Contains testing utilities and mock implementations.
//! - `transform`: Defines transformation requests and attempts.
//! - `worker`: Manages worker instances and their interactions.

#![feature(associated_type_defaults)]

pub mod config;
pub mod consumer;
pub mod database;
pub mod emitter;
pub mod instance;
#[cfg(feature = "instrumentation")]
pub mod instrumentation;
pub mod processor;
pub mod transform;
pub mod worker;