shepherd-rs 0.2.0

Shepherd is a resilient, non-blocking orchestrator that persistently transforms and delivers data—built for remote, compute-heavy workloads.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

![shepherd-rs logo](shepherd-rs-banner.png)

# Shepherd Framework

An scheduler framework that reliably undertakes data transformation and submission of results with perseverance.

## Overview
The scheduler framework is designed to:
- **Retry Mechanism**: Ensure reliable transformation and submission of data by retrying tasks until completion.
- **Non-blocking Architecture**: Handle compute-heavy tasks efficiently, especially when data transforming units (DTUs) are hosted remotely (e.g., zkVM provers).

## Features
- **Flexible Integration**: Supports various transformation units and consumer endpoints.
- **Robustness**: Guarantees task completion through persistent retries.
- **Scalability**: Optimized for compute-heavy workloads.

## Requirements
Ensure you have [rustup](https://rustup.rs/) installed. Compile the system using the Rust nightly channel, as it utilizes nightly-only features such as [associated_type_defaults](https://github.com/rust-lang/rust/issues/29661).

## Testing
PostgreSQL Database test requires docker to be installed on system & may pose intermittent failures if ran parallely. Use:
```sh
cargo test database::postgresql -- --test-threads=1
```

## Documentation
Generate and view the documentation:
```sh
cargo doc --open
```

## License
This project is licensed under the terms of the [MIT License](LICENSE).