Rusty Chain

This library abstracts over functional processing units called ChainLinks and Chains. Each link in the chain is meant to be independent, immutable, idempotent, parallelizable, and highly testable.

Features

• A ChainLink is an independent processing unit that receives an input and sends an output.
  • By using the chain_link! macro you can quickly construct the internals of the mapping from input to output.
• A Chain is a concatenation of ChainLinks (and other Chains) and is a natural extension of this methodology for processing.
  • By using the chain! macro you can concatenate both ChainLinks and Chains naturally.
• A split_merge! macro permits parallel processing multiple ChainLink implementations, round-robin iterating over them per send.
  • If a ChainLink returns None, it will try the next one, etc.

Usage

You will want to determine what the smallest unit of processing your project consists of so that you can begin to create ChainLinks. Defend the quality of your ChainLinks by creating rigorous unit tests. After you have created a few ChainLinks bring it all together with a Chain.

Each type of processing unit (ChainLink and Chain) accept in an optional initializer, allowing for dependency injection. Now, it is possible to share dependencies between ChainLinks of a Chain, but that is highly discouraged without unit tests around the Chain.

Examples

Mapper

This example demonstrates how a ChainLink may exist to pull records from a database and map them to a model.

ETL

This example demonstrates how a file-loaded ETL process could be separated out into three ChainLinks, all connected together as a Chain, allowing you to pass in file paths and get back at the end if the current line processed was successful. This example also covers basic usage of the nom crate and how the initializer can be used as a mutable buffer.

Inspiration

I have always wanted highly testable code and to work in an environment where the logic of my processes was absolutely dependable.

Future work

• split_merge! conditions
  • This would allow the send from one ChainLink to make its way to different destination ChainLinks based on a conditional block per destination, allowing logical, asynchronous splitting of processing.