ParaGraphs - Parallel Graph Execution Library

Under Construction

Understanding ParaGraphs

The ThreadPool

The ThreadPool manages one or more Workers. Each Worker is responsible for managing a single thread. Nodes, the units of execution, are moved into Worker threads over an mpsc::Channel, and returned over a second channel when execution completes. It is the responsibility of the Graph to move the Nodes back out of the channel.

The ThreadPool receives Jobs, which are comprised of a Node, inputs (Vec<Arc<Data>>), and a unique identifier (usize). The ThreadPool then dispatches each job to one of its workers by sending it via an mpsc::Sender. The contents of the Job are moved, so it is important that Node is Send and Data is Send + Sync. The inputs to the Node are captured in a vector of atomically reference counted pointers (Arcs), so that multiple output nodes can access the same values without the need for Clone-ing and Sending.

Each Worker listens for Jobs on the corresponding mpsc::Receiver. Upon receiving a new Job, it immediately begins execution. Upon completion, the Worker sends a message over its own mpsc::Sender with the Node, its output (Data) as well the unique identifier (usize) for the Job.

Graph

Under Construction