Crate rayon_logs

This crate provides logging facilities to evaluate performances of code parallelized with the rayon parallel computing library. It also doubles down as a debugging tool.

Ideally using it should be as easy as adding extern crate rayon_logs as rayon; at top of your main file (replacing extern crate rayon).

However there are currently limitations because we do not currently log all parts of rayon.

• the global ThreadPool is not logged so it is required to use a ThreadPoolBuilder.
• not all of rayon’s traits are implemented. In particular no IndexedParallelIterator (no zip), no FromParallelIterator (no collect)…
• par_sort is logged but it is not directly rayon’s par_sort but a copy-pasted version of it (as a demonstration). so the algorithm is hard-coded into rayon_logs.
• you should not mix logged and not logged computations.
• each call to ThreadPool::install generates a json file which can then be converted to svg using json2svg.
• each log generates an overhead of around 1 micro seconds. This is due to thread_local being very slow.

With this being said, here is a small example:

Example:

extern crate rayon_logs as rayon; // comment me out to go back to using rayon
use rayon::prelude::*;
use rayon::ThreadPoolBuilder;
let v = vec![1; 100_000];
// let's create a logged pool of threads
let pool = ThreadPoolBuilder::new().num_threads(2).build().expect("failed creating pool");
// run and log some computations
assert_eq!(100_000, pool.install(|| v.par_iter().sum::<u32>()));

Running this code will create a log_0.json file. You can then use cargo run --bin json2svg -- log_0.json example_sum.svg to view the log. The resulting file should be viewed in a web browser since it is animated. The bars below the graph represent idle times.

Modules

prelude

We redefine (not all of them yet) rayon traits.

Structs

Comparator

The comparator structure enables you to easily compare performances of different algorithms.

Logged

Logged is an iterator that logs all tasks created.

RunLog

Logged information.

Scope

Represents a fork-join scope which can be used to spawn any number of tasks. See scope() for more information.

ThreadPool

We wrap rayon’s pool into our own struct to overload the install method.

ThreadPoolBuilder

We rewrite ThreadPoolBuilders since we need to overload the start handler in order to give each thread a place to write its logs.

Functions

current_num_threads

We re-export rayon’s current_num_threads. Returns the number of threads in the current registry. If this code is executing within a Rayon thread-pool, then this will be the number of threads for the thread-pool of the current thread. Otherwise, it will be the number of threads for the global thread-pool.

current_thread_index

If called from a Rayon worker thread, returns the index of that thread within its current pool; if not called from a Rayon thread, returns None.

custom_subgraph

Tag a subgraph with a custom value. The start function will be called just before running the graph and produce an S. The end function will be called just after running the graph on this S and produce a usize which will the be stored for display.

end_subgraph

Stop current task (virtually) and end a subgraph. You most likely don’t need to call this function directly but subgraph instead.

join

Takes two closures and potentially runs them in parallel. It returns a pair of the results from those closures.

join_context

Identical to join, except that the closures have a parameter that provides context for the way the closure has been called, especially indicating whether they’re executing on a different thread than where join_context was called. This will occur if the second job is stolen by a different thread, or if join_context was called from outside the thread pool to begin with.

scope

Create a “fork-join” scope s and invokes the closure with a reference to s. This closure can then spawn asynchronous tasks into s. Those tasks may run asynchronously with respect to the closure; they may themselves spawn additional tasks into s. When the closure returns, it will block until all tasks that have been spawned into s complete.

start_subgraph

Stop current task (virtually) and start a subgraph. You most likely don’t need to call this function directly but subgraph instead.

subgraph

We tag all the tasks that op makes as one subgraph.

visualisation

Computes a graphical view of a log. This is intended for the development of logs viewers.