Struct flowrlib::run_state::RunState

pub struct RunState { /* private fields */ }

RunState is a structure that maintains the state of all the functions in the currently executing flow.

A function maybe blocking multiple others trying to send data to it. Those others maybe blocked trying to send to multiple different function.

The Semantics of a Flow’s RunState

The semantics of the state of each function in a flow and the flow over are described here and the tests of the struct attempt to reproduce and confirm as many of them as is possible

Terminology

• function - an entry in the manifest and the flow graph that may take inputs, will execute an implementation on a Job and may produce an Output
• input - a function may have 0 or more inputs that accept values required for it’s execution
• implementation - the code that is run, accepting 0 or more input values performing some calculations and possibly producing an output value. One implementation can be used by multiple functions in a flow
• destinations - a set of other functions and their specific inputs that a function is connected to and hence where the output value is sent when execution is completed
• job - a job is the bundle of information necessary to execute. It consists of the function’s id, the input values, the implementation to run, and the destinations to send the output value to
• execution - the act of running an implementation on the input values to produce an output
• output - a function when ran produces an output. The output contains the id of the function that was ran, the input values (for debugging), the result (optional value plus an indicator if the function wishes to be ran again when ready), the destinations to send any value to and an optional error string.

Start-up

At start-up all functions are initialized. For each of the functions inputs their init_inputs() function will be called, meaning that some inputs may be initialized (filled):

• Other functions that send to these initialized inputs will be blocked initially If all inputs are full then the Function maybe able to run, depending on existence of blocks on other functions it sends to.

One-time Execution or Stopping repetitive execution

A function may need to only run once, or to stop being executed repeatedly at some point. So each implementation when ran returns a “run again” flag to indicate this. An example of functions that may decide to stop running are:

• args: produces arguments from the command line execution of a flow once at start-up
• readline: read a line of input from standard input, until End-of-file (EOF) is detected. If this was not done, then the flow would never stop running as the readline function would always be re-run and waiting for more input, but none would ever be received after EOF.

Unused Functions

If a pure function has an output but it is not used (not connected to any input) then the function should have no affect on the execution of the flow and the optimizer may remove it and all connections to its input. That in turn may affect other functions which can be removed, until there are no more left to remove. Thus at run-time, a pure function with it’s output unused is not expected and no special handling of that case is taken. If a manifest is read where a pure function has no destinations, then it will be run (when it received inputs) and it’s output discarded. That is sub-optimal execution but no errors should result. Hence the role of the optimizer at compile time. Tests: pure_function_no_destinations()

Unconnected inputs

If a function’s output is used but one or more of it’s inputs is unconnected, then the compiler should throw an error. If for some reason an alternative compiler did not reject this and generated a manifest with no other function sending to that input, then at run-time that functions inputs will never be full and the function will never run. This could produce some form of deadlock or incomplete execution, but it should not produce any run-time error.

A run-time is within it’s rights to discard this function, and then potentially other functions connected to it’s output and other inputs until no more can be removed. This run-time does not do that, in order to keep things simple and start-up time to a minimum. It relies on the compiler having done that previously.

Initializers

There are two types of initializers on inputs:

• “Once” - the input is filled with the specified value once at start-up.
• “Constant” - after the functions runs the input refilled with the same value.

Runtime Rules

• A function won’t be run until all of its inputs are ready and all of the inputs on other functions it sends to are empty and able to receive its output. (See ‘Loops’ below)
• An input maybe initialized at start-up once by a “Once” input initializer
• An input maybe initialized after each run by a “Constant” input initializer that ensures that the same value always re-fills the input
• When one or more inputs on other functions that a function sends to are full, then the sending function will be in the blocked state

State Transitions

From To State Event causing transition and additional conditions Test

---

Init Ready No inputs and no destination input full to_ready_1_on_init All inputs initialized and no destination input full to_ready_2_on_init All inputs initialized and no destinations to_ready_3_on_init Init Blocked Some destination input is full to_blocked_on_init Init Waiting At least one input is not full to_waiting_on_init

Ready Running NextJob: called to fetch the function_id for execution ready_to_running_on_next

Blocked Ready Output: function that was blocking another completes blocked_to_ready_on_done

Waiting Ready Output: last empty input on a function is filled waiting_to_ready_on_input Waiting Blocked Output: last empty input on a function is filled, blocked waiting_to_blocked_on_input

Running Ready Output: it’s inputs are all full, so it can run again running_to_ready_on_done Running Waiting Output: it has one input or more empty, to it can’t run running_to_waiting_on_done Running Blocked Output: a destination input is full, so can’t run running_to_blocked_on_done

Iteration and Recursion

A function may send values to itself using a loop-back connector, in order to perform something similar to iteration or recursion, in procedural programming. A function sending a value to itself will not create any blocks, and will not be marked as blocked due to the loop, and thus avoid deadlocks.

Blocks on other senders due to Always Initializers and Loops

After a function runs, its Always Initializers are used to refill inputs, and outputs (possibly to itself) are sent, before determining that other functions sending to it should unblocked. The initializers and loops to it’s inputs have priority and the input(s) will be refilled but another function wishing to send to it, and blocked, is NOT yet unblocked. TODO TEST

Parallel Execution of Jobs

Multiple functions (jobs) may execute in parallel, providing there is no data dependency preventing it. Example dependencies:

• a function lacks an input and needs to get it from another function that has not completed
• a function cannot run, as it’s output iss connected to another function’s input that is full Respecting this rule, a RunTime can dispatch as many Jobs in parallel as it desires. This one takes the parameter max_jobs on RunState::new() to specify the maximum number of jobs that are launched in parallel. The minimum value for this is 1

Implementations

impl RunState

pub fn new(submission: Submission) -> Self

Create a new RunState struct from the list of functions provided and the Submission that was sent to be executed

pub fn get_function_states(&self, function_id: usize) -> Vec<State>

Return the states a function is in

pub fn get_blocked(&self) -> &HashSet<usize>

Get the list of blocked function ids

pub fn get_running(&self) -> &HashMap<usize, Job>

Get a Set (job_id) of the currently running jobs

pub fn get_completed(&self) -> &HashSet<usize>

Get the list of completed function ids

pub fn get_function(&self, id: usize) -> Option<&RuntimeFunction>

Get a reference to the function with id

pub fn get_blocks(&self) -> &HashSet<Block>

Get the HashSet of blocked function ids

pub fn get_busy_flows(&self) -> &MultiMap<usize, usize>

Return the list of busy flows and what functions in each flow are busy

pub fn get_flow_blocks(&self) -> &HashMap<usize, HashSet<usize>>

Return the list of pending unblocks

pub fn get_number_of_jobs_created(&self) -> usize

get the number of jobs created to date in the flow’s execution

pub fn get_output_blockers(&self, id: usize) -> Vec<usize>

Get the set of (blocking_function_id, function’s IO number causing the block) of blockers for a specific function of id

pub fn number_jobs_running(&self) -> usize

Return how many jobs are currently running

pub fn number_jobs_ready(&self) -> usize

Return how many jobs are ready to be run, but not running yet

pub fn get_input_blockers(&self, target_id: usize) -> Result<Vec<usize>>

An input blocker is another function that is the only function connected to an empty input of target function, and which is not ready to run, hence target function cannot run.

pub fn num_functions(&self) -> usize

Return how many functions exist in this flow being executed

Trait Implementations

impl Clone for RunState

fn clone(&self) -> RunState

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'de> Deserialize<'de> for RunState

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for RunState

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Serialize for RunState

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>where __S: Serializer,

Serialize this value into the given Serde serializer.