# Apis v0.0.1 - initial implementation {#v0.0.1}
**Issues**
- [*Process results*](#process-results) (resolved in [v0.0.11](#v0.0.11)):
Session results are a uniform `Option<()>` type for all sessions. There is
syntax defined in the `def_session!` macro to give each process a unique
result type. These could probably be wrapped in a global `Result` type for
the entire session (so they can still be stored in a `VecMap`).
- *Custom initialization* (resolved in [v0.0.11](#v0.0.11)): processes might
require special initialization to be performed but currently the only place
to do that is in a continuation block.
- *Asynchronous polling* (resolved in [v0.0.11](#v0.0.11)): The case of a
process which blocks on some external code awaiting a result as the main form
of input is supported by creating a synchronous process with a tick length of
`0` ms. There doesn't seem to be anything especially wrong with this
solution, but some modification needs to be made to prevent "late tick"
warnings from being generated (a tick length of `0` means that the next tick
deadline is seen as being instantaneous with the last). As long as the user
code reliably blocks and the process is defined to have one update per tick,
the process will not be able to "catch up" by processing very many ticks.
- [*Reusable processes*](#reusable-processes): It is not possible to re-use
process definitions. Therefore, if two sessions use a process that is
identical, the code must be duplicated. This can be eased somewhat by giving
both processes a single "inner" type that handles the logic uniformly for
both, so that they are essentially thin wrappers around this type.
- *Program coherence*: There are some coherence requirements for programs that
are checked by debug assertions or at runtime. A better solution would be
along the lines of what is done with sessions where a definition is built and
verified at initialization time. These requirements are that processes to be
mapped as continuations are mapped one-to-one, and that main processes can be
mapped to eachother but not to other processes. Other checks may be done on
the graph of transitions to ensure that all modes are reachable.
- *State machine usage*: State machines are used at a few points in the
implementation; mostly they are "simple" transitions that lack actions,
guards, or parameters. More could be done there, but it's unlikely to be
possible to capture *all* affected program logic within events.
- *Session typed program control channels*: To allow for continuations,
processes need to be able to return results to the session and to receive
continuations from the session. Currently this is done with standard
`std::sync::mpsc` channels, but it might be possible to replace these with
session-typed channels, adding some safety guarantees to communication on
these channels.
The following are *implementation issues* that don't as much affect users:
- *Session dotfile functions*: The methods for producing session dotfiles rely
on macro expansion. It would be better for maintainability and modifiability
if this could be re-implemented in a purely functional manner, since session
definition information is already available in the form of `session::Def`.
**Further development**
The next phase of development is to implement an example of a graphical,
interactive application in `apis`.
`glutin`, `glium`
## Process results
☞ This has been resolved in [Apis v0.0.11](#v0.0.11)
Currently processes uniformly return an `Option<()>` type, and these are
returned by the `Session::run_*` methods in a `VecMap` which is used to
determine the next transition in a program. This is essentially equal to
each process returning a boolean, which gives only a limited amount of
information to choose transitions. The options for extending this are:
- each process has its own result type, and these are convertible with a
"global" result type, which can be added to the context, so that a session
then produces a `VecMap` of `CTX::GRES`
- a session defines only a single result type and all processes return this
type
The first option seems more flexible. Often the result of a single process will
"decide" what transition to take. In a way this breaks the strict separation
between different session definitions, but the actual logic for choosing a
transition is still limited to the program definition: sessions simply return
the result and don't take any further action themselves.
One issue with this is that process result types need to be unique since there
needs to be a constraint that maps result types to global results. If two
processes returned the same base type, such as `Option <()>`, a unique mapping
would not exist to the global process type since it could be a result for
either of the processes.
This seems to work however the unwrapping of the value is somewhat cumbersome.
To get the inner result from a process requires:
1. Getting the global presult for the corresponding process ID
2. Use the `try_from` method on the presult newtype to unwrap the global
presult to the local presult
3. Destructure the local presult in a `match` statement to get the inner result
A better solution may be to make the global presult type a simple enum where
each variant is named after the corresponding process and contains the inner
process result type directly.
Returning the result can be made more automatic as well by adding a generic
parameter to the process trait that can be accessed by getter/setter methods
`result_ref`, `result_mut`. This type might define a default and possibly allow
overloading by the process definition. Note that we don't want to add this
result to the `Inner` type because this would not allow them to be constructed
generically for all processes; the alternative would be to move the `Inner`
type construction inside required `process::Id` methods `spawn` and `gproc`
which are defined inside the macro where the result type information is
available. The rest of the inner type fields would then become arguments to
those functions, and the inner type itself would need to be parameterized by
the result type, so it is easier in the end to just go the route of adding the
above mentioned result reference functions to the process trait.
To create a global process result from the local result type, another required
method is added to the `Process` trait: `global_result`. At first an attempt
was made to add a generic conversion method to the global presult trait itself,
but this won't work because the inner presult type is a generic parameter and
to construct the global presult requires a concrete type.
## Reusable processes
A difference between a trait with a generic parameter and a trait with an
associated type is that the trait with the generic parameter can be implemented
multiple times for the same type with different type parameters. Therefore a
process could in theory implement the `Process <Context>` trait for multiple
contexts. This is complicated somewhat by the fact that process `Id` types
define the mapping to process definitions.
A concrete process is really just a type that holds an `Inner` process state
machine and defined behavior methods (`update` and `handle_message`), and is
free to have additional data fields. If a process were to be made re-usable, it
should probably be parameterized by the `Context` type as well, so that it can
hold an `Inner <Context>` for any kind of context. However for each instance, a
different `update` and `handle_message` method must be defined. Since channels
and peers are referenced by channel IDs and process IDs, respectively, it
doesn't seem like these methods could be re-used safely where sessions are
allowed have different sets of processes. One way to get around this would be
to give channel and process IDs as strings and have them converted to the
appropriate ID enum type. As long as the session has a channel or process ID
that resolves from the given string, the process could be used there.
# Apis v0.0.11 - prototype in use {#v0.0.11}
A number of example programs now exist in the `./examples/` directory and the
library has developed to suit the needs of an interactive graphical
application.
**Issues**
The following issues from v0.0.1 have been *resolved*:
- *Process results*: Processes can optionally specify a process result or
"presult" type. Each session context defines a "global presult type"
(`Context::GPRES`) which is collected into a `VecMap` of results for each
individual process and returned by the session `run` function. In a program
these results are accessible within the "transition choice" block for each
mode (session). The `Process` trait adds the required methods (automatically
implemented in the `def_session!` macro) `result_ref`, `result_mut` for
getting and setting the internal return value and `global_result` for
wrapping the internal return type as a global presult type. As a convenience
another required method `extract_result` is added to the `Process` interface
(and automatically implemented by the `def_session!` macro) that attempts to
unwrap a given global presult into the local result type for that process.
- *Custom initialization*: Processes can optionally provide `initialize` and
`terminate` actions to perform just before running (in the `Ready` state) and
just after running (in the `Ended` state), respectively.
- *Asynchronous polling*: This process kind was added which has the message
polling behavior of synchronous processes with the ungoverned loop of
asynchronous processes. This is mostly intended for situations where the
`update` function will block on some external mechanism (e.g. waiting on
input events from system or vsync in a frameloop). Process kinds are defined
in terms of two aspects:
* *Synchronous* vs. *Asynchronous*: With and without *loop timing*; the
former will not poll or update more than a specified number of ticks per
second, and will "catch up" if a tick takes longer than the alotted time,
while the latter will always loop immediately.
* *Polling* vs. *Blocking*: How message are *received*; the former will
poll each endpoint to exhaustion with `try_recv`, while the latter is
only allowed a single endpoint on which it blocks waiting for messages
with `recv`.
`Kind::Synchronous` processes only allow polling for messages and
`Kind::Asynchronous` processes are always blocking, while the new process
kind `Kind::AsynchronousPolling` is always polling. Keep in mind that in the
*other* sense of "asynchrony", all *sends* from processes are *asynchronous*
in that they do not wait for the receiver to retrieve the message. It was
debated whether to adopt "isochronous" and "anisochronous" to describe
looping behavior to avoid confusion, but for now it will be left as
"synchronous" and "asynchronous".
The following issues *remain* from v0.0.1:
- [*Reusable processes*](#reusable-processes): It is not possible to re-use
process definitions. Therefore, if two sessions use a process that is
identical, the code must be duplicated. This can be eased somewhat by giving
both processes a single "inner" type that handles the logic uniformly for
both, so that they are essentially thin wrappers around this type.
- *Program coherence*: There are some coherence requirements for programs that
are checked by debug assertions or at runtime. A better solution would be
along the lines of what is done with sessions where a definition is built and
verified at initialization time. These requirements are that processes to be
mapped as continuations are mapped one-to-one, and that main processes can be
mapped to eachother but not to other processes. Other checks may be done on
the graph of transitions to ensure that all modes are reachable.
- *State machine usage*: State machines are used at a few points in the
implementation; mostly they are "simple" transitions that lack actions,
guards, or parameters. More could be done there, but it's unlikely to be
possible to capture *all* affected program logic within events.
- *Session typed program control channels*: To allow for continuations,
processes need to be able to return results to the session and to receive
continuations from the session. Currently this is done with standard
`std::sync::mpsc` channels, but it might be possible to replace these with
session-typed channels, adding some safety guarantees to communication on
these channels.
The following implementation issues *remain* from v0.0.1:
- *Session dotfile functions*: The methods for producing session dotfiles rely
on macro expansion. It would be better for maintainability and modifiability
if this could be re-implemented in a purely functional manner, since session
definition information is already available in the form of `session::Def`.
The following issues are *added* to v0.0.11:
- [Control flow mechanisms](#control-flow-mechanisms): Currently process
control flow is controlled by returning `process::ControlFlow::Break` from
either `handle_message` or `update` methods of the process; this can affect
different results depending on the kind of the process. The goal is not to
enforce policy on how processes are synchronized, but that the mechanisms
available make it possible to feasibly implement a policy that doesn't result
in orphan messages or send failures.
## Control flow mechanisms
We want control flow mechanisms that make it feasible to implement
synchronization policies that don't result in *orphan messages* or *send
failures*. Whether or not *receive failures* (disconnections) can or should be
avoided is an open question. (As noted below, receive failures are less likely
to happen and easier to avoid as long as a terminating "finish" or "stop"
message is always the last one sent on a channel before the sender drops: the
actual receiving end is *not* disconnected until the last message has been
received.)
An *orphan message* is a message that was sent but never received
(*automatically*) within the process run loop. Each process can run a
`terminate` action after the loop has finished where further messages can be
received, but this requires that any remaining messages are received *manually*
by the user. After this point, before destroying the channels they are checked
for orphan messages (which will generate warnings).
A *send failure* is an attempt to send on a channel for which the receiver has
disconnected. The message will never be deliverable, and is returned along with
the send error so it can be recovered. Because all sends are *manual*, it is up
to the user to respond appropriately to a send failure.
*Receive failures* are a little more complicated due to there being two kinds
of reception:
- For a blocking receive (`Kind::Asynchronous` processes) there is only one
endpoint, and due to the in-order reception of messages, once a "final"
message has been received the channel should never contain any further
messages, and once the last message is received (if the sender has
disconnected) the next attempt will produce a disconnection error. If the
process is *expecting* to be notified of a "final" message, then seeing a
disconnected channel truly is a sign of an error since the only alternative
would be for the asynchronous process in question to be the one intended to
signal a "finish" event (either directly or indirectly through other outgoing
channels) which can never be received since that other peer has definitely
dropped.
Receives are also complicated by *sink* channels. An asynchronous endpoint
of a sink channel can receive from multiple other peers (currently it is
not automatically indicated in the message which specific other peer it
originates from, only that it came from *some* peer sender to that channel
is known), and will only be able to tell when *all* the sending peers have
dropped by seeing a disconnected endpoint. It is also more difficult to
guarantee that a "finish" message received on a sink channel is going to be
the last message on the channel: without some other synchronization between
the sending peers themselves, one of them may independently send further
messages. A more reasonable solution for synchronizing "finish" conditions
on a sink channel would probably be to require a separate "finish" message
from each peer connected to the channel (a kind of policy left to users to
implement).
- For polling receive (`Kind::Synchronous` and `Kind::AsynchronousPolling`
processes), the situation for processes holding single endpoints is similar
to asynchronous processes, and those holding multiple endpoints is also
similar to a single-endpoint sink channel (see above). That is, if a polling
process has a single input channel, that channel should never be seen as
disconnected by the receiver before a "final" event is received on that
channel (or sent via a backchannel if the direction of control is reversed).
If a polling process has more than one input channel, it is possible that
there is some delay between reception of each separate "finish" message, so
the process may need to continue polling disconnected channels several times
before the last channel is "closed" (after which the process in question can
be assured that no further messages will be received).
It should be noted that for a receiver, the channel is not disconnected until
all messages have been received. Ensuring a "stop" message is the last message
sent on such a channel ensures that receiving from a disconnected channel never
happens (since such a message will cause a break in the control flow, see
below).
The run loops for each process kind are roughly as follows:
*Synchronous* (Polling):
while running {
if time for tick {
'poll_outer: for each endpoint {
'poll_inner: loop match endpoint.try_recv() {
Ok(message) => if self.handle_message(message) == break {
running = false; break 'poll_inner
}
Err(Empty) => break 'poll_inner
Err(Disconnected) => {
running = false; break 'poll_inner
}
}
}
if tick count == ticks per update {
if self.update() == break {
running = false
}
}
} else not time for tick {
warn early tick
}
if not time for next tick {
sleep until next tick
} else time for next tick {
warn late tick
}
}
*Asynchronous* (Blocking):
while running {
match endpoint.recv() {
Ok(message) => if self.handle_message(message) == break {
running = false
}
Err(Disconnected) => {
running = false
}
}
if message count == messages per update {
if self.update() == break {
running = false
}
}
}
*AsynchronousPolling*:
while running {
'poll_outer: for each endpoint {
'poll_inner: loop match endpoint.try_recv() {
Ok(message) => if self.handle_message(message) == break {
running = false; break 'poll_inner
}
Err(Empty) => break 'poll_inner
Err(Disconnected) => {
running = false; break 'poll_inner
}
}
}
if self.update() == break {
running = false
}
}
A motivating example is the following session with three processes, all of
which are polling:
A * ----> * C
| ^
| |
+-> * --+
B
There are two ways for process `A` to signal "stop" to peers `B` and `C`:
1. Process `A` sends individual "stop" messages directly to each of `B` and
`C`. This introduces two hazards depending on the order that `B` and `C` are
dropped:
i. If `B` drops before `C`, `C` may be currently trying to *receive* on
the disconnected channel `B->C`
ii. If `C` drops before `B`, `B` may be trying to *send* messages on the
disconnected channel `B->C`
2. Process `A` sends an initial "stop" message to process `B` which then
forwards a "stop" message to process `C`. In this case the hazard is that if
`A` drops after sending the initial "stop" message, process `C` will
continue trying to poll the disconnected channel `A->C` until the "stop"
message is received from process `B`, even if (actually, precisely when)
there are no pending messages from `A` and no further messages were sent on
`A->C` after the initial "stop".
If (1.) and (2.) are combined, a third possible scheme is that process `A`
sends "stop" messages on each of its channels, and process `B` sends another
"stop" message on its channel to `C`. This only eliminates hazard (1i.) above--
`B` will always send the "stop" message before dropping so `C` will not see a
disconnected channel until it is emptied.
It may be useful to back up and explore some other primitive configurations. We
will denote a channel from process `A` to process `B` as `AB`, sending a
message as `AB^x` and receiving a message as`AB_x` where `q` denotes a "finish"
or "stop" message after which no further messages will be sent on that channel,
and dropping a process `X` is denoted `drop(X)`.
1. `A -> B` -- The only possible finite message trace for this session is:
[ ..., AB^q, ..., AB_q ]
where the first `...` contains an ordered interleaving of sends and
corresponding receives, the second `...` can contain receives for sends
prior to the "stop" message but not yet received, and `drop(A)`, `drop(B)`
may appear freely after `AB^q` or `AB_q`, respectively.
2. `A <=> B` -- (one channel in each direction) Here `A` must stay alive so that
`B` can continue to send messages until the `AB^q` send is received (`AB_q`):
[ ..., AB^q, ..., AB_q, ..., BA^q, ..., BA_q ]
and `drop(A)` may appear freely after `BA_q` and `drop(B)` may appear
freely after `BA^q` (note here that unlike the example (1.) of a single
channel, process `B` may drop before process `A` in this case with
bidirectional messages). The directionality here is essentially symmetric,
the role of either `A` or `B` is exchangeable and it is even possible that
either process may initiate the "quit" sequence, even simultaneously,
without a problem.
3. `A -> B + C` -- (channels `AB` and `AC`) Here `A` is a source sending to
both `B` and `C`. Each channel is an isolated case of (1.) above.
4. `A + B -> C` -- (channels `AC` and `BC`) There is no way in this situation
to initiate a termination of the session from a single point. Either `A` and
`B` must decide independently at some point to signal completion to `C`, or
else there must be another channel involved to synchronize this action.
5. `A -> B -> C` -- (channels `AB` and `BC`) This is identical to two instances
of single channels (1.) above.
6. `A -> B -> C -> A` -- (channels `AB`, `BC`, `CA`) This defines a cyclic
topology which is like (5.) combined with (2.) in that `A` must remain alive
until a "stop" message is received from `C`:
[ ..., AB^q, ..., AB_q, ..., BC^q, ..., BC_q, ..., CA^q, ..., CA_q ]
7. `A -> (B -> C)` -- (channels `AB`, `AC`, and `BC`) This is the "motivating
example" given above. The key feature is that the problem of (4.) is
encountered again: a receiver (`C`) with two independent senders (`A` and
`B`) requires some kind of coordination between the senders. Here however
there is a channel `AB` connecting them; `A` can initiate "stoppage" by
sending `AB^q` and `AC^q` in any order. Reception of `AC_q` does not cause
`C` to terminate however, since `B` may not have processed `AB_q` yet and
can continue sending messages on `BC` until `BC^q`. Since `A` is not
connected by any backchannels to `B` or `C`, it cannot be constrained to
stay alive, and so may drop any time after both `AB^q` and `AC^q` are sent.
This exposes the key issue with the current message handling semantics:
- *Process `C` will continue to poll `AC` after `AC_q` (until `BC_q`),
possibly after `A` has dropped and `AC` is disconnected.* It is also
possible that `BC_q` is received before `AC_q`, in which case `BC` will
be polled one final time (possibly disconnected) in a normal round of
polling all endpoints.
This case (7.) as a motivating example is not an unreasonable one: it
represents a simple directed acyclic graph of data flow.
<U>The bottom line is that *if* the current scheme of polling *all* endpoints
on *every* round of polling, it *is* possible (and even likely) in this
situation that a disconnected channel will be polled.</U>
☞ Note that the situation changes if `AC` and `BC` are actually a single
*sink* channel `(A+B)->C`, since either endpoint and any pending messages will
keep the single channel alive. Here we can implicitly assume that if `A` is the
"initiator" and `BC_q` is received, then no further messages will be received
since `A` initiated the "stoppage", no messages can appear on `(A+B)->C` after
`BC_q`.
**Ideas**
- One idea for handling this situation is to allow channels to marked as
"finished" by a special message or an additional control flow type.
* "Stop" messages
* "Stop" control flow
this is somewhat more complicated than the current "break/continue"
semantics, an easier solution might be possible:
**Working solution**
Since current simple examples already work without issue, we don't want to
change the current scheme too much. A minimal change would be to consider
"breaking" coming from a `handle_message` call only marks *that channel* as
"done" ("finished", "stopped"); the process will not transition to `End` unless
it is the *last channel* to do so. Processes may still break unconditionally in
the `update` action. A process with multiple endpoints (polling, by definition)
would only issue a warning if `try_recv` sees a disconnected channel before
seeing "stop" on that channel, and seeing a disconnection after "stop" would no
longer be considered a warning.
This also means that `Asynchronous` processes can remain unchanged.
## Renaming of process kinds
Arrived at above is the naming scheme for process kinds:
polling blocking
===================-================
timed Synchronous
untimed AsynchronousPolling Asynchronous
We want to add a third type of polling process that is "rate limiting" in that
it will loop immediately if enough time passed since the last polling operation
started, otherwise it sleeps until the next polling time. Compared to the
`Synchronous` process kind above, there is no "catching up". The naming scheme
may be modified as follows to include the new process kind:
polling blocking
===================-================
timed-absolute Isochronous
timed-rate limited Mesochronous
untimed Anisochronous Asynchronous
# Apis v0.2.2 - publish {#v0.2.2}
Version published to github and crates.io.
**Issues**
The following issues from v0.0.11 have been *resolved*:
- *Control flow mechanisms*: these were left unchanged, but the run loop
implementations of synchronous (polling) processes were modified to track
'open channels' to prevent attempting to receive on a closed channel while
other channels have not yet closed; see the section on
[control flow mechanisms](#control-flow-mechanisms) for details
The following implementation issues from v0.0.11 have been *resolved*:
- *Session dotfile functions*: instead of generating dotfile functions in the
`def_session!` macro definition, helper functions are implemented and the
dotfile function becomes a normal trait function of `Session`.
The following issues *remain* from v0.0.11:
- [*Reusable processes*](#reusable-processes)
- *Program coherence*
- *State machine usage*
- *Session typed program control channels*
