Enum Event 

#[non_exhaustive]
pub enum Event {
    Input {
        id: DataId,
        metadata: Metadata,
        data: ArrowData,
    },
    InputClosed {
        id: DataId,
    },
    Stop(StopCause),
    Reload {
        operator_id: Option<OperatorId>,
    },
    Error(String),
}

Represents an incoming Dora event.

Events might be triggered by other nodes, by Dora itself, or by some external user input.

It’s safe to ignore event types that are not relevant to the node.

This enum is marked as non_exhaustive because we might add additional variants in the future. Please ignore unknown event types instead of throwing an error to avoid breakage when updating Dora.

Variants (Non-exhaustive)

Non-exhaustive enums could have additional variants added in future. Therefore, when matching against variants of non-exhaustive enums, an extra wildcard arm must be added to account for any future variants.

Input

An input was received from another node.

This event corresponds to one of the inputs of the node as specified in the dataflow YAML file.

Fields

id: DataId

The input ID, as specified in the YAML file.

Note that this is not the output ID of the sender, but the ID assigned to the input in the YAML file.

metadata: Metadata

Meta information about this input, e.g. the timestamp.

data: ArrowData

The actual data in the Apache Arrow data format.

InputClosed

An input was closed by the sender.

The sending node mapped to an input exited, so this input will receive no more data.

Fields

id: DataId

The ID of the input that was closed, as specified in the YAML file.

Note that this is not the output ID of the sender, but the ID assigned to the input in the YAML file.

Stop(StopCause)

Notification that the event stream is about to close.

The StopCause field contains the reason for the event stream closure.

Typically, nodes should exit once the event stream closes. One notable exception are nodes with no inputs, which will receive aa Event::Stop(StopCause::AllInputsClosed) right at startup. Source nodes might want to keep producing outputs still. (There is currently an open discussion of changing this behavior and not sending AllInputsClosed to nodes without inputs.)

Note: Stop events with StopCause::Manual indicate a manual stop operation issued through dora stop or a ctrl-c. Nodes must exit once receiving such a stop event, otherwise they will be killed by Dora.

Reload

Instructs the node to reload itself or one of its operators.

This event is currently only used for reloading Python operators that are started by a dora runtime process. So this event should not be sent to normal nodes yet.

Fields

operator_id: Option<OperatorId>

The ID of the operator that should be reloaded.

There is currently no case where operator_id is None.

Error(String)

Notifies the node about an unexpected error that happened inside Dora.

It’s a good idea to output or log this error for debugging.

Trait Implementations

impl Debug for Event

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.