Node

Struct Node 

Source
pub struct Node {
Show 18 fields pub id: NodeId, pub name: Option<String>, pub description: Option<String>, pub path: Option<String>, pub args: Option<String>, pub env: Option<BTreeMap<String, EnvValue>>, pub operators: Option<RuntimeNode>, pub operator: Option<SingleOperatorDefinition>, pub custom: Option<CustomNode>, pub outputs: BTreeSet<DataId>, pub inputs: BTreeMap<DataId, Input>, pub send_stdout_as: Option<String>, pub build: Option<String>, pub git: Option<String>, pub branch: Option<String>, pub tag: Option<String>, pub rev: Option<String>, pub deploy: Option<Deploy>,
}
Expand description

§Dora Node Configuration

A node represents a computational unit in a Dora dataflow. Each node runs as a separate process and can communicate with other nodes through inputs and outputs.

Fields§

§id: NodeId

Unique node identifier. Must not contain / characters.

Node IDs can be arbitrary strings with the following limitations:

  • They must not contain any / characters (slashes).
  • We do not recommend using whitespace characters (e.g. spaces) in IDs

Each node must have an ID field.

§Example

nodes:
  - id: camera_node
  - id: some_other_node
§name: Option<String>

Human-readable node name for documentation.

This optional field can be used to define a more descriptive name in addition to a short id.

§Example

nodes:
  - id: camera_node
    name: "Camera Input Handler"
§description: Option<String>

Detailed description of the node’s functionality.

§Example

nodes:
  - id: camera_node
    description: "Captures video frames from webcam"
§path: Option<String>

Path to executable or script that should be run.

Specifies the path of the executable or script that Dora should run when starting the dataflow. This can point to a normal executable (e.g. when using a compiled language such as Rust) or a Python script.

Dora will automatically append a .exe extension on Windows systems when the specified file name has no extension.

§Example

nodes:
  - id: rust-example
    path: target/release/rust-node
  - id: python-example
    path: ./receive_data.py

§URL as Path

The path field can also point to a URL instead of a local path. In this case, Dora will download the given file when starting the dataflow.

Note that this is quite an old feature and using this functionality is not recommended anymore. Instead, we recommend using a git and/or build key.

§args: Option<String>

Command-line arguments passed to the executable.

The command-line arguments that should be passed to the executable/script specified in path. The arguments should be separated by space. This field is optional and defaults to an empty argument list.

§Example

nodes:
  - id: example
    path: example-node
    args: -v --some-flag foo
§env: Option<BTreeMap<String, EnvValue>>

Environment variables for node builds and execution.

Key-value map of environment variables that should be set for both the build operation and the node execution (i.e. when the node is spawned through path).

Supports strings, numbers, and booleans.

§Example

nodes:
  - id: example-node
    path: path/to/node
    env:
      DEBUG: true
      PORT: 8080
      API_KEY: "secret-key"
§operators: Option<RuntimeNode>

Multiple operators running in a shared runtime process.

Operators are an experimental, lightweight alternative to nodes. Instead of running as a separate process, operators are linked into a runtime process. This allows running multiple operators to share a single address space (not supported for Python currently).

Operators are defined as part of the node list, as children of a runtime node. A runtime node is a special node that specifies no path field, but contains an operators field instead.

§Example

nodes:
  - id: runtime-node
    operators:
      - id: processor
        python: process.py
§operator: Option<SingleOperatorDefinition>

Single operator configuration.

This is a convenience field for defining runtime nodes that contain only a single operator. This field is an alternative to the operators field, which can be used if there is only a single operator defined for the runtime node.

§Example

nodes:
  - id: runtime-node
    operator:
      id: processor
      python: script.py
      outputs: [data]
§custom: Option<CustomNode>

Legacy node configuration (deprecated).

Please use the top-level path, args, etc. fields instead.

§outputs: BTreeSet<DataId>

Output data identifiers produced by this node.

List of output identifiers that the node sends. Must contain all output_id values that the node uses when sending output, e.g. through the send_output function.

§Example

nodes:
  - id: example-node
    outputs:
      - processed_image
      - metadata
§inputs: BTreeMap<DataId, Input>

Input data connections from other nodes.

Defines the inputs that this node is subscribing to.

The inputs field should be a key-value map of the following format:

input_id: source_node_id/source_node_output_id

The components are defined as follows:

  • input_id is the local identifier that should be used for this input.

    This will map to the id field of Event::Input events sent to the node event loop.

  • source_node_id should be the id field of the node that sends the output that we want to subscribe to

  • source_node_output_id should be the identifier of the output that that we want to subscribe to

§Example

nodes:
  - id: example-node
    outputs:
      - one
      - two
  - id: receiver
    inputs:
        my_input: example-node/two
§send_stdout_as: Option<String>

Redirect stdout/stderr to a data output.

This field can be used to send all stdout and stderr output of the node as a Dora output. Each output line is sent as a separate message.

§Example

nodes:
  - id: example
    send_stdout_as: stdout_output
  - id: logger
    inputs:
        example_output: example/stdout_output
§build: Option<String>

Build commands executed during dora build. Each line runs separately.

The build key specifies the command that should be invoked for building the node. The key expects a single- or multi-line string.

Each line is run as a separate command. Spaces are used to separate arguments.

Note that all the environment variables specified in the env field are also applied to the build commands.

§Special treatment of pip

Build lines that start with pip or pip3 are treated in a special way: If the --uv argument is passed to the dora build command, all pip/pip3 commands are run through the uv package manager.

§Example

nodes:
- id: build-example
  build: cargo build -p receive_data --release
  path: target/release/receive_data
- id: multi-line-example
  build: |
      pip install requirements.txt
      pip install -e some/local/package
  path: package

In the above example, the pip commands will be replaced by uv pip when run through dora build --uv.

§git: Option<String>

Git repository URL for downloading nodes.

The git key allows downloading nodes (i.e. their source code) from git repositories. This can be especially useful for distributed dataflows.

When a git key is specified, dora build automatically clones the specified repository (or reuse an existing clone). Then it checks out the specified branch, tag, or rev, or the default branch if none of them are specified. Afterwards it runs the build command if specified.

Note that the git clone directory is set as working directory for both the build command and the specified path.

§Example

nodes:
  - id: rust-node
    git: https://github.com/dora-rs/dora.git
    build: cargo build -p rust-dataflow-example-node
    path: target/debug/rust-dataflow-example-node

In the above example, dora build will first clone the specified git repository and then run the specified build inside the local clone directory. When dora run or dora start is invoked, the working directory will be the git clone directory too. So a relative path will start from the clone directory.

§branch: Option<String>

Git branch to checkout after cloning.

The branch field is only allowed in combination with the git field. It specifies the branch that should be checked out after cloning. Only one of branch, tag, or rev can be specified.

§Example

nodes:
  - id: rust-node
    git: https://github.com/dora-rs/dora.git
    branch: some-branch-name
§tag: Option<String>

Git tag to checkout after cloning.

The tag field is only allowed in combination with the git field. It specifies the git tag that should be checked out after cloning. Only one of branch, tag, or rev can be specified.

§Example

nodes:
  - id: rust-node
    git: https://github.com/dora-rs/dora.git
    tag: v0.3.0
§rev: Option<String>

Git revision (e.g. commit hash) to checkout after cloning.

The rev field is only allowed in combination with the git field. It specifies the git revision (e.g. a commit hash) that should be checked out after cloning. Only one of branch, tag, or rev can be specified.

§Example

nodes:
  - id: rust-node
    git: https://github.com/dora-rs/dora.git
    rev: 64ab0d7c
§deploy: Option<Deploy>

Unstable machine deployment configuration

Trait Implementations§

Source§

impl Clone for Node

Source§

fn clone(&self) -> Node

Returns a duplicate of the value. Read more
1.0.0 · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for Node

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter. Read more
Source§

impl<'de> Deserialize<'de> for Node

Source§

fn deserialize<__D>( __deserializer: __D, ) -> Result<Node, <__D as Deserializer<'de>>::Error>
where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer. Read more
Source§

impl JsonSchema for Node

Source§

fn schema_name() -> Cow<'static, str>

The name of the generated JSON Schema. Read more
Source§

fn schema_id() -> Cow<'static, str>

Returns a string that uniquely identifies the schema produced by this type. Read more
Source§

fn json_schema(generator: &mut SchemaGenerator) -> Schema

Generates a JSON Schema for this type. Read more
Source§

fn inline_schema() -> bool

Whether JSON Schemas generated for this type should be included directly in parent schemas, rather than being re-used where possible using the $ref keyword. Read more
Source§

impl NodeExt for Node

Source§

fn kind(&self) -> Result<NodeKind<'_>>

Source§

impl Serialize for Node

Source§

fn serialize<__S>( &self, __serializer: __S, ) -> Result<<__S as Serializer>::Ok, <__S as Serializer>::Error>
where __S: Serializer,

Serialize this value into the given Serde serializer. Read more

Auto Trait Implementations§

§

impl Freeze for Node

§

impl RefUnwindSafe for Node

§

impl Send for Node

§

impl Sync for Node

§

impl Unpin for Node

§

impl UnwindSafe for Node

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> DynClone for T
where T: Clone,

Source§

fn __clone_box(&self, _: Private) -> *mut ()

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

impl<T> Allocation for T
where T: RefUnwindSafe + Send + Sync,

Source§

impl<T> DeserializeOwned for T
where T: for<'de> Deserialize<'de>,