Module integration_testing 

This document describes how to write integration tests for Dora nodes.

Usage

There are currently three ways to run integration tests for Dora nodes:

• calling setup_integration_testing before invoking the node’s main function
• testing the compiled executable by setting environment variables
• using DoraNode::init_testing tests independent of the main function

Using setup_integration_testing

The most straightforward way to write integration tests for a Dora node is to call the setup_integration_testing function in the test function and then invoke the node’s main function. This way, the full behavior of the node (including the main function) is tested.

This approach requires that the node’s main function uses the DoraNode::init_from_env function for initialization.

See the setup_integration_testing function documentation for details.

Example

#[test]
fn test_main_function() -> eyre::Result<()> {
   // specify the events that should be sent to the node
   let events = vec![
       TimedIncomingEvent {
           time_offset_secs: 0.01,
           event: IncomingEvent::Input {
               id: "tick".into(),
               metadata: None,
               data: None,
           },
       },
       TimedIncomingEvent {
           time_offset_secs: 0.055,
           event: IncomingEvent::Stop,
       },
   ];
   let inputs = dora_node_api::integration_testing::TestingInput::Input(
       IntegrationTestInput::new("node_id".parse().unwrap(), events),
   );

   // send the node's outputs to a channel so we can verify them later
   let (tx, rx) = flume::unbounded();
   let outputs = dora_node_api::integration_testing::TestingOutput::ToChannel(tx);

   // don't include time offsets in the outputs to make them deterministic
   let options = TestingOptions {
       skip_output_time_offsets: true,
   };

    // setup the integration testing environment -> this will make `DoraNode::init_from_env``
    // initialize the node in testing mode
    integration_testing::setup_integration_testing(inputs, outputs, options);

    // call the node's main function
    crate::main()?;

    // collect the nodes's outputs and compare them
    let outputs = rx.try_iter().collect::<Vec<_>>();
    assert_eq!(outputs, expected_outputs);

    Ok(())
}

Testing through Environment Variables

Dora also supports testing nodes after compilation. This is the most comprehensive way to ensure that a node behaves as expected, as the testing will be done on the exact same executable as used in the dataflow.

To enable this, the DoraNode::init_from_env function provides built-in support for integration testing through environment variables.

To start an integration test, run a node executable directly (i.e. don’t go through the dora cli) with the following environment variables set:

• DORA_TEST_WITH_INPUTS: Path to a JSON file that contains the inputs that should be sent to the node.
  • The file format is defined through the IntegrationTestInput struct (encoded as JSON).
• DORA_TEST_WRITE_OUTPUTS_TO (optional): Path at which the output JSONL file should be written.
  • defaults to a outputs.jsonl file next to the given inputs file
  • See the TestingOutput struct for details on the output file format.
• DORA_TEST_NO_OUTPUT_TIME_OFFSET (optional): If set to any value, the output JSONL file will not contain time offsets for the outputs.
  • This is useful to get deterministic outputs that can be compared against expected outputs. (The time offsets depend on your machine, system load, OS, etc and thus vary between runs.)

In integration test mode, the node will not connect to any Dora daemon or other nodes. Instead, it will read all incoming events from the given inputs file and write all outputs to the output file. The output file can then be compared against expected outputs to verify that the node behaves as intended.

Input File Format

The input file must be a JSON file that can be deserialized to a IntegrationTestInput instance.

Example

> DORA_TEST_WITH_INPUTS=path/to/inputs.json DORA_TEST_NO_OUTPUT_TIME_OFFSET=1 cargo r -p rust-dataflow-example-status-node

Using DoraNode::init_testing

While we recommend to include the main function in integration tests (as described above), there might also be cases where you want to run additional tests independent of the main function. To make this easier, Dora provides a DoraNode::init_testing initialization function to initialize an integration testing environment directly.

This function is roughly equivalent to calling setup_integration_testing followed by DoraNode::init_from_env, but it does not modify any global state or thread-local state. Thus, it can be even used to run multiple integration tests as part of a single test function.

Example

#[test]
fn test_run_function() -> eyre::Result<()> {
    // specify the events that should be sent to the node
    let events = vec![...]; // see above for details
    let inputs = dora_node_api::integration_testing::TestingInput::Input(
         IntegrationTestInput::new("node_id".parse().unwrap(), events),
    );

    let outputs = dora_node_api::integration_testing::TestingOutput::ToFile(
       std::path::PathBuf::from("path/to/outputs.jsonl"),
    );

    let (node, events) = DoraNode::init_testing(inputs, outputs, Default::default())?;
    do_something_with_node_and_events(node, events)?;
}

Generating Input Files

While manually writing input files is possible, it is often more convenient to autogenerate them by recording actual dataflow runs. This can be done by running a Dora dataflow with the DORA_WRITE_EVENTS_TO environment variable set to a folder path. This will instruct all nodes in the started dataflow to write out their received inputs to a inputs-{node_id}.json file in the given folder.

The file format used for these input files is identical to the format expected by the DORA_TEST_WITH_INPUTS environment variable. Thus, the generated files can be directly used as input files for integration tests.

Note that the implementation of this feature is currently not optimized. All incoming events are buffered in memory before being written out at the end of the node’s execution. Thus, it is not recommended to use this feature with long-running dataflows or dataflows that process large amounts of data.

Modules

integration_testing_format

Use these types for integration testing nodes.

Structs

IntegrationTestInput

Defines the input data and events for integration testing a node.

TestingOptions

Options for integration testing.

Enums

TestingInput

Provides input data for integration testing.

TestingOutput

Specifies where to write the output data of an integration test.

Functions

setup_integration_testing

Sets up an integration testing environment for the current thread.