runtara-workflows

Workflow compilation library for Runtara. Compiles JSON DSL scenarios into optimized native Rust binaries.

Overview

This crate provides:

• JSON DSL Parsing: Load workflow definitions from JSON
• Code Generation: Generate Rust code from workflow specifications
• Native Compilation: Compile generated code to optimized binaries
• Incremental Builds: Cache compiled dependencies for fast subsequent compilations

Installation

Add to your Cargo.toml:

[dependencies]
runtara-workflows = "1.0"

Usage

Compiling a Scenario

use runtara_workflows::{compile_scenario, CompilationInput, Scenario};
use std::fs;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Load scenario from JSON
    let scenario_json = fs::read_to_string("workflow.json")?;
    let scenario: Scenario = serde_json::from_str(&scenario_json)?;

    // Configure compilation
    let input = CompilationInput {
        tenant_id: "tenant-123".to_string(),
        scenario_id: "order-processing".to_string(),
        version: 1,
        execution_graph: scenario.into(),
        debug_mode: false, // Set to true to emit step telemetry events
        child_scenarios: vec![],
        connection_service_url: Some("https://my-product.com/api/connections".to_string()),
    };

    // Compile to native binary
    let result = compile_scenario(input)?;

    println!("Binary path: {}", result.binary_path);
    println!("Checksum: {}", result.binary_checksum);

    Ok(())
}

Workflow JSON Format

Workflows are defined as JSON scenarios:

{
  "name": "Data Processing",
  "description": "Fetch and transform data",
  "steps": {
    "fetch": {
      "stepType": "Agent",
      "id": "fetch",
      "agentId": "http",
      "capabilityId": "request",
      "inputMapping": {
        "url": { "valueType": "reference", "value": "data.endpoint" },
        "method": { "valueType": "immediate", "value": "GET" }
      }
    },
    "transform": {
      "stepType": "Agent",
      "id": "transform",
      "agentId": "transform",
      "capabilityId": "map-fields",
      "inputMapping": {
        "source": { "valueType": "reference", "value": "steps.fetch.outputs" },
        "mapping": {
          "valueType": "immediate",
          "value": {
            "result": "{{ body }}"
          }
        }
      }
    },
    "finish": {
      "stepType": "Finish",
      "id": "finish",
      "inputMapping": {
        "result": { "valueType": "reference", "value": "steps.transform.outputs" }
      }
    }
  },
  "entryPoint": "fetch",
  "executionPlan": [
    { "fromStep": "fetch", "toStep": "transform" },
    { "fromStep": "transform", "toStep": "finish" }
  ]
}

Step Types

Value Mappings

Child Workflows

Workflows can invoke other workflows:

let input = CompilationInput {
    tenant_id: "tenant-123".to_string(),
    scenario_id: "parent-workflow".to_string(),
    version: 1,
    execution_graph: parent_scenario.into(),
    debug_mode: false,
    // Include child scenario binaries
    child_scenarios: vec![
        ChildScenario {
            scenario_id: "child-workflow".to_string(),
            binary_path: "/path/to/child_binary".to_string(),
        },
    ],
    connection_service_url: None,
};

Using the CLI

The crate provides runtara-compile for command-line compilation:

# Compile a scenario
runtara-compile --workflow workflow.json --tenant tenant-123 --scenario my-workflow

# Copy to specific location
runtara-compile --workflow workflow.json --tenant tenant-123 --scenario my-workflow --output ./my-workflow

# With debug mode (emits step telemetry events)
runtara-compile --workflow workflow.json --tenant tenant-123 --scenario my-workflow --debug

Debug Mode

When compiling with debug_mode: true (or --debug CLI flag), the compiled workflow emits telemetry events for each step execution. These events are stored as custom events in runtara-core and can be used for:

• Step-level tracing: See exactly which steps executed and in what order
• Performance analysis: Measure step execution duration
• Input/output inspection: Capture step inputs and outputs (truncated to 10KB)
• Debugging failures: Understand workflow state at each step

Debug mode emits two event types per step:

Example payload for step_debug_start:

{
  "step_id": "fetch-order",
  "step_name": "Fetch Order",
  "step_type": "Agent",
  "timestamp_ms": 1703001234567,
  "inputs": { "order_id": "ORD-123" },
  "input_mapping": { "order_id": { "valueType": "reference", "value": "data.orderId" } }
}

Example payload for step_debug_end:

{
  "step_id": "fetch-order",
  "step_name": "Fetch Order", 
  "step_type": "Agent",
  "timestamp_ms": 1703001234717,
  "duration_ms": 150,
  "outputs": { "status": 200, "body": "..." }
}

Query debug events from the database:

SELECT subtype, payload, created_at 
FROM instance_events 
WHERE instance_id = $1 AND event_type = 'custom'
ORDER BY created_at;

Environment Variables

Custom Workflow Stdlib

Products can extend the standard library with custom agents:

1. Create a crate that re-exports runtara-workflow-stdlib:

   // my-product-stdlib/src/lib.rs
   pub use runtara_workflow_stdlib::*;
   
   // Add product-specific agents
   pub mod my_custom_agents;
   

2. Compile to .rlib and place in your native library directory

3. Set environment variables:

   export RUNTARA_NATIVE_LIBRARY_DIR=/path/to/native_cache
   export RUNTARA_STDLIB_NAME=my_product_stdlib
   

Compilation Output

The compile_scenario function returns:

pub struct CompilationResult {
    /// Path to the compiled binary
    pub binary_path: String,
    /// SHA-256 checksum of the binary (for caching)
    pub binary_checksum: String,
    /// Compilation metadata
    pub metadata: CompilationMetadata,
}

Integration with Management SDK

After compilation, register the binary with runtara-environment:

use runtara_management_sdk::{ManagementSdk, SdkConfig, RegisterImageOptions};
use runtara_workflows::{compile_scenario, CompilationInput};
use std::fs;

// Compile the workflow
let result = compile_scenario(input)?;

// Read the compiled binary
let binary = fs::read(&result.binary_path)?;

// Register with runtara-environment
let sdk = ManagementSdk::new(SdkConfig::localhost())?;
sdk.connect().await?;

let registration = sdk.register_image(
    RegisterImageOptions::new("tenant-123", "my-workflow", binary)
).await?;

// Start instances using the image_id
sdk.start_instance(StartInstanceOptions::new(&registration.image_id, "tenant-123")).await?;

Related Crates

• runtara-dsl - DSL type definitions
• runtara-workflow-stdlib - Standard library for compiled workflows
• runtara-agents - Built-in agent implementations
• runtara-management-sdk - Register and run compiled workflows

License

This project is licensed under AGPL-3.0-or-later.