rustvello 0.5.2

Documentation: https://rustvello.readthedocs.io

Source Code: https://github.com/pynenc/rustvello

Rustvello is a distributed task orchestration engine — broker, orchestrator, state backend, trigger system, client data store, and runner — implemented in Rust for performance and safety. It works standalone from both Rust and Python (via PyO3 bindings), and also integrates with pynenc as an optional high-performance backend plugin.

Repository Structure

This is a multi-crate Rust workspace with Python bindings:

For the full architecture, see ARCHITECTURE.md.

Key Features

• Typed Task System: proc-macro #[rustvello::task] generates serializable params, deterministic call IDs, and compile-time auto-discovery via inventory
• Invocation State Machine: 13-state FSM with guarded transitions, ownership tracking, and automatic recovery
• Pluggable Backends: Swap between in-memory, SQLite, Redis, PostgreSQL, MongoDB, and RabbitMQ backends via feature flags
• Concurrency Control: Four levels (Unlimited, Task, Argument, None) enforced at both registration and execution time
• Queues and Priorities: Named logical queues, configurable runner selection, and finite float priorities with FIFO ties
• Trigger System: Event-driven and cron-scheduled task execution with durable event/run evidence in memory and SQLite
• Client Data Store: SHA-256 content-addressed external storage for large arguments/results with LRU caching
• Workflow System: Explicit #[rustvello::workflow] roots, child identity propagation, and root-scoped deterministic replay
• Recovery & Heartbeat: Automatic detection and re-routing of stale invocations from crashed runners
• Monitoring Dashboard: Browser-based UI for invocations, runners, workflows, trigger evidence, and timelines (Axum + Askama + HTMX)
• Cross-Language Support: Closed TaskLanguage, canonical language::module.name task IDs, typed foreign tasks, and physical language queues
• Builder Pattern: Fluent configuration with env var overrides (RUSTVELLO__*), TOML file support, and .memory()/.sqlite() presets
• Python Bindings: Full PyO3 bridge for standalone Python usage and optional pynenc integration
• CLI Tool: Run workers, inspect invocations, and purge data from the command line
• Shared Test Suite: Macro-generated backend compliance tests ensuring all implementations satisfy the same contracts

Installation

Rust

cargo add rustvello

Feature flags:

• mem (default) — in-memory backends
• sqlite — SQLite backends
• redis — Redis backends
• mongodb — MongoDB backends
• mongodb3 — MongoDB backends (legacy driver v2)
• rabbitmq — RabbitMQ backends
• postgres — PostgreSQL backends
• full — all backends

[dependencies]
rustvello = { version = "0.5.2", features = ["full"] }

Python

pip install rustvello

CLI

cargo install rustvello-cli

Quick Start (Rust)

use rustvello::prelude::*;

// Define a task with the proc macro
#[rustvello::task(max_retries = 2, concurrency = "task", queue = "orders", priority = 25.5)]
fn process_order(order_id: String) -> String {
    format!("processed {}", order_id)
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Build an app with in-memory backends and auto-discovered tasks
    let mut app = Rustvello::builder()
        .app_id("my-app")
        .memory()
        .auto_discover_tasks()
        .build()?;

    // Submit a task — unified call routing (sync/distributed)
    let invocation = app.call(
        &ProcessOrderTask,
        ProcessOrderParams { order_id: "123".into() },
    ).await?;

    // Get result (async for distributed, immediate for dev mode)
    let result: String = invocation.result().await?;
    println!("Result: {result}");

    Ok(())
}

Quick Start (Python)

from rustvello import App, workflow_root

app = App(backend="sqlite", db_path="./tasks.db")

@app.task(max_retries=2)
def add(x: int, y: int) -> int:
    return x + y

@app.workflow
def process_order(order_id: str) -> dict[str, str]:
    root = workflow_root()
    return {"order_id": order_id, "run_id": root.uuid()}

# Submit and wait for result
inv = add(1, 2)
result = inv.result(timeout=30)  # 3

Pynenc Integration

Rustvello also serves as an optional high-performance backend for pynenc. Install the plugin with pip install pynenc-rustvello to use Rust-powered backends inside pynenc apps:

from pynenc import Pynenc

app = Pynenc()

@app.task
def add(x: int, y: int) -> int:
    return x + y

result = add(1, 2).result  # 3

Development

Prerequisites: Rust 1.85+, Python 3.12+, uv, maturin

# Install dependencies and pre-commit hooks
make install

# Run all checks (Rust + Python + pre-commit)
make check

# Run all tests (Rust + Python)
make test

# Build the Python wheel
make build

# Build and serve docs locally
make docs-serve

Run make help for the full list of targets.

Contributing

See CONTRIBUTING.md for guidelines on reporting bugs, submitting PRs, commit conventions, and the development workflow.

Contact or Support

• GitHub Issues: Bug reports and feature requests
• GitHub Discussions: Questions and ideas

License

Rustvello is released under the MIT License.