obelisk 0.19.3

Deterministic workflow engine
obelisk-0.19.3 is not a library.

[!WARNING] Pre-release: Expect changes in CLI, gRPC, WIT, and database schema.

Obelisk in Action

Watch the Demo Video

The Stargazers app features:

  • A webhook listening to GitHub star events.
  • Activities for interacting with Turso DB, OpenAI, and GitHub.
  • A workflow orchestrating the activities.

What's Included

  • Obelisk Runtime: A single binary executing deterministic workflows, activities, and webhook endpoints, persisting steps in execution log using SQLite.
  • Control Interfaces:
    • CLI: Manage executions via the obelisk command.
    • gRPC API: Programmatic interaction.
    • Web UI: View executions and submit function executions.

Core Principles

  • Replayable Workflows: Deterministic execution ensures reliable recovery, debugging, and auditing.
  • Resilient Activities: Automatic retries on errors and timeouts, with persistent input and results.
  • Schema-first design with end-to-end type safety: Uses WASM Component Model and WIT IDL for generating API bindings.

Use Cases

  • Periodic Tasks: Automate periodic checks with complex logic using regular code.
  • Background Jobs: Offload tasks with built-in error handling and retries.
  • Batch jobs: Manage large-scale tasks like faas deployments.
  • End-to-End Testing: Automate tests with detailed logs of each step.

Key Features

Activities

  • Must be idempotent (retriable). This contract must be fulfilled by the activity itself.
  • Executed in a WASM sandbox.
  • Support for HTTP requests via WASI 0.2 HTTP client.
  • Execution timeout handling.
  • Automatic retries on errors, timeouts, and panics.
  • Persistent execution results.

Deterministic Workflows

  • Fully replayable with persistent execution log.
  • Executed in a WASM sandbox.
  • Automatic retries on failures.
  • Support for spawning child executions with structured concurrency.
  • Distributed sagas (planned).

Webhook Endpoints

  • Mounted as a URL path, serving HTTP traffic.
  • Executed in a WASM sandbox.
  • Support for spawning child executions.

Work Stealing Executor

  • Concurrency limits and customizable retry handling.

Installation

Supported Platforms

  • Linux x64, arm64 (musl, glibc v2.35+, NixOS)
  • macOS 13 x64
  • macOS 14 arm64

Pre-built Binary

curl -L --tlsv1.2 -sSf https://raw.githubusercontent.com/obeli-sk/obelisk/main/download.sh | bash

Or use cargo-binstall:

cargo binstall obelisk

Docker

docker run --net=host getobelisk/obelisk

From Source

Requires protoc.

cargo install --locked obelisk

Using Nix:

nix run github:obeli-sk/obelisk/latest

Getting Started

Check out the Getting Started Guide for details.

Start the Server

obelisk server run

CLI Usage

obelisk client component list
# Call fibonacci(10) activity from the workflow 500 times in series.
obelisk client execution submit testing:fibo-workflow/workflow.fiboa '[10, 500]' --follow

Web UI

Visit localhost:8080 to manage components, functions and execution history.

Contributing

This project has a roadmap and features are added in a certain order. Discuss features via GitHub Discussions before contributing. A Contributor License Agreement is required.

Development

Set up dependencies via Nix:

nix develop

Or manually install dependencies (see dev-deps.txt).

Run the program:

cargo run --release

Running Tests

./scripts/test.sh