Forge Orchestration

Rust-Native Orchestration Platform for Distributed Workloads

A high-performance orchestration platform for Rust, designed to manage distributed workloads at hyper-scale — with intelligent bin-packing, a closed reconcile → schedule → bind → persist control loop, and simulation/agent-native scheduling primitives (gang co-scheduling and tick-deadline ordering).

Performance Benchmarks

Measured on an Intel Core i9-10980XE (18C/36T @ 3.0 GHz), Windows 11, rustc 1.95.0, release build via Criterion (30 samples, 2 s measurement). These are absolute throughput numbers on one machine — not a comparison against Kubernetes (Forge does not benchmark K8s here), and as wall-clock microbenchmarks on a non-isolated machine they vary run-to-run (≈1.5×). Throughput = workloads ÷ median per-batch time. Reproduce:

cargo bench --bench bind_cycle_benchmark --bench scheduler_benchmark

Honest bind cycle — OptimizedScheduler, sequential commit path

Each call finds the best node and commits the allocation (decrements capacity in the node and the score cache). This is the number to quote for "how fast can we actually schedule and bind a workload?"

¹ adds serde_json serialization of the bind record + a MemoryStore write per workload (approximates the etcd/state-store write a real control plane pays). The ~510K binds/s at 100 nodes is the most realistic single-workload figure. ² schedule_and_commit_batch over 500 workloads.

Monolithic Scheduler — full affinity / taint / preemption semantics

Fresh cluster, 500 workloads, per-workload median:

Online LearnedScheduler score + feedback update: ~0.15 µs (~6.8M updates/s).

Resolved: schedule_fast is now sequential (an earlier Rayon path was a pessimization)

OptimizedScheduler::schedule_fast previously switched to Rayon par_iter above 16 nodes. Benchmarks showed that was a 40–300× pessimization for the trivial integer scoring — the per-call fan-out cost dwarfed the work. The Rayon path (and the rayon dependency) has been removed; schedule_fast now uses the same sequential scan as schedule_fast_commit. Measured score_only, before vs after:

It now tracks schedule_commit within noise (scoring without committing does marginally less work). This also retired the earlier "10–2000× faster than K8s / 1M decisions/sec" numbers, which came from the non-committing scoring loop and did not reflect a real bind.

Genuine performance properties (confirmed by the numbers above)

• Integer-only scoring — the hot path is allocation-free u32 math, no floating point.
• Pre-computed score cache — schedule_fast_commit reads/decrements a cached view in O(nodes).
• Sequential committing bind scales smoothly: ~1 µs/bind at 100 nodes, ~5 µs at 500 (no Rayon).
• First-Fit Decreasing bin-packing for utilization.

Features

Installation

[dependencies]

forge-orchestration = "0.5"

tokio = { version = "1", features = ["full"] }

Quick Start

Control Plane

use forge_orchestration::{ForgeBuilder, AutoscalerConfig, Job, Task, Driver};

#[tokio::main]
async fn main() -> forge_orchestration::Result<()> {
    // Build the orchestrator
    let forge = ForgeBuilder::new()
        .with_autoscaler(AutoscalerConfig::default()
            .upscale_threshold(0.8)
            .downscale_threshold(0.3))
        .build()?;

    // Define and submit a job
    let job = Job::new("my-service")
        .with_group("api", Task::new("server")
            .driver(Driver::Exec)
            .command("/usr/bin/server")
            .args(vec!["--port", "8080"])
            .resources(500, 256));

    forge.submit_job(job).await?;

    // Run the control plane
    forge.run().await?;
    Ok(())
}

Workload SDK

The SDK is included in the main crate under forge_orchestration::sdk:

use forge_orchestration::sdk::{ready, allocate_port, graceful_shutdown, shutdown_signal};

#[tokio::main]
async fn main() -> forge_orchestration::Result<()> {
    // Signal readiness to orchestrator
    ready()?;

    // Allocate a port dynamically
    let port = allocate_port(8000..9000)?;
    println!("Listening on port {}", port);

    // Install graceful shutdown handlers
    graceful_shutdown();

    // ... your server logic ...

    // Wait for shutdown signal
    shutdown_signal().await;
    Ok(())
}

Architecture

[User App] --> [Forge SDK] (ready(), allocate(), shutdown())
              |
              v
[Forge Control Plane]
  - Tokio Runtime (async loops)
  - Rayon (parallel alloc)
  - Raft (single-node consensus, opt-in `raft` feature)
  - State: in-memory / file (default), single-node Raft (opt-in); RocksDB + etcd planned
  - MoE Router (gating to experts)
  |
  v
[Nomad Scheduler] (jobs: containers/binaries)
  |
  v
[Workers/Nodes]
  - QUIC/TLS Networking
  - Prometheus Metrics

API Reference

Modules

MoE Routing

Built-in routers:

• DefaultMoERouter: Hash-based consistent routing
• LoadAwareMoERouter: Routes to least-loaded expert with affinity
• RoundRobinMoERouter: Sequential distribution

Custom router:

use forge_orchestration::moe::{MoERouter, RouteResult};
use async_trait::async_trait;

struct MyRouter;

#[async_trait]
impl MoERouter for MyRouter {
    async fn route(&self, input: &str, num_experts: usize) -> RouteResult {
        RouteResult::new(input.len() % num_experts)
    }
    fn name(&self) -> &str { "my-router" }
}

Autoscaling

use forge_orchestration::AutoscalerConfig;

let config = AutoscalerConfig::default()
    .upscale_threshold(0.8)
    .downscale_threshold(0.3)
    .hysteresis_secs(300)
    .bounds(1, 100);

Storage

use forge_orchestration::storage::{MemoryStore, FileStore};

let memory = MemoryStore::new();
let file = FileStore::open("/var/lib/forge/state.json")?;

Metrics

use forge_orchestration::ForgeMetrics;

let metrics = ForgeMetrics::new()?;
metrics.record_job_submitted();
metrics.record_scale_event("my-job", "up");
let text = metrics.gather_text()?;

SDK Functions

Environment Variables

Builder Configuration

use forge_orchestration::ForgeBuilder;

ForgeBuilder::new()
    .with_nomad_api("http://localhost:4646")
    .with_nomad_token("secret-token")
    .with_store_path("/var/lib/forge/state.json")
    .with_node_name("forge-1")
    .with_datacenter("dc1")
    .with_autoscaler(AutoscalerConfig::default())
    .with_metrics(true)
    .build()?

License

Apache 2.0