Crate forge_orchestration 

Forge Orchestration

A Rust-native orchestration platform for distributed workloads with Mixture of Experts (MoE) routing, autoscaling, and Nomad integration.

Features

• Job Management: Define and submit jobs with task groups
• MoE Routing: Intelligent request routing to expert workers
• Autoscaling: Threshold and predictive scaling policies
• Nomad Integration: Schedule via HashiCorp Nomad
• Metrics: Prometheus-compatible metrics export
• SDK: Embedded SDK for workloads (lifecycle, ports, heartbeats)

Quick Start

use forge_orchestration::{ForgeBuilder, AutoscalerConfig, Job, Task, Driver};

#[tokio::main]
async fn main() -> forge_orchestration::Result<()> {
    let forge = ForgeBuilder::new()
        .with_autoscaler(AutoscalerConfig::default())
        .build()?;

    let job = Job::new("my-service")
        .with_group("api", Task::new("server")
            .driver(Driver::Exec)
            .command("/usr/bin/server"));

    forge.submit_job(job).await?;
    forge.run().await?;
    Ok(())
}

SDK Usage (for workloads)

use forge_orchestration::sdk::{ready, allocate_port, graceful_shutdown};

#[tokio::main]
async fn main() -> forge_orchestration::Result<()> {
    ready()?;
    let port = allocate_port(8000..9000)?;
    graceful_shutdown();
    // ... serve on port ...
    Ok(())
}

Re-exports

pub use autoscaler::Autoscaler;

pub use autoscaler::AutoscalerConfig;

pub use autoscaler::ScalingDecision;

pub use builder::ForgeBuilder;

pub use error::ForgeError;

pub use error::Result;

pub use job::Driver;

pub use job::Job;

pub use job::Task;

pub use job::TaskGroup;

pub use metrics::ForgeMetrics;

pub use metrics::MetricsExporter;

pub use metrics::MetricsHook;

pub use moe::DefaultMoERouter;

pub use moe::LoadAwareMoERouter;

pub use moe::MoERouter;

pub use moe::RoundRobinMoERouter;

pub use moe::RouteResult;

pub use moe::GpuAwareMoERouter;

pub use moe::VersionAwareMoERouter;

pub use networking::HttpServer;

pub use networking::HttpServerConfig;

pub use nomad::NomadClient;

pub use runtime::Forge;

pub use storage::FileStore;

pub use storage::MemoryStore;

pub use storage::StateStore;

pub use types::Expert;

pub use types::GpuResources;

pub use types::NodeId;

pub use types::Region;

pub use types::Shard;

pub use types::ShardId;

Modules

autoscaler

Autoscaling module for Forge

builder

ForgeBuilder for configuring and constructing Forge instances

controlplane

Control Plane for Forge Orchestration

error

Error types for Forge

federation

Multi-Region Federation for Forge Orchestration

inference

AI/ML inference utilities for Forge

job

Job and Task definitions for Forge orchestration

metrics

Metrics and monitoring for Forge

moe

Mixture of Experts (MoE) routing for Forge

networking

Networking module for Forge

nomad

Nomad client integration for Forge

prelude

Prelude module for convenient imports

resilience

Resilience patterns for distributed systems

runtime

Forge runtime and control plane

scheduler

Distributed Scheduler for Forge Orchestration

sdk

Forge SDK

storage

Storage backends for Forge state management

types

Core types for Forge orchestration