Crate eshanized_polaris

Crate eshanized_polaris 

Source
Expand description

§Polaris

Rust-native distributed compute and orchestration framework for scaling concurrent workloads across threads, processes, or machines.

§Quick Start

use polaris::prelude::*;

#[polaris::task]
async fn compute_square(x: i32) -> PolarisResult<i32> {
    Ok(x * x)
}

#[tokio::main]
async fn main() -> PolarisResult<()> {
    // Create a local cluster for development
    let cluster = Cluster::builder()
        .with_local_node()
        .build()
        .await?;
     
    // Submit a task
    let task = Task::new("compute", bytes::Bytes::from("5"));
    let handle = cluster.submit(task).await?;
     
    // Wait for result
    let result = handle.result().await?;
    println!("Result: {:?}", result);
     
    Ok(())
}

§Features

  • Distributed Task Execution: Schedule and execute tasks across a cluster
  • Intelligent Scheduling: Pluggable schedulers (RoundRobin, LoadAware, custom)
  • Fault Tolerance: Automatic retries with exponential backoff
  • Security: mTLS by default, optional JWT authentication
  • Observability: Built-in tracing and Prometheus metrics
  • Flexible Storage: In-memory, RocksDB, or Sled backends
  • DAG Support: Define complex task dependencies

§Architecture

Polaris follows a modular architecture:

  • Cluster: Top-level orchestration API
  • Scheduler: Pluggable task scheduling strategies
  • Node: Worker nodes that execute tasks
  • Transport: Network layer (QUIC, TLS, gRPC)
  • Storage: Persistent state backends

§Examples

§Connect to a Cluster

use polaris::prelude::*;

let cluster = Cluster::connect(["10.0.0.1:7001", "10.0.0.2:7001"]).await?;

§Submit Tasks with Retry

use polaris::prelude::*;
use std::time::Duration;

let task = Task::new("my_task", bytes::Bytes::from("input"))
    .with_priority(TaskPriority::High)
    .with_timeout(Duration::from_secs(60))
    .with_max_retries(5);

let handle = cluster.submit(task).await?;

§Custom Scheduler

use polaris::prelude::*;

let cluster = Cluster::builder()
    .with_scheduler(LoadAwareScheduler::new())
    .with_local_node()
    .build()
    .await?;

Modules§

prelude
Convenient prelude for common imports Prelude module with commonly used types and traits.

Structs§

Cluster
Main cluster handle
ClusterBuilder
Builder for creating a cluster
ClusterConfig
Cluster configuration
ClusterStats
Cluster statistics
DagExecutor
DAG executor for managing task dependencies
InMemoryStorage
In-memory storage backend (default)
LoadAwareScheduler
Load-aware scheduler
MetricsCollector
Metrics collector for cluster operations
NetworkConfig
Network configuration
Node
A node in the cluster
NodeConfig
Node configuration
NodeId
Unique identifier for a node
NodeInfo
Node information
NodeRegistry
Registry for managing cluster nodes
ResourceLimits
Resource limits for a node
ResourceUsage
Current resource usage for a node
RoundRobinScheduler
Round-robin scheduler
SchedulerConfig
Scheduler configuration
SecurityConfig
Security configuration
StorageConfig
Storage configuration
Task
A task to be executed in the cluster
TaskHandle
Handle to a submitted task
TaskId
Unique identifier for a task
TaskMetadata
Task metadata
TaskResult
Task execution result

Enums§

NodeStatus
Node status
PolarisError
The main error type for Polaris operations.
TaskPriority
Task priority level
TaskStatus
Task execution status

Traits§

Scheduler
Trait for implementing custom schedulers
Storage
Storage backend trait

Functions§

default_serializer
Get default serializer based on enabled features
init_tracing
Initialize tracing subscriber

Type Aliases§

PolarisResult
Result type alias for Polaris operations.

Attribute Macros§

task
Attribute macro for defining Polaris tasks.