Module execution_config

Expand description

Execution Engine Configuration System

This module provides comprehensive configuration management for the composable execution engine, including resource constraints, performance goals, fault tolerance settings, and monitoring configurations. The configuration system is designed to be flexible, extensible, and production-ready with validation and best practices built-in.

§Configuration Architecture

The configuration system follows a hierarchical structure:

ExecutionEngineConfig
├── ResourceConstraints       // CPU, memory, GPU, network limits
├── PerformanceGoals          // Throughput, latency, utilization targets
├── EnergyEfficiencyGoal      // Power consumption optimization
├── FaultToleranceConfig      // Retry, failover, circuit breaker settings
├── MonitoringConfig          // Metrics collection and observability
└── IoLimits                  // I/O bandwidth and operation limits

§Configuration Examples

§High-Performance Computing Configuration

use sklears_compose::execution_config::*;

let config = ExecutionEngineConfig {
    name: "hpc_cluster".to_string(),
    default_strategy: "distributed".to_string(),
    resource_constraints: ResourceConstraints {
        max_cpu_cores: Some(128),
        max_memory: Some(1024 * 1024 * 1024 * 1024), // 1TB
        max_gpu_devices: Some(8),
        max_concurrent_tasks: Some(1000),
        max_task_time: Some(Duration::from_hours(24)),
        memory_limit_per_task: Some(8 * 1024 * 1024 * 1024), // 8GB per task
        ..Default::default()
    },
    performance_goals: PerformanceGoals {
        target_throughput: Some(10000.0), // 10k tasks/sec
        target_latency: Some(1.0),         // 1ms avg latency
        target_utilization: Some(95.0),   // 95% resource utilization
        target_efficiency: Some(98.0),    // 98% efficiency
        ..Default::default()
    },
    energy_efficiency: EnergyEfficiencyGoal {
        target_power_consumption: Some(50000.0), // 50kW
        enable_power_capping: true,
        dynamic_frequency_scaling: true,
        energy_aware_scheduling: true,
        ..Default::default()
    },
    fault_tolerance: FaultToleranceConfig {
        enable_retry: true,
        max_retries: 5,
        backoff_strategy: BackoffStrategy::ExponentialWithJitter {
            base_delay: Duration::from_millis(100),
            max_delay: Duration::from_secs(60),
            jitter_factor: 0.1,
        },
        enable_circuit_breaker: true,
        circuit_breaker_threshold: 50,
        enable_failover: true,
        failover_targets: vec!["backup_cluster".to_string()],
        health_check: HealthCheckConfig {
            enabled: true,
            interval: Duration::from_secs(30),
            timeout: Duration::from_secs(5),
            failure_threshold: 3,
            recovery_threshold: 2,
        },
        ..Default::default()
    },
    monitoring: MonitoringConfig {
        collection_interval: Duration::from_secs(1),
        enable_detailed_metrics: true,
        enable_distributed_tracing: true,
        enable_profiling: true,
        export_prometheus: true,
        export_jaeger: true,
        custom_metrics: vec!["gpu_utilization", "memory_bandwidth"].iter().map(|s| s.to_string()).collect(),
        ..Default::default()
    },
    io_limits: IoLimits {
        max_read_bandwidth: Some(100 * 1024 * 1024 * 1024), // 100 GB/s
        max_write_bandwidth: Some(80 * 1024 * 1024 * 1024), // 80 GB/s
        max_iops: Some(1000000), // 1M IOPS
        max_concurrent_io: Some(10000),
        io_priority: IoPriority::High,
        ..Default::default()
    },
};

§Edge Computing Configuration

let edge_config = ExecutionEngineConfig {
    name: "edge_device".to_string(),
    default_strategy: "streaming".to_string(),
    resource_constraints: ResourceConstraints {
        max_cpu_cores: Some(4),
        max_memory: Some(8 * 1024 * 1024 * 1024), // 8GB
        max_concurrent_tasks: Some(50),
        max_task_time: Some(Duration::from_secs(30)),
        memory_limit_per_task: Some(100 * 1024 * 1024), // 100MB per task
        ..Default::default()
    },
    energy_efficiency: EnergyEfficiencyGoal {
        target_power_consumption: Some(50.0), // 50W
        enable_power_capping: true,
        dynamic_frequency_scaling: true,
        thermal_throttling: true,
        ..Default::default()
    },
    performance_goals: PerformanceGoals {
        target_latency: Some(10.0), // 10ms for real-time processing
        target_efficiency: Some(85.0),
        ..Default::default()
    },
    ..Default::default()
};

§Development Configuration

let dev_config = ExecutionEngineConfig {
    name: "development".to_string(),
    default_strategy: "sequential".to_string(),
    resource_constraints: ResourceConstraints {
        max_cpu_cores: Some(2),
        max_memory: Some(4 * 1024 * 1024 * 1024), // 4GB
        max_concurrent_tasks: Some(10),
        ..Default::default()
    },
    monitoring: MonitoringConfig {
        collection_interval: Duration::from_secs(5),
        enable_detailed_metrics: true,
        enable_debug_logging: true,
        ..Default::default()
    },
    fault_tolerance: FaultToleranceConfig {
        enable_retry: true,
        max_retries: 2,
        backoff_strategy: BackoffStrategy::Linear {
            delay: Duration::from_millis(500),
        },
        ..Default::default()
    },
    ..Default::default()
};

Structs§

AlertChannel: Alert channel configuration
AlertRateLimit: Alert rate limiting configuration
AlertRule: Alert rule definition
AlertingConfig: Alerting system configuration
CarbonAwareness: Carbon footprint awareness settings
EnergyEfficiencyGoal: Energy efficiency optimization goals
ExecutionEngineConfig: Main configuration for the execution engine
FaultToleranceConfig: Fault tolerance and reliability configuration
HealthCheckConfig: Health check configuration
HealthCheckEndpoint: Health check endpoint configuration
IoCompression: I/O compression settings
IoLimits: I/O operation limits and constraints
MonitoringConfig: Monitoring and observability configuration
PerformanceGoals: Performance optimization goals and targets
ResourceConstraints: Resource allocation constraints and limits
SamplingConfig: Sampling configuration for tracing

Enums§

AlertPriority: Alert priority levels
AlertSeverity: Alert severity levels
BackoffStrategy: Backoff strategies for retry mechanisms
ComparisonOperator: Comparison operators for alert rules
CompressionAlgorithm: Compression algorithms
FailureDetectionStrategy: Failure detection strategies
IoPriority: I/O priority levels
IoScheduler: I/O scheduler types
PerformanceMode: Performance optimization modes
PowerMode: Power consumption modes
QualityOfService: Quality of Service levels
RecoveryStrategy: Recovery strategies after failures
ResourceIsolationMode: Resource isolation modes

Type Aliases§

DefaultExecutionEngineConfig: Type alias for convenient default configuration

Module execution_config

Module execution_config Copy item path

§Configuration Architecture

§Configuration Examples

§High-Performance Computing Configuration

§Edge Computing Configuration

§Development Configuration

Structs§

Enums§

Type Aliases§

Module execution_config