Module resource_manager 

Global Resource Manager

This module provides centralized resource governance across the entire application, preventing resource oversubscription when processing multiple files concurrently.

Architecture Pattern: Two-Level Resource Governance

Problem: Without global limits, multiple concurrent files can overwhelm the system:

• 10 files × 8 workers/file = 80 concurrent tasks on an 8-core machine
• Result: CPU oversubscription, cache thrashing, poor throughput

Solution: Two-level coordination:

1. Global limits (this module) - Cap total system resources
2. Local limits (per-file semaphores) - Cap per-file concurrency

Educational Example

use adaptive_pipeline::infrastructure::runtime::RESOURCE_MANAGER;

async fn process_file() -> Result<()> {
    // 1. Acquire global CPU token (waits if system is saturated)
    let _cpu_permit = RESOURCE_MANAGER.acquire_cpu().await?;

    // 2. Acquire local per-file token
    let _local_permit = file_semaphore.acquire().await?;

    // 3. Do CPU-intensive work
    compress_data().await?;

    // 4. Both permits released automatically (RAII)
    Ok(())
}

Resource Types

CPU Tokens

• Purpose: Limit total CPU-bound work across all files
• Default: available_cores - 1 (leave one for OS/I/O)
• Use: Acquire before Rayon work or CPU-intensive operations

I/O Tokens

• Purpose: Prevent I/O queue overrun
• Default: Device-specific (NVMe: 24, SSD: 12, HDD: 4)
• Use: Acquire before file reads/writes

Memory Tracking

• Purpose: Monitor memory usage (gauge only, no enforcement yet)
• Default: No limit (soft monitoring)
• Future: Can add hard cap in Phase 3

Structs

GlobalResourceManager

Global resource manager for system-wide resource coordination

ResourceConfig

Configuration for global resource manager

Enums

StorageType

Storage device type for I/O queue depth optimization

Statics

RESOURCE_MANAGER

Legacy alias for backward compatibility

Functions

init_resource_manager

Initialize the global resource manager with custom configuration

resource_manager

Access the global resource manager