Struct TensorBudgetManager 

pub struct TensorBudgetManager {
    pub sessions: HashMap<u64, SessionBudget>,
    pub next_session_id: u64,
}

Orchestrates computational budgets across multiple concurrent inference sessions.

Example

use ipfrs_tensorlogic::budget_manager::{TensorBudgetManager, ResourceType};

let mut mgr = TensorBudgetManager::new();
let sid = mgr.open_session(vec![
    (ResourceType::Flops,       1_000_000),
    (ResourceType::MemoryBytes, 256 * 1024 * 1024),
]);

mgr.consume(sid, ResourceType::Flops, 500_000).expect("example: should succeed in docs");
assert!(mgr.close_session(sid));

Fields

sessions: HashMap<u64, SessionBudget>

Active sessions keyed by session id.

next_session_id: u64

Monotonically-increasing session id counter.

Implementations

impl TensorBudgetManager

pub fn new() -> Self

Create a new, empty budget manager.

pub fn open_session(&mut self, limits: Vec<(ResourceType, u64)>) -> u64

Open a new session with the supplied resource limits.

Returns the unique session id assigned to the new session.

pub fn consume( &mut self, session_id: u64, resource: ResourceType, amount: u64, ) -> Result<(), String>

Consume amount units of resource for the given session.

Errors

• Err("session not found") — unknown session_id.
• Err("budget exceeded") — the resource budget would be exceeded.

pub fn close_session(&mut self, session_id: u64) -> bool

Close (remove) a session.

Returns true if the session existed and was removed, false otherwise.

pub fn stats(&self) -> BudgetManagerStats

Compute aggregate statistics across all currently open sessions.

pub fn session_utilization( &self, session_id: u64, ) -> Option<Vec<(ResourceType, f64)>>

Return per-resource utilisation for the given session.

Returns None if the session does not exist.

Trait Implementations

impl Debug for TensorBudgetManager

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for TensorBudgetManager

fn default() -> TensorBudgetManager

Returns the “default value” for a type.