Struct InferencePoolConfig 

pub struct InferencePoolConfig { /* private fields */ }

Configuration for the world’s per-model inference concurrency limits.

A model listed in per_model uses that limit; any other model uses default_limit (or is unbounded when that is None). With a single world today this is just a global config table.

Implementations

impl InferencePoolConfig

pub fn new() -> Self

An empty config: every model unbounded.

pub fn with_default(self, limit: Option<usize>) -> Self

Set the fallback limit applied to models with no explicit entry. None leaves unlisted models unbounded.

pub fn set_limit(&mut self, model: impl Into<String>, limit: usize)

Set the concurrency limit for a specific model key.

pub fn limit_for(&self, model: &str) -> Option<usize>

The configured limit for model: its explicit entry if present, else the default. None means unbounded.

Trait Implementations

impl Clone for InferencePoolConfig

fn clone(&self) -> InferencePoolConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for InferencePoolConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for InferencePoolConfig

fn default() -> InferencePoolConfig

Returns the “default value” for a type.