Struct ProviderCapacity 

pub struct ProviderCapacity {
    pub max_concurrent_requests: u32,
    pub active_requests: u32,
    pub requests_per_second: u32,
    pub max_batch_size: u32,
    pub mtp_enabled: bool,
    pub drafter_vram_gb: Option<f32>,
    pub moe_holdings: Vec<MoeExpertHolding>,
    pub moe_roles: Vec<MoeProviderRole>,
    pub iroh_endpoint_id: Option<String>,
}

Provider capacity information

Fields

max_concurrent_requests: u32

Maximum concurrent requests

active_requests: u32

Current active requests

requests_per_second: u32

Requests per second capacity

max_batch_size: u32

Maximum batch size

mtp_enabled: bool

Multi-Token Prediction availability. Set by the provider at tenzro_registerProvider time when their serving runtime has the target’s paired drafter co-loaded (HfModelEntry.drafter_id

• mtp_kind == DraftMtp or Generic). When true, the InferenceRouter may route MTP-eligible requests preferentially to this provider; when false, it falls back to standard autoregressive providers.

drafter_vram_gb: Option<f32>

VRAM headroom (GB) the provider has reserved for the speculative drafter alongside the target. Unsloth measures ~2 GB extra for Gemma 4 MTP heads. None means the provider hasn’t declared a drafter footprint, which is fine when mtp_enabled = false.

moe_holdings: Vec<MoeExpertHolding>

MoE expert-shard declaration. When a provider can’t fit an entire MoE model (e.g. Qwen 3.5 397B-A17B) on its hardware, it can host a subset of expert weights and serve as one peer in a decentralized expert-parallel dispatch. Empty holdings means the provider does not participate in MoE expert serving for any model and is treated as a full-model replica only.

moe_roles: Vec<MoeProviderRole>

MoE-pipeline role this provider plays. Replica is the default — the provider holds the full model and serves single-peer inference. Router provides the gating-network step and fans out batched expert calls. ExpertHolder participates in the expert-shard pool. PrefillDecode runs both phases co-located (the centralized SOTA default). Providers can declare more than one role; the router picks the matching role per request.

iroh_endpoint_id: Option<String>

Iroh endpoint id of this provider. Used by the MoE router to dispatch batched expert calls over QUIC directly to the holder peer without going through the OpenAI-compatible HTTP endpoint. Required when moe_roles includes Router or ExpertHolder.

Implementations

impl ProviderCapacity

pub fn has_capacity(&self) -> bool

Checks if provider has capacity for a new request

pub fn utilization(&self) -> u8

Returns the utilization percentage (0-100)

Trait Implementations

impl Clone for ProviderCapacity

fn clone(&self) -> ProviderCapacity

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ProviderCapacity

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ProviderCapacity

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for ProviderCapacity

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for ProviderCapacity

fn eq(&self, other: &ProviderCapacity) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for ProviderCapacity

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for ProviderCapacity