Struct InputBudget 

pub struct InputBudget {
    pub tokens: u32,
    pub basis: BudgetBasis,
}

The most input tokens one inference call may carry, and which limit set it.

Fields

tokens: u32

basis: BudgetBasis

Implementations

impl InputBudget

pub fn of(window: u32, cap: u32, ceiling: Option<u32>) -> Self

min(ceiling, window − cap), saturating at 0; the ceiling wins a tie.

Trait Implementations

impl Clone for InputBudget

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for InputBudget

impl Debug for InputBudget

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for InputBudget

impl PartialEq for InputBudget

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for InputBudget