pub struct ContextBudget {
pub max_tokens: u32,
pub margin_percent: u32,
pub overflow: BudgetOverflow,
}Expand description
A prompt-assembly budget, in estimated tokens (see the module docs for
the accounting rule). ContextBudget is the seam a model-precise
tokenizer plugs into later: the type stays, the estimate behind
estimated_tokens is what a later wave may sharpen.
Fields§
§max_tokens: u32The maximum estimated tokens the assembly may carry.
margin_percent: u32The declared safety margin of the estimate, in percent
(default DEFAULT_TOKEN_MARGIN_PERCENT; absent from the wire at
the default).
overflow: BudgetOverflowWhat to do when the next-ranked record does not fit (default
BudgetOverflow::Truncate).
Implementations§
Source§impl ContextBudget
impl ContextBudget
Sourcepub fn new(max_tokens: u32) -> Self
pub fn new(max_tokens: u32) -> Self
A budget of max_tokens estimated tokens with the declared default
margin and truncate overflow.
Sourcepub fn with_margin_percent(self, margin_percent: u32) -> Self
pub fn with_margin_percent(self, margin_percent: u32) -> Self
Builder-style: override the safety margin.
Sourcepub fn with_overflow(self, overflow: BudgetOverflow) -> Self
pub fn with_overflow(self, overflow: BudgetOverflow) -> Self
Builder-style: override the overflow policy.
Trait Implementations§
Source§impl Clone for ContextBudget
impl Clone for ContextBudget
Source§fn clone(&self) -> ContextBudget
fn clone(&self) -> ContextBudget
Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreSource§impl Debug for ContextBudget
impl Debug for ContextBudget
Source§impl<'de> Deserialize<'de> for ContextBudget
impl<'de> Deserialize<'de> for ContextBudget
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Deserialize this value from the given Serde deserializer. Read more
impl Eq for ContextBudget
Source§impl PartialEq for ContextBudget
impl PartialEq for ContextBudget
Source§impl Serialize for ContextBudget
impl Serialize for ContextBudget
impl StructuralPartialEq for ContextBudget
Auto Trait Implementations§
impl Freeze for ContextBudget
impl RefUnwindSafe for ContextBudget
impl Send for ContextBudget
impl Sync for ContextBudget
impl Unpin for ContextBudget
impl UnsafeUnpin for ContextBudget
impl UnwindSafe for ContextBudget
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more