pub struct ContextMeter {
pub prompt_chars: usize,
pub memory_chars: usize,
pub tools_chars: usize,
pub attachments_chars: usize,
pub transcript_chars: usize,
pub max_tokens: u64,
}Expand description
Character counts per category of input that contributes to the model context window. All five categories are tracked separately so a UI can show where the budget is going.
Fields§
§prompt_chars: usize§memory_chars: usize§tools_chars: usize§attachments_chars: usize§transcript_chars: usize§max_tokens: u64The model’s hard context limit, in tokens. Used by usage_ratio.
Implementations§
Source§impl ContextMeter
impl ContextMeter
pub fn new(max_tokens: u64) -> Self
pub fn total_chars(&self) -> usize
pub fn estimated_tokens(&self) -> u64
Sourcepub fn usage_ratio(&self) -> f64
pub fn usage_ratio(&self) -> f64
Fraction of the budget consumed, in [0.0, +inf). > 1.0 means the
estimate already exceeds the limit.
Sourcepub fn should_compact(&self, reserve_tokens: u64) -> bool
pub fn should_compact(&self, reserve_tokens: u64) -> bool
True if compaction should be triggered. reserve_tokens is how much
headroom callers want to keep for the next model response.
Trait Implementations§
Source§impl Clone for ContextMeter
impl Clone for ContextMeter
Source§fn clone(&self) -> ContextMeter
fn clone(&self) -> ContextMeter
Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreSource§impl Debug for ContextMeter
impl Debug for ContextMeter
Source§impl Default for ContextMeter
impl Default for ContextMeter
Source§fn default() -> ContextMeter
fn default() -> ContextMeter
Returns the “default value” for a type. Read more
Source§impl<'de> Deserialize<'de> for ContextMeter
impl<'de> Deserialize<'de> for ContextMeter
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Deserialize this value from the given Serde deserializer. Read more
Auto Trait Implementations§
impl Freeze for ContextMeter
impl RefUnwindSafe for ContextMeter
impl Send for ContextMeter
impl Sync for ContextMeter
impl Unpin for ContextMeter
impl UnsafeUnpin for ContextMeter
impl UnwindSafe for ContextMeter
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> DeserializeOwned for Twhere
T: for<'de> Deserialize<'de>,
impl<A, B, T> HttpServerConnExec<A, B> for Twhere
B: Body,
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more