pub struct Plan {
pub prompt_tokens: usize,
pub max_new: usize,
pub cached_prefix: usize,
pub prompt_ids: Vec<u32>,
pub params: GenerateParams,
}Expand description
One accepted request, ready to run.
Everything that could have been refused already has been, which is the
point: holding a Plan means the response head can be committed.
Fields§
§prompt_tokens: usizeThe whole prompt, cached prefix included. This is the number the
client is billed in usage.prompt_tokens, not the suffix actually
prefilled — a cache hit is this server’s business, not a discount the
client can observe and come to depend on.
max_new: usizeReply cap for this turn, after clamping to the context.
cached_prefix: usizeLeading positions the KV cache already holds and this turn reuses.
prompt_ids: Vec<u32>The full sanitized prompt id sequence.
params: GenerateParamsSampling dials, merged from the request over the checkpoint defaults.
Implementations§
Trait Implementations§
impl StructuralPartialEq for Plan
Auto Trait Implementations§
impl Freeze for Plan
impl RefUnwindSafe for Plan
impl Send for Plan
impl Sync for Plan
impl Unpin for Plan
impl UnsafeUnpin for Plan
impl UnwindSafe for Plan
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more