pub struct Sampler { /* private fields */ }Expand description
Token sampler.
Owns a reusable probs_buf that is grown on first use and then reused across
all subsequent sample() calls, eliminating the ~1.8 MB per-call heap
allocation that a fresh Vec would require for a 151 936-token vocabulary.
Implementations§
Source§impl Sampler
impl Sampler
Sourcepub fn new(params: SamplingParams, seed: u64) -> Self
pub fn new(params: SamplingParams, seed: u64) -> Self
Create a new sampler with the given parameters and seed.
Sourcepub fn sample(&mut self, logits: &[f32]) -> RuntimeResult<u32>
pub fn sample(&mut self, logits: &[f32]) -> RuntimeResult<u32>
Sample a token index from logits.
Sourcepub fn params(&self) -> &SamplingParams
pub fn params(&self) -> &SamplingParams
Get current parameters.
Trait Implementations§
Auto Trait Implementations§
impl Freeze for Sampler
impl RefUnwindSafe for Sampler
impl Send for Sampler
impl Sync for Sampler
impl Unpin for Sampler
impl UnsafeUnpin for Sampler
impl UnwindSafe for Sampler
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more