pub struct Sampler { /* private fields */ }Expand description
Sampling strategy for text generation
Implementations§
Source§impl Sampler
impl Sampler
pub fn new() -> Self
Sourcepub fn sample_greedy(&self, logits: &[f32]) -> ModelResult<u32>
pub fn sample_greedy(&self, logits: &[f32]) -> ModelResult<u32>
Greedy sampling - always pick the highest probability token
Sourcepub fn sample_temperature(
&self,
logits: &[f32],
temperature: f32,
) -> ModelResult<u32>
pub fn sample_temperature( &self, logits: &[f32], temperature: f32, ) -> ModelResult<u32>
Temperature sampling - sample from temperature-scaled probability distribution
Sourcepub fn sample_top_p(
&self,
logits: &[f32],
top_p: f32,
temperature: f32,
) -> ModelResult<u32>
pub fn sample_top_p( &self, logits: &[f32], top_p: f32, temperature: f32, ) -> ModelResult<u32>
Top-p (nucleus) sampling
Auto Trait Implementations§
impl Freeze for Sampler
impl RefUnwindSafe for Sampler
impl Send for Sampler
impl Sync for Sampler
impl Unpin for Sampler
impl UnsafeUnpin for Sampler
impl UnwindSafe for Sampler
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<T> ErasedDestructor for Twhere
T: 'static,
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more