Skip to main content

Sampler

eredu_runtime::generation

Trait Sampler 

Source 
pub trait Sampler<B: SamplingBackend> {
    // Required method
    fn sample(
        &mut self,
        logits: &B::Logits,
        temperature: f32,
        random: Option<&mut B::RandomState>,
        context: &B::Context,
    ) -> Result<B::Token, B::Error>;

    // Provided method
    fn uses_checkpoint_defaults(&self) -> bool { ... }
}
Expand description

Strategy for choosing a token from model logits.

Required Methods§

Source

fn sample( &mut self, logits: &B::Logits, temperature: f32, random: Option<&mut B::RandomState>, context: &B::Context, ) -> Result<B::Token, B::Error>

Selects one token from raw model logits.

Provided Methods§

Source

fn uses_checkpoint_defaults(&self) -> bool

Whether loaded checkpoint defaults should wrap this policy.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§