Struct LlamaDrySampler 

pub struct LlamaDrySampler { /* private fields */ }

The stateful DRY repetition sampler.

Build with LlamaDrySampler::new, then apply it to the candidate array before sampling and accept each drawn token so its repetition history stays in sync.

Implementations

impl LlamaDrySampler

pub fn new(model: &LlamaModel, params: &DryParams) -> Result<Self, DryInitError>

Build a DRY sampler for model from params.

Errors

DryInitError::Nul if a sequence breaker contains an interior NUL byte; DryInitError::Init if the C-side initializer returns null.

pub fn apply( &mut self, ctx: &mut LlamaContext<'_>, arr: &mut LlamaTokenDataArray, )

Apply the DRY penalty to arr in place, based on the accepted history.

Call this before sampling / drawing a token.

pub fn accept(&mut self, token: LlamaToken)

Record a drawn token in the sampler’s repetition history.

pub fn reset(&mut self)

Clear the sampler’s repetition history.

pub fn try_clone(&self) -> Option<Self>

Deep-copy this sampler (independent history), or None if the C-side copy fails.

Trait Implementations

impl Debug for LlamaDrySampler

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Drop for LlamaDrySampler

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.