Struct LlamaTokenDataArray 

pub struct LlamaTokenDataArray {
    pub data: Vec<LlamaTokenData>,
    pub selected: Option<usize>,
    pub sorted: bool,
}

Fields

data: Vec<LlamaTokenData>

selected: Option<usize>

sorted: bool

Implementations

impl LlamaTokenDataArray

pub const fn new(data: Vec<LlamaTokenData>, sorted: bool) -> Self

pub fn from_iter<TIterator>(data: TIterator, sorted: bool) -> Self

where TIterator: IntoIterator<Item = LlamaTokenData>,

pub fn selected_token(&self) -> Option<LlamaToken>

impl LlamaTokenDataArray

pub unsafe fn modify_as_c_llama_token_data_array<TResult>( &mut self, modify: impl FnOnce(&mut llama_token_data_array) -> TResult, ) -> TResult

Panics

Panics if some of the safety conditions are not met. (we cannot check all of them at runtime so breaking them is UB)

Safety

The returned array formed by the data pointer and the length must entirely consist of initialized token data and the length must be less than the capacity of this array’s data buffer. If the data is not sorted, sorted must be false.

pub fn apply_sampler(&mut self, sampler: &LlamaSampler)

Panics

Panics if the vendored sampler throws a C++ exception. llama_sampler_apply is documented to be a pure logit transform and is not expected to throw; if it does the failure is propagated as a panic per the crash-fast invariant.

pub fn with_sampler(self, sampler: &mut LlamaSampler) -> Self

pub fn sample_token( &mut self, seed: u32, ) -> Result<LlamaToken, TokenSamplingError>

Errors

Returns TokenSamplingError::NoTokenSelected if the sampler fails to select a token.

pub fn sample_token_greedy(&mut self) -> Result<LlamaToken, TokenSamplingError>

Errors

Returns TokenSamplingError::NoTokenSelected if the sampler fails to select a token.

Trait Implementations

impl Clone for LlamaTokenDataArray

fn clone(&self) -> LlamaTokenDataArray

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LlamaTokenDataArray

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for LlamaTokenDataArray

fn eq(&self, other: &LlamaTokenDataArray) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for LlamaTokenDataArray