Struct LlamaTokenDataArray 

pub struct LlamaTokenDataArray {
    pub data: Vec<LlamaTokenData>,
    pub selected: Option<usize>,
    pub sorted: bool,
}

Fields

data: Vec<LlamaTokenData>

selected: Option<usize>

sorted: bool

Implementations

impl LlamaTokenDataArray

pub const fn new(data: Vec<LlamaTokenData>, sorted: bool) -> Self

pub fn from_iter<TIterator>(data: TIterator, sorted: bool) -> Self

where TIterator: IntoIterator<Item = LlamaTokenData>,

pub fn selected_token(&self) -> Option<LlamaToken>

impl LlamaTokenDataArray

pub unsafe fn modify_as_c_llama_token_data_array<TResult>( &mut self, modify: impl FnOnce(&mut llama_token_data_array) -> TResult, ) -> TResult

Panics

Panics if some of the safety conditions are not met. (we cannot check all of them at runtime so breaking them is UB)

Safety

The returned array formed by the data pointer and the length must entirely consist of initialized token data and the length must be less than the capacity of this array’s data buffer. If the data is not sorted, sorted must be false.

pub fn apply_sampler( &mut self, sampler: &LlamaSampler, ) -> Result<(), SamplerApplyError>

Errors

Returns SamplerApplyError if the sampler pointer is null, the vendored sampler runs out of memory, or it throws a C++ exception while applying.

pub fn with_sampler( self, sampler: &mut LlamaSampler, ) -> Result<Self, SamplerApplyError>

Errors

Returns SamplerApplyError if applying the sampler fails.

pub fn sample_token( &mut self, seed: u32, ) -> Result<LlamaToken, TokenSamplingError>

Errors

Returns TokenSamplingError::SamplerApply if applying the sampler fails, or TokenSamplingError::NoTokenSelected if the sampler fails to select a token.

pub fn sample_token_greedy(&mut self) -> Result<LlamaToken, TokenSamplingError>

Errors

Returns TokenSamplingError::SamplerApply if applying the sampler fails, or TokenSamplingError::NoTokenSelected if the sampler fails to select a token.

Trait Implementations

impl Clone for LlamaTokenDataArray

fn clone(&self) -> LlamaTokenDataArray

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LlamaTokenDataArray

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for LlamaTokenDataArray

fn eq(&self, other: &LlamaTokenDataArray) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for LlamaTokenDataArray