Struct llm_base::TokenBias

pub struct TokenBias(_);

A list of tokens to bias during the process of inferencing.

When a biased token is encountered, the bias will be used instead of the inferred logit during the sampling process.

This can be used to disable the generation of responses with specific tokens by setting their corresponding bias to -1.0.

Implementations

impl TokenBias

pub fn new(v: Vec<(TokenId, f32)>) -> Self

Create a TokenBias from an existing Vec.

pub fn get(&self, tid: TokenId) -> Option<f32>

Retrieves the bias for a given token, if available.

Trait Implementations

impl Clone for TokenBias

fn clone(&self) -> TokenBias

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenBias

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for TokenBias

fn default() -> TokenBias

Returns the “default value” for a type.

impl Display for TokenBias

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl FromStr for TokenBias

fn from_str(s: &str) -> Result<Self, Self::Err>

A comma separated list of token biases. The list should be in the format “TID=BIAS,TID=BIAS” where TID is an integer token ID and BIAS is a floating point number. For example, “1=-1.0,2=-1.0” sets the bias for token IDs 1 (start of document) and 2 (end of document) to -1.0 which effectively disables the model from generating responses containing those token IDs.

type Err = String

The associated error which can be returned from parsing.

impl PartialEq<TokenBias> for TokenBias

fn eq(&self, other: &TokenBias) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for TokenBias