Enum OptDiscriminants 

pub enum OptDiscriminants {
    Model,
    ApiKey,
    NThreads,
    MaxTokens,
    MaxContextSize,
    MaxBatchSize,
    StopSequence,
    Stream,
    FrequencyPenalty,
    PresencePenalty,
    TokenBias,
    TopK,
    TopP,
    Temperature,
    RepeatPenalty,
    RepeatPenaltyLastN,
    TfsZ,
    TypicalP,
    Mirostat,
    MirostatTau,
    MirostatEta,
    PenalizeNl,
    NBatch,
    User,
    ModelType,
    NGpuLayers,
    MainGpu,
    TensorSplit,
    VocabOnly,
    UseMmap,
    UseMlock,
}

Auto-generated discriminant enum variants

Variants

Model

The name or path of the model used.

ApiKey

The API key for the model service.

NThreads

The number of threads to use for parallel processing. This is common to all models.

MaxTokens

The maximum number of tokens that the model will generate. This is common to all models.

MaxContextSize

The maximum context size of the model.

MaxBatchSize

The maximum batch size of the model. This is used by llama models.

StopSequence

The sequences that, when encountered, will cause the model to stop generating further tokens. OpenAI models allow up to four stop sequences.

Stream

Whether or not to use streaming mode. This is common to all models.

FrequencyPenalty

The penalty to apply for using frequent tokens. This is used by OpenAI and llama models.

PresencePenalty

The penalty to apply for using novel tokens. This is used by OpenAI and llama models.

TokenBias

A bias to apply to certain tokens during the inference process. This is known as logit bias in OpenAI and is also used in ai-chain-local.

TopK

The maximum number of tokens to consider for each step of generation. This is common to all models, but is not used by OpenAI.

TopP

The cumulative probability threshold for token selection. This is common to all models.

Temperature

The temperature to use for token selection. Higher values result in more random output. This is common to all models.

RepeatPenalty

The penalty to apply for repeated tokens. This is common to all models.

RepeatPenaltyLastN

The number of most recent tokens to consider when applying the repeat penalty. This is common to all models.

TfsZ

The TfsZ parameter for ai-chain-llama.

TypicalP

The TypicalP parameter for ai-chain-llama.

Mirostat

The Mirostat parameter for ai-chain-llama.

MirostatTau

The MirostatTau parameter for ai-chain-llama.

MirostatEta

The MirostatEta parameter for ai-chain-llama.

PenalizeNl

Whether or not to penalize newline characters for ai-chain-llama.

NBatch

The batch size for ai-chain-local.

User

The username for ai-chain-openai.

ModelType

The type of the model.

NGpuLayers

MainGpu

TensorSplit

VocabOnly

UseMmap

UseMlock

Trait Implementations

impl Clone for OptDiscriminants

fn clone(&self) -> OptDiscriminants

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for OptDiscriminants

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'_enum> From<&'_enum Opt> for OptDiscriminants

fn from(val: &'_enum Opt) -> OptDiscriminants

Converts to this type from the input type.

impl From<Opt> for OptDiscriminants

fn from(val: Opt) -> OptDiscriminants

Converts to this type from the input type.

impl PartialEq for OptDiscriminants

fn eq(&self, other: &OptDiscriminants) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for OptDiscriminants

impl Eq for OptDiscriminants

impl StructuralPartialEq for OptDiscriminants