Struct CommonParams 

pub struct CommonParams {

    pub n_predict: i32,
    pub n_ctx: i32,
    pub n_batch: i32,
    pub n_ubatch: i32,
    pub n_keep: i32,
    pub n_chunks: i32,
    pub n_parallel: i32,
    pub n_sequences: i32,
    pub grp_attn_n: i32,
    pub grp_attn_w: i32,
    pub n_print: i32,
    pub rope_freq_base: f32,
    pub rope_freq_scale: f32,
    pub yarn_ext_factor: f32,
    pub yarn_attn_factor: f32,
    pub yarn_beta_fast: f32,
    pub yarn_beta_slow: f32,
    pub yarn_orig_ctx: i32,
    pub defrag_thold: f32,
    pub prompt: String,
}

Struct containing common parameters for processing.

See more

https://github.com/ggerganov/llama.cpp/blob/master/common/common.h#L109

Fields

n_predict: i32

New tokens to predict

n_ctx: i32

Context size

n_batch: i32

Logical batch size for prompt processing (must be >=32 to use BLAS)

n_ubatch: i32

Physical batch size for prompt processing (must be >=32 to use BLAS)

n_keep: i32

Number of tokens to keep from initial prompt

n_chunks: i32

Max number of chunks to process (-1 = unlimited)

n_parallel: i32

Number of parallel sequences to decode

n_sequences: i32

Number of sequences to decode

grp_attn_n: i32

Group-attention factor

grp_attn_w: i32

Group-attention width

n_print: i32

Print token count every n tokens (-1 = disabled)

rope_freq_base: f32

RoPE base frequency

rope_freq_scale: f32

RoPE frequency scaling factor

yarn_ext_factor: f32

YaRN extrapolation mix factor

yarn_attn_factor: f32

YaRN magnitude scaling factor

yarn_beta_fast: f32

YaRN low correction dim

yarn_beta_slow: f32

YaRN high correction dim

yarn_orig_ctx: i32

YaRN original context length

defrag_thold: f32

KV cache defragmentation threshold

prompt: String

prompt for the model to consume

Trait Implementations

impl Clone for CommonParams

fn clone(&self) -> CommonParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CommonParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CommonParams

fn default() -> Self

Returns the “default value” for a type.