Struct LlamaCppEngineConfig 

pub struct LlamaCppEngineConfig {
    pub binary: String,
    pub mode: LlamaCppMode,
    pub host: String,
    pub port: u16,
    pub n_ctx: u32,
    pub n_threads: u32,
    pub n_gpu_layers: i32,
    pub extra_args: Vec<String>,
}

Fields

binary: String

Path or PATH-name of the python interpreter (mode = “python_server”) or the standalone server binary (mode = “binary”). Default: python.

mode: LlamaCppMode

“python_server” → invoked as <binary> -m llama_cpp.server …. “binary” → invoked as <binary> --model … --host … --port ….

host: String

port: u16

n_ctx: u32

Context window. Mapped to --n_ctx.

n_threads: u32

CPU thread count. Mapped to --n_threads.

n_gpu_layers: i32

GPU layer offload count. -1 = “offload everything to GPU”, 0 = “CPU only”. Mapped to --n_gpu_layers.

extra_args: Vec<String>

Arguments appended after the auto-rendered base flags.

Trait Implementations

impl Clone for LlamaCppEngineConfig

fn clone(&self) -> LlamaCppEngineConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LlamaCppEngineConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for LlamaCppEngineConfig

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for LlamaCppEngineConfig

where LlamaCppEngineConfig: Default,

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for LlamaCppEngineConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.