Struct RKLLMInferParam 

pub struct RKLLMInferParam {
    pub mode: RKLLMInferMode,
    pub lora_params: Option<String>,
    pub prompt_cache_params: Option<RKLLMPromptCacheParam>,
    pub keep_history: KeepHistory,
}

Structure for defining parameters during inference.

Fields

mode: RKLLMInferMode

Inference mode, such as generating text or getting the last hidden layer.

lora_params: Option<String>

Optional Lora adapter parameters.

prompt_cache_params: Option<RKLLMPromptCacheParam>

Optional prompt cache parameters.

keep_history: KeepHistory

Trait Implementations

impl Clone for RKLLMInferParam

fn clone(&self) -> RKLLMInferParam

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RKLLMInferParam

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for RKLLMInferParam

fn default() -> RKLLMInferParam

Returns the “default value” for a type.