Struct ModelInputsConfig 

pub struct ModelInputsConfig<'a> {
    pub input_ids: &'a str,
    pub logits: &'a str,
    pub attention_mask: &'a str,
    pub cache_position: &'a str,
    pub kv_caches: Vec<KVCachePair<'a>>,
    pub position_ids: &'a str,
    pub use_cache_flag: &'a str,
}

Specifies the names of model inputs and outputs.

The Default impl for this struct returns an instance whose names follow the configuration of Hugging Face’s Optimum tool.

Any inputs that are not present in the model are ignored.

Fields

input_ids: &'a str

Model input that contains the token IDs of the prompt and output generated so far.

logits: &'a str

Model output that contains logits.

attention_mask: &'a str

Model input that contains an attention mask.

cache_position: &'a str

Model input that contains KV cache positions for each position.

This input does not have a batch dimension.

kv_caches: Vec<KVCachePair<'a>>

Patterns for inputs and outputs used for key-value caches.

position_ids: &'a str

Model input that contains position IDs for each position.

This input has a batch dimension.

use_cache_flag: &'a str

Boolean input that is set to false on the first run and true on subsequent runs.

Trait Implementations

impl Default for ModelInputsConfig<'_>

fn default() -> Self

Return default model input names.

These are based on Hugging Face’s Optimum model exporter.