Skip to main content

InferenceModel

aha::models::common

Trait InferenceModel 

Source 
pub trait InferenceModel {
    // Required methods
    fn forward_step(
        &mut self,
        input_ids: &Tensor,
        seqlen_offset: usize,
    ) -> Result<Tensor>;
    fn clear_cache(&mut self);
    fn stop_token_ids(&self) -> Vec<u32>;

    // Provided method
    fn forward_initial(
        &mut self,
        input_ids: &Tensor,
        seqlen_offset: usize,
        data: MultiModalData,
    ) -> Result<Tensor> { ... }
}

Required Methods§

Source

fn forward_step( &mut self, input_ids: &Tensor, seqlen_offset: usize, ) -> Result<Tensor>

后续前向传播（自回归步骤）

Source

fn clear_cache(&mut self)

清理 KV cache

Source

fn stop_token_ids(&self) -> Vec<u32>

获取结束 token IDs

Provided Methods§

Source

fn forward_initial( &mut self, input_ids: &Tensor, seqlen_offset: usize, data: MultiModalData, ) -> Result<Tensor>

初始前向传播（考虑多模态输入） 默认实现无特殊数据

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§