Struct PrefillOutput 

pub struct PrefillOutput {
    pub logits: TensorRef,
    pub kv_cache: Arc<dyn KvCacheHandle>,
    pub recurrent_state: Option<Arc<dyn RecurrentStateHandle>>,
    pub hidden_states: Option<Vec<TensorRef>>,
    pub attention_weights: Option<Vec<TensorRef>>,
}

Output from prefill phase

Fields

logits: TensorRef

Logits for all positions [batch_size, sequence_length, vocab_size]

kv_cache: Arc<dyn KvCacheHandle>

KV cache handle populated with prompt states

recurrent_state: Option<Arc<dyn RecurrentStateHandle>>

Recurrent-state handle populated with prompt state, when used.

hidden_states: Option<Vec<TensorRef>>

Hidden states at each layer (optional, for analysis)

attention_weights: Option<Vec<TensorRef>>

Attention weights (optional, for analysis)

Implementations

impl PrefillOutput

pub fn new(logits: TensorRef, kv_cache: Arc<dyn KvCacheHandle>) -> Self

Create new prefill output

pub fn with_recurrent_state( self, recurrent_state: Arc<dyn RecurrentStateHandle>, ) -> Self

Attach updated recurrent state to the prefill output.

pub fn last_token_logits(&self) -> Result<TensorRef>

Get logits for last position (for next token generation)

Trait Implementations

impl Clone for PrefillOutput

fn clone(&self) -> PrefillOutput

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PrefillOutput

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.