Struct AttentionPlan 

#[non_exhaustive]
pub struct AttentionPlan {
    pub kernel: AttentionKernel,
    pub scratch_bytes: usize,
    pub max_intermediate_bytes: usize,
    pub capped: bool,
}

How an attention call runs and the memory it holds (see plan_attention).

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

kernel: AttentionKernel

The kernel family.

scratch_bytes: usize

Bytes held beyond the inputs and the output while the call runs (for one block when blocked; see the # Memory section of scaled_dot_product_attention).

max_intermediate_bytes: usize

The largest single score-sized intermediate: what max_score_bytes bounds.

capped: bool

Whether the cap changed the choice.

Trait Implementations

impl Clone for AttentionPlan

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for AttentionPlan

impl Debug for AttentionPlan

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for AttentionPlan

impl PartialEq for AttentionPlan

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for AttentionPlan