Struct MultiViewSpatialTransformer 

pub struct MultiViewSpatialTransformer { /* private fields */ }

A spatial transformer that includes multi-view attention in every block. Replaces the standard SpatialTransformer from SD 2.1.

Implementations

impl MultiViewSpatialTransformer

pub fn new( vs: VarBuilder<'_>, in_channels: usize, n_heads: usize, d_head: usize, depth: usize, context_dim: usize, ip_dim: usize, num_views: usize, num_groups: usize, use_linear_projection: bool, ) -> Result<Self>

Create a new multi-view spatial transformer with standard attention.

pub fn new_with_flash( vs: VarBuilder<'_>, in_channels: usize, n_heads: usize, d_head: usize, depth: usize, context_dim: usize, ip_dim: usize, num_views: usize, num_groups: usize, use_linear_projection: bool, use_flash_attention: bool, flash_block_size: usize, ) -> Result<Self>

Create a new multi-view spatial transformer with optional flash attention.

Arguments

• vs - Variable builder for weight initialization
• in_channels - Number of input channels
• n_heads - Number of attention heads
• d_head - Dimension per head
• depth - Number of transformer blocks
• context_dim - Text cross-attention context dimension
• ip_dim - IP-adapter context dimension
• num_views - Number of views for cross-view attention
• num_groups - Number of groups for group normalization
• use_linear_projection - Whether to use linear projection
• use_flash_attention - Whether to use flash attention
• flash_block_size - Block size for flash attention tiling

pub fn forward( &self, xs: &Tensor, context: Option<&Tensor>, ip_tokens: Option<&Tensor>, ) -> Result<Tensor>

Forward pass.

• xs: (B*V, C, H, W) feature map
• context: optional text cross-attention context
• ip_tokens: optional IP-adapter tokens

Trait Implementations

impl Debug for MultiViewSpatialTransformer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.