Struct VisionTransformer 

pub struct VisionTransformer {
    pub patch_embed: PatchEmbedding,
    pub cls_token: Tensor,
    pub pos_embed: Tensor,
    pub encoder_blocks: Vec<TransformerEncoderBlock>,
    pub ln_gamma: Tensor,
    pub ln_beta: Tensor,
    pub head_w: Tensor,
    pub head_b: Tensor,
    pub embed_dim: usize,
    pub num_classes: usize,
}

Vision Transformer (ViT) for image classification (inference-mode).

Architecture:

1. Patch embedding
2. Prepend learnable class token
3. Add learnable position embeddings
4. N transformer encoder blocks
5. Layer norm on output
6. Linear classification head on class token

Input: [batch, H, W, C] (NHWC) Output: [batch, num_classes]

Fields

patch_embed: PatchEmbedding

cls_token: Tensor

Class token: [1, embed_dim]

pos_embed: Tensor

Position embeddings: [1, num_patches + 1, embed_dim]

encoder_blocks: Vec<TransformerEncoderBlock>

Transformer encoder blocks

ln_gamma: Tensor

Final layer norm gamma: [embed_dim]

ln_beta: Tensor

Final layer norm beta: [embed_dim]

head_w: Tensor

Classification head weight: [embed_dim, num_classes]

head_b: Tensor

Classification head bias: [num_classes]

embed_dim: usize

num_classes: usize

Implementations

impl VisionTransformer

pub fn new( image_size: usize, patch_size: usize, in_channels: usize, embed_dim: usize, num_heads: usize, num_layers: usize, num_classes: usize, mlp_ratio: f32, ) -> Result<Self, ModelError>

Creates a zero-initialized Vision Transformer.

pub fn forward(&self, input: &Tensor) -> Result<Tensor, ModelError>

Forward inference: [batch, H, W, C] -> [batch, num_classes].