Struct VisualInput 

pub struct VisualInput {
    pub kind: VisualKind,
    pub rows: Vec<f32>,
    pub grid_t: usize,
    pub grid_h: usize,
    pub grid_w: usize,
    pub patch_dim: usize,
    pub merge_size: usize,
    pub timestamps: Vec<f32>,
    pub frame_indices: Vec<usize>,
    pub resized: (usize, usize),
}

One preprocessed image or video: the patch rows fed to the ViT, the patch grid, and (video) the per-frame timestamps in seconds.

Fields

kind: VisualKind

rows: Vec<f32>

[grid_t · grid_h · grid_w, 3 · T · P · P], merge-block row order.

grid_t: usize

grid_h: usize

grid_w: usize

patch_dim: usize

merge_size: usize

timestamps: Vec<f32>

Video: one timestamp per frame after even padding (2 · grid_t).

frame_indices: Vec<usize>

Video: the source frame index of each entry of timestamps.

resized: (usize, usize)

The resized frame size (height, width).

Implementations

impl VisualInput

pub fn patches(&self) -> usize

ViT rows (patches).

pub fn tokens_per_step(&self) -> usize

LLM placeholder tokens for one temporal step.

pub fn tokens(&self) -> usize

LLM placeholder tokens for the whole item.

pub fn timestamp_labels(&self) -> Vec<String>

The “MM:SS” label of every temporal step (video only): the timestamp of the first frame of each pair.

Trait Implementations

impl Clone for VisualInput

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for VisualInput

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.