Struct VisionProcessor 

pub struct VisionProcessor {
    pub patch: usize,
    pub pool_k: usize,
    pub image_budget: usize,
    pub video_budget: usize,
    pub video_fps: f64,
    pub max_frames: usize,
    pub overflow_truncate: bool,
}

The processor settings (processor_config.json).

Fields

patch: usize

pool_k: usize

image_budget: usize

soft tokens per image (image_processor.max_soft_tokens)

video_budget: usize

soft tokens per video frame (video_processor.max_soft_tokens)

video_fps: f64

frames sampled per second of video

max_frames: usize

sampled frames are cut to this many, evenly (overflow_strategy)

overflow_truncate: bool

uniform (default) or truncate

Implementations

impl VisionProcessor

pub fn from_config(cfg: &Value) -> Self

From processor_config.json (missing fields keep the defaults).

pub fn check_budget(budget: usize) -> Result<usize, String>

budget checked against the five the model was trained for.

impl VisionProcessor

pub fn resized_size( &self, w: usize, h: usize, budget: usize, ) -> Result<(usize, usize), String>

The size an image of w × h is resized to at budget soft tokens.

pub fn prepare_image( &self, img: &RgbFrame, budget: usize, ) -> Result<VisionInput, String>

Resize (when the size changes) and patchify one image.

pub fn prepare_frames( &self, frames: &[RgbFrame], budget: usize, ) -> Result<Vec<VisionInput>, String>

Video frames (already sampled, one size): each resized to the size frame 0 takes at budget, then patchified.

pub fn sample_indices(&self, total: usize, src_fps: Option<f64>) -> Vec<usize>

The frames of a total-frame video at src_fps the processor keeps: int(i · src_fps / fps) for i < max(1, int(total / src_fps · fps)), then cut to max_frames (evenly, np.linspace(..., dtype=int), or the first ones under truncate). src_fps = None (frames with no rate) keeps every frame and applies only the cap.

Trait Implementations

impl Clone for VisionProcessor

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for VisionProcessor

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for VisionProcessor

fn default() -> Self

Returns the “default value” for a type.

impl PartialEq for VisionProcessor

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for VisionProcessor