Struct PreprocessedImages 

pub struct PreprocessedImages {
    pub pixel_values: ArrayD<f32>,
    pub num_img_tokens: Vec<usize>,
    pub image_sizes: Vec<(u32, u32)>,
    pub model_specific: HashMap<String, ModelSpecificValue>,
}

Preprocessed images ready for model consumption.

This struct contains all the outputs needed by the SGLang scheduler to construct MultimodalInputs for the model.

Fields

pixel_values: ArrayD<f32>

Pixel values as a dynamic-dimensional float32 tensor.

This is the primary input to the vision encoder. Shape varies by model:

• Standard: [B, C, H, W] (4D)
• Phi3-Vision: [B, num_crops+1, C, H, W] (5D)

num_img_tokens: Vec<usize>

Number of image tokens per image in the batch.

Used to expand placeholder tokens in the text input. For example, LLaVA with 336x336 and patch_size=14 produces 576 tokens.

image_sizes: Vec<(u32, u32)>

Original image sizes as (width, height) before preprocessing.

Some models need this for proper attention masking or position encoding.

model_specific: HashMap<String, ModelSpecificValue>

Model-specific auxiliary outputs.

Examples:

• Qwen-VL: image_grid_thw for rotary position encoding
• LLaMA-Vision: aspect_ratio_ids, aspect_ratio_mask
• Phi3-Vision: num_img_tokens per crop

Implementations

impl PreprocessedImages

pub fn new( pixel_values: Array4<f32>, num_img_tokens: Vec<usize>, image_sizes: Vec<(u32, u32)>, ) -> Self

Create a new PreprocessedImages with required fields (4D pixel values).

pub fn new_dynamic( pixel_values: ArrayD<f32>, num_img_tokens: Vec<usize>, image_sizes: Vec<(u32, u32)>, ) -> Self

Create a new PreprocessedImages with dynamic-dimensional pixel values.

Use this for models like Phi3-Vision that have 5D tensors.

pub fn with_extra( self, key: impl Into<String>, value: ModelSpecificValue, ) -> Self

Add a model-specific value.

pub fn batch_size(&self) -> usize

Get the batch size.

pub fn channels(&self) -> Result<usize, TransformError>

Get the number of channels.

For 4D tensors [B, C, H, W], returns shape[1]. For 5D tensors [B, N, C, H, W] (Phi3-Vision), returns shape[2].

Errors

Returns TransformError::InvalidShape if pixel_values is not 4D or 5D.

pub fn height(&self) -> Result<usize, TransformError>

Get the height of processed images.

For 4D tensors [B, C, H, W], returns shape[2]. For 5D tensors [B, N, C, H, W] (Phi3-Vision), returns shape[3].

Errors

Returns TransformError::InvalidShape if pixel_values is not 4D or 5D.

pub fn width(&self) -> Result<usize, TransformError>

Get the width of processed images.

For 4D tensors [B, C, H, W], returns shape[3]. For 5D tensors [B, N, C, H, W] (Phi3-Vision), returns shape[4].

Errors

Returns TransformError::InvalidShape if pixel_values is not 4D or 5D.

pub fn ndim(&self) -> usize

Get the number of dimensions of pixel_values.

pub fn total_tokens(&self) -> usize

Get total number of image tokens across all images.

pub fn pixel_values_flat(&self) -> Cow<'_, [f32]>

Get pixel values as a flat f32 slice without copying if possible.

pub fn pixel_values_shape(&self) -> Vec<usize>

Get the shape of pixel values as a vector.

pub fn num_images(&self) -> usize

Number of images in this batch.

pub fn batched_keys(layouts: &HashMap<String, FieldLayout>) -> Vec<String>

Extract batched tensor keys from explicit field layout declarations.

pub fn flat_keys( layouts: &HashMap<String, FieldLayout>, ) -> HashMap<String, String>

Extract flat-slicing tensor keys from explicit field layout declarations.

Returns a map of tensor name → sizes tensor name.

Trait Implementations

impl Clone for PreprocessedImages

fn clone(&self) -> PreprocessedImages

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PreprocessedImages

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.