Expand description
Host preprocessor for vision input (lane/vision): bytes -> ViT patch rows.
Qwen2VLImageProcessorFast semantics: smart_resize to multiples of
factor = patch(16) * merge(2) = 32 with the pixel-area budget, rescale 1/255,
normalize mean/std 0.5 -> [-1, 1], patchify to [ghgw, 321616 = 1536] rows in
row-major grid order with (c, t, ph, pw) inner order — the flatten of the conv
weight [1152, 3, 2, 16, 16], so VisionTower::forward consumes rows directly.
Images duplicate their frame across temporal_patch 2; videos fill the pair with
consecutive sampled frames.
Resize filter: CatmullRom (Keys bicubic a=-0.5, PIL-BICUBIC family). The HF fast processor runs torch bicubic (a=-0.75) antialias — close but not bit-equal; the merger-cosine parity gate arbitrates whether the difference matters.
Structs§
Constants§
- MAX_
PIXELS - MIN_
PIXELS - Area budget (pixels) from preprocessor_config: shortest_edge / longest_edge.
Functions§
- decode_
data_ uri - Parse a base64 data URI into raw bytes (any
data:*;base64,media type). - prep_
data_ uri data:image/...;base64,<payload>-> patch rows.- prep_
image_ bytes - Image bytes (png/jpeg/webp/gif/bmp) -> patch rows. The single frame fills both temporal slots (HF: images are tiled to temporal_patch_size).
- smart_
resize - smart_resize (HF): round each side to a multiple of 32 preserving aspect ratio, then scale into the [MIN_PIXELS, MAX_PIXELS] area budget.