Module preprocess 

Turning a rasterized page into the tensor a vision tower expects.

This is step 1 of the vision path (see super): everything between “we have pixels” and “the patch-embedding matmul can run”. It is deliberately pure arithmetic over pixel buffers — no model, no weights, no GGUF — so it is fully testable today, before any vision-capable runtime exists.

The pipeline, and why each step is where it is

RGB8 pixels ──resize──▶ square canvas ──normalize──▶ CHW f32 ──patchify──▶ [n_patches, patch_dim]

• Resize to a fixed square. Vision towers have a fixed input geometry baked into their learned position embeddings — there is one embedding per patch slot, so the patch grid cannot vary at inference without also interpolating those embeddings. Fixing the canvas keeps that whole problem out of scope.
• Normalize to the mean/std the tower was trained with. Getting these constants wrong does not error — it silently shifts every activation and quietly degrades the output, which is why they are named constants with their provenance recorded rather than magic numbers inline.
• Patchify into non-overlapping patch × patch tiles, each flattened to a row. That row-major matrix is exactly the operand a patch-embedding matmul wants.

Why there is no conv2d here

ViT patch embedding is conventionally written as a conv2d with kernel = stride = patch_size. With non-overlapping patches that convolution is identical to flattening each tile and doing one matmul against the reshaped kernel — every input element lands in exactly one output window, so there is no sliding-window reuse for a convolution to exploit. kopitiam-tensor has matmul but no conv2d, and this identity is why that costs us nothing. (It stops being true for overlapping patches; a tower using stride < kernel would genuinely need a convolution.)

Channel order: CHW, not HWC

Pixels arrive interleaved (RGBRGBRGB…, i.e. HWC) and leave planar (RRR…GGG…BBB…, i.e. CHW), because that is the layout the tower’s weights are stored against. Mixing these up is the classic silent bug in this area: the shapes match, nothing errors, and the model sees colour noise.

Structs

PreprocessConfig

How an image is prepared for a specific vision tower.

Rgb8

An 8-bit RGB image, row-major and interleaved (RGBRGB…), as a rasterizer hands it over.

Constants

SIGLIP_MEAN

Per-channel mean used by SigLIP-family vision towers (the encoder SmolVLM uses), in the 0..1 range, RGB order.

SIGLIP_STD

Per-channel standard deviation matching SIGLIP_MEAN.

Functions

gray_to_rgb8

Convenience for the OCR/routing path: a grayscale page raster promoted to RGB by replicating the single channel.

preprocess

Full preprocessing: resize, normalize, and patchify into the [n_patches, patch_dim] matrix a patch-embedding matmul consumes.

resize_square

Resizes to a square size × size canvas by nearest-neighbour sampling.