Crate combs_media 

combs-media

Media preprocessing — the traits-first basic block for non-text modalities. Vision (SigLIP-style image preprocessing) and audio (WAV → 16 kHz → Whisper log-mel) are both plain host-side code (no GPU dependency) producing normalized tensors the runtime hands to the model’s embedding entry points.

Structs

LogMel

Whisper log-mel extractor. Owns the FFT plan, window, and filterbank; build once, reuse per utterance.

PixelBatch

A preprocessed image: planar CHW f32, normalized, ready for Tensor::from_data(TensorData::new(data, [1, c, h, w])).

SiglipPreprocessor

SigLIP / Idefics3 single-image preprocessing (SmolVLM-256M/500M): RGB → resize longest edge to image_size (aspect preserved, bilinear) → pad to a square with 0.5 → rescale 1/255 → normalize mean/std 0.5. (Padding at the normalization mean maps to 0 after normalization.)

Enums

MediaError

Errors produced while decoding or preprocessing media.

Constants

CHUNK_FRAMES

Frames one 30 s window produces (CHUNK_SAMPLES / HOP_LENGTH).

CHUNK_SAMPLES

Samples in one 30 s model window.

HOP_LENGTH

STFT hop (10 ms at 16 kHz).

N_FFT

STFT window length (25 ms at 16 kHz).

N_MELS

Mel bins.

SAMPLE_RATE

Sample rate every speech model input is resampled to.

Traits

ImagePreprocessor

Turns encoded image bytes (PNG/JPEG/WebP) into normalized pixel batches.

Functions

decode_wav

Decodes a WAV payload to mono f32 samples in [-1, 1].

pad_or_trim

Zero-pads or truncates to exactly len samples (the 30 s model window).

resample_linear

Linear-interpolation resampler. Documented v1 simplification: no low-pass filter, which is adequate for speech into a 16 kHz pipeline; a windowed-sinc resampler can replace this without changing callers.

Type Aliases

Result

Convenient result alias for this crate.