Struct VisionEmbeddingModel 

pub struct VisionEmbeddingModel { /* private fields */ }

A loaded Qwen3.5 vision-language checkpoint, ready to pool image (and image+text) embeddings.

See docs/model.md for the general model-loading design; this type follows the same “load once, reuse” shape as NativeEmbeddingService’s wrapped models.

Implementations

impl VisionEmbeddingModel

pub fn new( weights: F16ModelWeights, config: Qwen35Config, vision_weights: Qwen35VisionWeights, tokenizer: BpeTokenizer, ) -> Self

Compose a model from already-loaded components (no I/O).

Use this when the checkpoint spans multiple safetensors shards (not supported by Self::from_directory) or when components are shared across other in-process model instances.

pub fn from_directory(dir: &Path) -> Result<Self>

Load a Qwen3.5 vision-language checkpoint directory: config.json, tokenizer.json, the model.visual.* vision-encoder tensors, and a single-shard decoder checkpoint. The directory must carry a model.safetensors.index.json (or quantize_index.json) manifest naming exactly one decoder shard file — the vision-tensor loader requires one of those manifests and runs before decoder-shard resolution, so a plain model.safetensors alone (with no manifest) is not sufficient. The canonical Qwen3.5-0.8B HF layout (a one-shard index) satisfies this.

Errors

Returns EmbedError::ModelInitialization if config.json is missing or invalid, if the checkpoint has no vision_config, if neither manifest is present, if the decoder weights are sharded across more than one file, or if any component tensor fails to load.

pub fn embed_image( &self, image_bytes: &[u8], prompt: &str, pooling: PoolingStrategy, ) -> Result<Vec<f32>>

Pool an image (plus an optional text prompt) into a single L2-normalized [dimensions()] embedding vector.

Same scaffold and pooling contract as lattice_inference::vision::embed_image_from_bytes_f16 (see that function’s docs for the exact prompt-assembly layout).

Errors

Returns EmbedError::InvalidInput if image_bytes cannot be decoded, its dimensions are not compatible with the checkpoint’s patch/merge geometry, or the assembled request otherwise fails validation (the error message names the offending field). Returns EmbedError::InferenceFailed for every other underlying failure — e.g. the prompt plus image tokens exceeding the checkpoint’s context window.

pub fn embed_text( &self, prompt: &str, pooling: PoolingStrategy, ) -> Result<Vec<f32>>

Pool a text-only prompt through the same decoder + pooling path as Self::embed_image, landing in the same vector space.

Errors

Returns EmbedError::InvalidInput if the prompt is empty or tokenizes to an out-of-vocabulary id. Returns EmbedError::InferenceFailed for every other underlying failure — e.g. the prompt exceeding the checkpoint’s context window.

pub fn dimensions(&self) -> usize

Output embedding dimension (the checkpoint’s decoder hidden size).