Module multi_modal_search 

This module provides a unified interface for multi-modal similarity search, supporting queries across different modalities (text, image, audio, video) with automatic alignment and fusion in a joint embedding space.

Features

• Multi-modal queries: Search with text, images, audio, or combinations
• Cross-modal retrieval: Find images with text queries, or vice versa
• Hybrid fusion: Combine results from multiple modalities intelligently
• Production-ready encoders: Real implementations for all modalities
• SPARQL integration: Query multi-modal RDF data with SPARQL

Example

use oxirs_vec::multi_modal_search::{MultiModalSearchEngine, MultiModalQuery, QueryModality};

// Create search engine
let engine = MultiModalSearchEngine::new_default()?;

// Text query
let query = MultiModalQuery::text("show me images of cats");
let results = engine.search(&query, 10)?;

// Image query
let image_data = std::fs::read("cat.jpg")?;
let query = MultiModalQuery::image(image_data);
let results = engine.search(&query, 10)?;

// Hybrid query (text + image)
let query = MultiModalQuery::hybrid(vec![
    QueryModality::Text("cute kitten".to_string()),
    QueryModality::Image(image_data),
]);
let results = engine.search(&query, 10)?;

Structs

MultiModalConfig

Configuration for multi-modal search

MultiModalQuery

A multi-modal query combining one or more modalities

MultiModalSearchEngine

Multi-modal search engine that handles queries across different modalities

MultiModalStatistics

Statistics about the multi-modal search engine

ProductionAudioEncoder

Production audio encoder using MFCC and spectral features

ProductionGraphEncoder

Production graph encoder for knowledge graphs

ProductionImageEncoder

Production image encoder using ResNet-style features

ProductionTextEncoder

Production text encoder using TF-IDF and sentence embeddings

ProductionVideoEncoder

Production video encoder using temporal features

QueryFilter

Filter for query results

SearchResult

Search result from multi-modal query

Enums

FilterOperator

QueryModality

Query modality with associated data

SearchStrategy

Search strategy for multi-modal queries