Struct MultiModalIndex 

pub struct MultiModalIndex {
    pub config: MultiModalIndexConfig,
    pub documents: HashMap<String, MultiModalDocument>,
    pub projection_matrices: HashMap<(Modality, Modality), Vec<Vec<f64>>>,
    /* private fields */
}

A unified index for multimodal content supporting cross-modal similarity search.

Overview

MultiModalIndex stores MultiModalDocument entries — each carrying one or more modality embeddings — and provides:

• Same-modality search via same_modality_search
• Cross-modal search via search (projects query into each target modality space when a projection matrix is registered)
• Configurable score fusion via FusionStrategy
• Metadata filtering on result sets

Fields

config: MultiModalIndexConfig

Index configuration (fusion strategy, dims, normalization).

documents: HashMap<String, MultiModalDocument>

Indexed documents keyed by their string ID.

projection_matrices: HashMap<(Modality, Modality), Vec<Vec<f64>>>

Learned cross-modal projection matrices. Key (from, to) maps query space from into document space to.

Implementations

impl MultiModalIndex

pub fn new(config: MultiModalIndexConfig) -> Self

Create a new empty MultiModalIndex with the given configuration.

pub fn add_document(&mut self, doc: MultiModalDocument) -> Result<(), MmiError>

Index a new document.

Returns Err(MmiError::DocumentAlreadyExists) if a document with the same ID is already present.

pub fn remove_document(&mut self, doc_id: &str) -> bool

Remove a document by ID. Returns true if the document existed.

pub fn doc_count(&self) -> usize

Total number of indexed documents.

pub fn modality_coverage(&self) -> HashMap<Modality, usize>

Count the number of documents that contain each modality.

pub fn add_projection( &mut self, from: Modality, to: Modality, matrix: Vec<Vec<f64>>, ) -> Result<(), MmiError>

Register a cross-modal projection matrix from modality from to to.

matrix[i][j] is the weight from input dimension j to output dimension i. So matrix must be cross_modal_dims × from_dims.

Returns Err(MmiError::ProjectionDimsMismatch) if the matrix has zero rows or inconsistent column counts.

pub fn cosine_similarity(a: &[f64], b: &[f64]) -> f64

Compute cosine similarity between two equal-length vectors. Returns 0.0 for zero-norm vectors.

pub fn project(embedding: &[f64], matrix: &[Vec<f64>]) -> Vec<f64>

Project an embedding vector through a matrix.

result[i] = Σ_j matrix[i][j] * embedding[j]

Output length equals the number of rows in matrix.

pub fn l2_normalize(v: &[f64]) -> Vec<f64>

L2-normalize a vector. Returns the zero vector if the norm is zero.

pub fn search(&self, query: &CrossModalQuery) -> Vec<CrossModalResult>

Search the index using the given cross-modal query.

For each document:

1. Compute per-modality cosine similarities (with optional cross-modal projection).
2. Fuse scores using config.fusion_strategy.
3. Filter by query.min_similarity and metadata filters.
4. Return the top-query.top_k results sorted by combined score descending, with 1-based ranks assigned.

pub fn same_modality_search( &self, query: &CrossModalQuery, ) -> Vec<CrossModalResult>

Convenience search restricted to query.query_modality only, with no cross-modal projection.

This is equivalent to calling search with target_modalities = [query_modality] and is more efficient because it skips the projection lookup entirely.

pub fn stats(&self) -> MmiStats

Return aggregate statistics for this index.