Crate lumosai_vector

Expand description

§Lumosai Vector Storage System

A unified, high-performance vector storage system for Lumos.ai that provides a consistent interface across multiple storage backends.

§Features

Unified Interface: Single API for all vector storage backends
Multiple Backends: Memory, Qdrant, PostgreSQL, MongoDB, and more
High Performance: Optimized for speed and scalability
Type Safety: Strong typing with comprehensive error handling
Async/Await: Full async support with tokio
Extensible: Easy to add new storage backends

§Quick Start

use lumosai_vector::prelude::*;

#[tokio::main]
async fn main() -> lumosai_vector_core::Result<()> {
    // Create a memory storage instance
    let storage = lumosai_vector::memory::MemoryVectorStorage::new().await?;

    // Create an index
    let config = IndexConfig::new("documents", 384)
        .with_metric(SimilarityMetric::Cosine);
    storage.create_index(config).await?;

    // Insert documents
    let docs = vec![
        Document::new("doc1", "Hello world")
            .with_embedding(vec![0.1; 384])
            .with_metadata("type", "greeting"),
    ];
    storage.upsert_documents("documents", docs).await?;

    // Search
    let request = SearchRequest::new("documents", vec![0.1; 384])
        .with_top_k(5);
    let results = storage.search(request).await?;

    println!("Found {} results", results.results.len());
    Ok(())
}

§Storage Backends

§Memory Storage

Fast in-memory storage for development and testing:

let storage = MemoryVectorStorage::new().await?;

§Qdrant Storage

High-performance vector database (requires qdrant feature):

use lumosai_vector::qdrant::QdrantVectorStorage;

let storage = QdrantVectorStorage::new("http://localhost:6334").await?;

§PostgreSQL Storage

SQL database with pgvector extension (requires postgres feature):

use lumosai_vector::postgres::PostgresVectorStorage;

let storage = PostgresVectorStorage::new("postgresql://user:pass@localhost/db").await?;

§Architecture

The system is built on a layered architecture:

Core Layer (lumosai-vector-core): Defines traits and types
Implementation Layer: Specific storage backends
Unified API: This crate provides a single entry point

All storage backends implement the VectorStorage trait, ensuring consistent behavior and easy swapping between implementations.

Re-exports§

pub use lumosai_vector_core as core;
pub use lumosai_vector_memory as memory;

Modules§

filter: Default filter evaluator implementation
prelude: Prelude module for convenient imports
similarity: Default similarity calculator implementations
utils: Utility functions for working with vector storage

Structs§

BackendInfo: Backend information
CacheConfig: 缓存配置
CacheEntry: 缓存条目
CacheStats: 缓存统计信息
ConnectionPool: 连接池
ConnectionPoolConfig: 连接池配置
ConnectionPoolStats: 连接池统计信息
Document: Document representation with embedding support
IndexConfig: Index configuration
IndexCreateConfig: Index creation configuration
IndexInfo: Index statistics and information
IndexOptions: Index-specific options
LRUCache: LRU缓存
PerformanceMetrics: 性能指标
PerformanceMonitor: 性能监控器
PooledConnection: 池化连接
SearchConfig: Search configuration
SearchRequest: Search request for querying vectors
SearchResponse: Search response
SearchResult: Search result item
StorageConfigBuilder: Builder for creating storage configurations
VectorSystemConfig: Complete vector storage system configuration

Enums§

EmbeddingConfig: Embedding model configuration
FilterCondition: Filter conditions for querying vectors
MetadataValue: Metadata value that can hold various types
SearchQuery: Search query can be either a vector or text
SimilarityMetric: Similarity metrics for vector comparison
SqliteSynchronous: SQLite synchronous mode
StorageConfig: Storage backend configuration
VectorError: Comprehensive error type for vector storage operations

Traits§

Embeddable: Trait for documents that can be embedded
EmbeddingModel: Trait for embedding models
FilterEvaluator: Trait for filter evaluation
SimilarityCalculator: Trait for vector similarity calculation
VectorStorage: Core trait for vector storage backends

Type Aliases§

DocumentId: Document ID type
Metadata: Metadata type for storing arbitrary key-value pairs
Result: Result type alias for vector operations
Vector: Vector type alias for f32 vectors