Crate lumosai_vector_lancedb

Source
Expand description

§LumosAI LanceDB Integration

This crate provides LanceDB integration for LumosAI vector storage, offering high-performance columnar vector database capabilities.

§Features

  • High Performance: Columnar storage optimized for vector operations
  • ACID Transactions: Full transaction support with consistency guarantees
  • Rich Indexing: Multiple index types (IVF, HNSW, LSH)
  • Metadata Filtering: Complex filtering with SQL-like expressions
  • Versioning: Built-in dataset versioning and time travel
  • Compression: Advanced compression for storage efficiency

§Quick Start

use lumosai_vector_lancedb::{LanceDbStorage, LanceDbConfig};
use lumosai_vector_core::traits::VectorStorage;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create LanceDB storage
    let config = LanceDbConfig::new("./lance_data");
    let storage = LanceDbStorage::new(config).await?;
     
    // Create an index
    let index_config = IndexConfig::new("documents", 384)
        .with_metric(SimilarityMetric::Cosine);
    storage.create_index(index_config).await?;
     
    // Insert documents
    let docs = vec![
        Document::new("doc1", "Hello world")
            .with_embedding(vec![0.1; 384])
            .with_metadata("category", "greeting"),
    ];
    storage.upsert_documents("documents", docs).await?;
     
    Ok(())
}

Re-exports§

pub use storage::LanceDbStorage;
pub use config::LanceDbConfig;
pub use config::LanceDbConfigBuilder;
pub use error::LanceDbError;
pub use error::LanceDbResult;

Modules§

config
LanceDB configuration module
conversion
Conversion utilities for LanceDB integration
error
Error types for LanceDB integration
index
Index management for LanceDB
storage
LanceDB storage implementation
utils
Utility functions for LanceDB operations

Structs§

DatabaseStats
Database statistics
Document
Document representation with embedding support
IndexConfig
Index configuration
IndexInfo
Index statistics and information
LanceDbClient
LanceDB client for managing connections and databases
SearchRequest
Search request for querying vectors
SearchResponse
Search response
SearchResult
Search result item

Enums§

FilterCondition
Filter conditions for querying vectors
MetadataValue
Metadata value that can hold various types
SearchQuery
Search query can be either a vector or text
SimilarityMetric
Similarity metrics for vector comparison

Traits§

VectorStorage
Core trait for vector storage backends

Functions§

create_lancedb_storage
Create a new LanceDB storage instance
create_lancedb_storage_with_config
Create a new LanceDB storage instance with configuration

Type Aliases§

DocumentId
Document ID type
Metadata
Metadata type for storing arbitrary key-value pairs
Vector
Vector type alias for f32 vectors