Struct SemanticIndexer 

pub struct SemanticIndexer { /* private fields */ }

Implementations

impl SemanticIndexer

pub fn new(embedder_type: &str, data_dir: Option<&Path>) -> Result<Self>

pub fn with_batch_size(self, batch_size: usize) -> Result<Self>

pub fn batch_size(&self) -> usize

pub fn embedder_id(&self) -> &str

pub fn embedder_dimension(&self) -> usize

pub fn embed_messages( &self, messages: &[EmbeddingInput], ) -> Result<Vec<EmbeddedMessage>>

pub fn embed_messages_with_sink( &self, messages: &[EmbeddingInput], sink: &SemanticProgressSink, ) -> Result<Vec<EmbeddedMessage>>

Variant of [embed_messages] that emits embed_batch_* events into the given JSONL sink. The sink is silent unless CASS_SEMANTIC_PROGRESS_JSONL is set, so this path is safe to take in production.

pub fn build_and_save_index<I>( &self, embedded_messages: I, data_dir: &Path, ) -> Result<FsVectorIndex>

where I: IntoIterator<Item = EmbeddedMessage>,

pub fn build_and_save_index_shards<I>( &self, embedded_messages: I, data_dir: &Path, plan: SemanticShardBuildPlan, ) -> Result<SemanticShardBuildOutcome>

where I: IntoIterator<Item = EmbeddedMessage>,

pub fn append_to_index( &self, embedded_messages: impl IntoIterator<Item = EmbeddedMessage>, data_dir: &Path, ) -> Result<usize>

Append new embeddings to an existing FSVI index via the WAL.

Used for incremental semantic indexing in watch mode. Opens the existing index, appends a batch of new embeddings, and compacts if the WAL has grown large enough.

Returns the number of entries appended.

pub fn run_backfill_batch( &self, messages: &[EmbeddingInput], data_dir: &Path, manifest: &mut SemanticManifest, plan: SemanticBackfillBatchPlan, ) -> Result<SemanticBackfillBatchOutcome>

pub fn run_backfill_batch_with_sink( &self, messages: &[EmbeddingInput], data_dir: &Path, manifest: &mut SemanticManifest, plan: SemanticBackfillBatchPlan, last_message_id: Option<i64>, sink: &SemanticProgressSink, ) -> Result<SemanticBackfillBatchOutcome>

Variant of [run_backfill_batch] that emits semantic progress events to the given JSONL sink and persists last_message_id into the resumable checkpoint when supplied. The sink is silent unless CASS_SEMANTIC_PROGRESS_JSONL is set, so this path is safe to take in production.

pub fn run_backfill_from_storage( &self, storage: &FrankenStorage, data_dir: &Path, manifest: &mut SemanticManifest, plan: SemanticBackfillStoragePlan, ) -> Result<SemanticBackfillBatchOutcome>

pub fn run_backfill_from_storage_with_sink( &self, storage: &FrankenStorage, data_dir: &Path, manifest: &mut SemanticManifest, plan: SemanticBackfillStoragePlan, sink: &SemanticProgressSink, ) -> Result<SemanticBackfillBatchOutcome>

Variant of [run_backfill_from_storage] that emits semantic progress events to a JSONL sink and persists last_message_id in the resumable checkpoint. The sink is silent unless CASS_SEMANTIC_PROGRESS_JSONL is set.

pub fn run_capped_backfill_from_storage_with_sink( &self, storage: &FrankenStorage, data_dir: &Path, manifest: &mut SemanticManifest, plan: SemanticBackfillStoragePlan, sink: &SemanticProgressSink, ) -> Result<SemanticBackfillBatchOutcome>

Variant of [run_backfill_from_storage_with_sink] for CLI backfill runs. It applies operator checkpoint caps from CASS_SEMANTIC_MAX_MESSAGES_PER_CHECKPOINT and CASS_SEMANTIC_MAX_BYTES_PER_CHECKPOINT while keeping each selected conversation whole, so message-cursor resume cannot strand the tail of a partially selected conversation.

pub fn build_hnsw_index( &self, vector_index: &FsVectorIndex, data_dir: &Path, m: Option<usize>, ef_construction: Option<usize>, ) -> Result<PathBuf>

Build and save an HNSW index for approximate nearest neighbor search.

This creates an HNSW graph structure from the existing VectorIndex, enabling O(log n) approximate search with the --approximate flag.

Arguments

• vector_index - The VectorIndex to build HNSW from
• data_dir - Directory to save the HNSW index
• m - Max connections per node (default: 16)
• ef_construction - Search width during build (default: 200)

Returns

Path to the saved HNSW index file