Struct ContextStore 

pub struct ContextStore {
    pub compaction_config: CompactionConfig,
    /* private fields */
}

Persistent Lance-backed context store.

Fields

compaction_config: CompactionConfig

Implementations

impl ContextStore

pub async fn evaluate( &self, query_set: &EvalQuerySet, config: &EvalConfig, ) -> LanceResult<EvalReport>

Run a labeled query set against this context at its current version and return a reproducible EvalReport.

Each query is retrieved with config.mode (vector or hybrid) at the config.k cutoff, with config.filters / config.lifecycle applied, then scored against its relevant labels by external_id.

pub async fn evaluate_versions( &mut self, query_set: &EvalQuerySet, config: &EvalConfig, baseline_version: u64, candidate_version: u64, ) -> LanceResult<AbReport>

A/B the same query set across two dataset versions and report per-metric deltas (candidate - baseline). The store is restored to its current version before returning.

impl ContextStore

pub async fn export_training( &mut self, config: &ExportConfig, output_path: &str, ) -> LanceResult<ExportManifest>

Curate stored records and export them as task-shaped JSONL plus a sibling <output_path>.manifest.json, returning the manifest.

If config.version is set the export is pinned to that dataset version (time-travel) and the store is restored to its current version afterward. Output is written line-by-line, so the full JSONL is never held in memory at once.

impl ContextStore

pub async fn open(uri: &str) -> LanceResult<Self>

Open an existing context dataset or create a new one with the project schema.

pub async fn open_with_options( uri: &str, options: ContextStoreOptions, ) -> LanceResult<Self>

Open a dataset with explicit object store configuration (e.g. S3 credentials).

pub fn embedding_dim(&self) -> i32

Embedding vector width persisted in this context dataset schema.

pub fn uri(&self) -> &str

URI of the underlying Lance dataset.

pub fn distance_metric(&self) -> DistanceMetric

Distance metric this context ranks vector-search results with.

pub async fn add(&mut self, entries: &[ContextRecord]) -> LanceResult<u64>

Append context records to the store and return the new dataset version.

pub async fn delete_by_id(&mut self, id: &str) -> LanceResult<bool>

Logically forget a record by internal storage id.

This writes a tombstone with the same primary key, preserving prior dataset versions while hiding the record from default reads.

pub async fn delete_by_external_id( &mut self, external_id: &str, ) -> LanceResult<bool>

Logically forget a record by caller-supplied external id.

pub async fn upsert_by_external_id( &mut self, record: ContextRecord, ) -> LanceResult<UpsertResult>

Insert a record or replace the currently-visible record with the same external id.

Replacement is append-only: the new record keeps the same external_id and gets supersedes_id set to the old record id. Default reads hide the superseded record while include_retired reads can still inspect both versions. Caller-supplied supersession fields are ignored because this method manages replacement by external_id.

pub async fn upsert_many_by_external_id( &mut self, records: Vec<ContextRecord>, ) -> LanceResult<Vec<UpsertResult>>

Insert-or-replace a batch of records keyed by external_id, in one logical operation.

For each record: if a currently-visible record with the same external_id exists, it is replaced append-only (the successor gets supersedes_id set to the existing record id and the original is hidden from default reads); otherwise the record is inserted. All rows are written in a single pass, so records sharing a shard land in a single version bump.

Semantics (parity with Self::upsert_by_external_id and add_many):

• every record must carry a non-empty external_id;
• duplicate ids or external_ids within the batch are rejected;
• validation is all-or-nothing — if any record is invalid, nothing is written;
• caller-supplied supersession fields are ignored (replacement is managed by external_id);
• an insert whose external_id already exists on a non-tombstone but hidden record is rejected, exactly as a single insert would be.

Existing-key resolution and id uniqueness validation are each done in a single scan for the whole batch (not per record), composing with the indexed id validation so a batch does not full-scan per record.

Returns one UpsertResult per input record, in input order, all carrying the final dataset version.

pub async fn update_by_id( &mut self, id: &str, patch: RecordPatch, ) -> LanceResult<Option<UpdateResult>>

Partially update mutable fields on a visible record by internal id.

The update is append-only: it writes a replacement record that supersedes the current visible record, preserving the original payload and embedding while changing only the requested patch fields.

pub async fn update_by_external_id( &mut self, external_id: &str, patch: RecordPatch, ) -> LanceResult<Option<UpdateResult>>

Partially update mutable fields on a visible record by external id.

Returns Ok(None) when no visible record currently has the external id.

pub fn version(&self) -> u64

Current dataset version.

pub async fn migrate_relationships_column(&mut self) -> LanceResult<bool>

Add the relationships column to an older dataset if it is missing.

Existing rows are stored as null in the new column and read back as an empty relationship list.

pub async fn checkout(&mut self, version_id: u64) -> LanceResult<()>

Checkout a specific dataset version.

pub async fn get(&self, id: &str) -> LanceResult<Option<ContextRecord>>

Retrieve a single record by its unique ID.

pub async fn list( &self, limit: Option<usize>, offset: Option<usize>, ) -> LanceResult<Vec<ContextRecord>>

List all records in the dataset.

pub async fn list_filtered( &self, limit: Option<usize>, offset: Option<usize>, filters: Option<&RecordFilters>, ) -> LanceResult<Vec<ContextRecord>>

List records matching filters.

pub async fn list_with_options( &self, limit: Option<usize>, offset: Option<usize>, options: LifecycleQueryOptions, ) -> LanceResult<Vec<ContextRecord>>

List records, applying lifecycle visibility and supersession before offset/limit.

pub async fn list_filtered_with_options( &self, limit: Option<usize>, offset: Option<usize>, filters: Option<&RecordFilters>, options: LifecycleQueryOptions, ) -> LanceResult<Vec<ContextRecord>>

List records matching filters, applying lifecycle visibility before offset/limit.

pub async fn get_by_id(&self, id: &str) -> LanceResult<Option<ContextRecord>>

Find a record by its internal storage id.

pub async fn get_by_external_id( &self, external_id: &str, ) -> LanceResult<Option<ContextRecord>>

Find a record by its caller-supplied external id.

pub async fn list_related( &self, target_id: &str, relation: Option<&str>, limit: Option<usize>, ) -> LanceResult<Vec<ContextRecord>>

List records that have a relationship targeting target_id.

pub async fn list_related_with_options( &self, target_id: &str, relation: Option<&str>, limit: Option<usize>, options: LifecycleQueryOptions, ) -> LanceResult<Vec<ContextRecord>>

List related records, applying lifecycle visibility before relationship matching.

pub async fn search( &self, query: &[f32], limit: Option<usize>, ) -> LanceResult<Vec<SearchResult>>

Perform a nearest-neighbor search over stored embeddings.

pub async fn search_filtered( &self, query: &[f32], limit: Option<usize>, filters: Option<&RecordFilters>, ) -> LanceResult<Vec<SearchResult>>

Perform a nearest-neighbor search over stored embeddings matching filters.

pub async fn search_with_options( &self, query: &[f32], limit: Option<usize>, options: LifecycleQueryOptions, ) -> LanceResult<Vec<SearchResult>>

Perform nearest-neighbor search after applying lifecycle visibility.

pub async fn search_filtered_with_options( &self, query: &[f32], limit: Option<usize>, filters: Option<&RecordFilters>, options: LifecycleQueryOptions, ) -> LanceResult<Vec<SearchResult>>

Perform nearest-neighbor search after applying filters and lifecycle visibility.

pub async fn retrieve_filtered_with_options( &self, text: Option<&str>, vector: Option<&[f32]>, limit: Option<usize>, filters: Option<&RecordFilters>, options: LifecycleQueryOptions, ) -> LanceResult<Vec<RetrieveResult>>

Retrieve records using optional text and vector channels, after filters and lifecycle visibility.

pub async fn compact( &mut self, options: Option<CompactionConfig>, ) -> LanceResult<CompactionMetrics>

Manually trigger compaction to merge small fragments.

pub async fn should_compact(&self) -> LanceResult<bool>

Check if compaction should run based on configuration thresholds.

pub async fn compaction_stats(&self) -> LanceResult<CompactionStats>

Get current compaction statistics.

pub async fn create_id_index(&mut self) -> LanceResult<()>

Create (or replace) the scalar index on the id column.

pub async fn stop_background_compaction(&mut self) -> LanceResult<()>

Stop background compaction task.

pub fn schema(blob_columns: &HashSet<String>) -> Schema

Lance schema for the context store.

When blob_columns contains a column name, that column is stored using Lance V1 blob encoding (out-of-line binary buffers). For text_payload, this also changes the Arrow type from LargeUtf8 to LargeBinary.

pub fn schema_with_embedding_dim( blob_columns: &HashSet<String>, embedding_dim: i32, ) -> Schema

Lance schema for a context store using a caller-selected embedding width.

Trait Implementations

impl Clone for ContextStore

fn clone(&self) -> ContextStore

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl ContextStoreApi for ContextStore

async fn add( &mut self, records: &[AddRecordRequest], ) -> ContextResult<AddRecordsResponse>

async fn upsert( &mut self, request: &UpsertRecordRequest, ) -> ContextResult<UpsertRecordResponse>

async fn upsert_many( &mut self, request: &UpsertRecordsRequest, ) -> ContextResult<UpsertRecordsResponse>

async fn update( &mut self, request: &UpdateRecordRequest, ) -> ContextResult<UpdateRecordResponse>

async fn get(&self, id: &str) -> ContextResult<Option<RecordDto>>

async fn get_by_external_id( &self, external_id: &str, ) -> ContextResult<Option<RecordDto>>

async fn delete_by_id( &mut self, id: &str, ) -> ContextResult<DeleteRecordResponse>

async fn delete_by_external_id( &mut self, external_id: &str, ) -> ContextResult<DeleteRecordResponse>

async fn list( &self, limit: Option<usize>, offset: Option<usize>, filters: Option<Value>, include_expired: bool, include_retired: bool, ) -> ContextResult<Vec<RecordDto>>

async fn related( &self, target_id: &str, relation: Option<&str>, limit: Option<usize>, include_expired: bool, include_retired: bool, ) -> ContextResult<Vec<RecordDto>>

async fn search( &self, request: &SearchRequest, ) -> ContextResult<Vec<SearchResultDto>>

async fn retrieve( &self, request: &RetrieveRequest, ) -> ContextResult<Vec<RetrieveResultDto>>

fn version(&self) -> u64

async fn checkout(&mut self, version: u64) -> ContextResult<()>

async fn compact( &mut self, options: Option<CompactRequest>, ) -> ContextResult<CompactResponse>

async fn compaction_stats(&self) -> ContextResult<CompactStatsResponse>

impl Drop for ContextStore

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.