Skip to main content

SqliteStore

Struct SqliteStore 

Source
pub struct SqliteStore { /* private fields */ }
Expand description

Every write (via with_txn or a direct execute) goes through writer, the single connection WAL reserves for the one process-wide writer. Reads go through readers: a pool of separate connections in file-backed mode, or the same connection as the writer for in_memory, where a second connection would just see an empty, unrelated database.

Implementations§

Source§

impl SqliteStore

Source

pub fn live_counts(&self) -> Result<LiveCounts>

Source

pub fn outbox_pending(&self) -> Result<u64>

Outbox rows the index drain has not applied yet. While any are pending the index counts lag the store by design, so a count mismatch then is not drift.

Source

pub fn integrity_report( &self, indexes: Option<&Indexes>, ) -> Result<IntegrityReport>

Source

pub fn repair( &self, indexes: Option<&mut Indexes>, embed: Option<EmbedFn<'_>>, dry_run: bool, ) -> Result<RepairStats>

Restores every invariant in IntegrityReport::drift, each step in its own transactions so a failure part-way leaves earlier steps committed. With indexes, the outbox is drained before the key garbage collection and before the consistency check, so one pass leaves nothing for the next. dry_run counts without writing.

Source

pub fn backfill_embeddings( &self, embed: EmbedFn<'_>, dry_run: bool, ) -> Result<(u64, u64)>

Embeds hot rows that have none, over the same context-plus-content body the ingest path embeds. updated_at is bumped so the peer’s last-writer-wins check accepts the row that now carries a vector. Runs only this step of repair, so a caller that must not merge or retire rows can fill vectors on its own schedule. Returns (rows embedded, rows whose embedding failed or had the wrong dimension); with dry_run it counts rows awaiting a vector and writes nothing.

Source

pub fn vacuum(&self) -> Result<(u64, u64)>

VACUUM cannot run inside a transaction and needs the writer alone; returns (bytes before, bytes after).

Source

pub fn sync_log_row_exists(&self, local_seq: u64) -> Result<bool>

Whether a sync_log row with this local_seq still exists; a test and diagnostic hook for the sync log garbage collector.

Source§

impl SqliteStore

Source

pub fn restore_explicit_memories(&self) -> Result<ExplicitRestoreStats>

Undoes what consolidation did to explicit memories before EXPLICIT_SOURCES was excluded from it: every archived explicit raw returns to the hot tier and the indexes, every marker on an explicit raw is dropped so no later pass retires it again, and a distillate whose raws were all explicit is superseded by the oldest of them, since nothing it summarised is missing from recall any more. A distillate that also stands in for transcript raws stays live. Every change replicates the way consolidation’s own writes do.

Source

pub fn refile_nested_transcript_projects(&self) -> Result<u64>

Re-files memories that an older adapter filed under the directory holding the transcript file (subagents, agent-transcripts) rather than the project directory under the projects root. Raws re-file from their own source_id; a distillate re-files to the project most of its raws now carry, because a consolidation batch is grouped by conversation and one project is the only value that fits a row with a single project_path. Every changed row is replicated as an Update and queued for the indexes, exactly as update_memory does.

Source

pub fn refile_stray_project_slugs(&self) -> Result<StraySlugReport>

Re-files explicit memories that an earlier remember filed under the bare directory name it was given (wardrobe) instead of the slug ingest stores for that directory (-mnt-data-wardrobe). A bare name resolves to the stored slugs ending in -<name>, with name encoded the way Claude Code encodes a path; the two spellings Claude Code and Cursor give one directory (with and without the leading -) count as one project. A name matching more than one project is reported, never guessed, and transcript rows under a bare slug are left alone because an adapter filed them there on purpose.

Source

pub fn refile_memory(&self, id: Uuid, project_path: &str) -> Result<bool>

Moves one memory to project_path, replicated and re-indexed as every refile is. False when it was already filed there or is absent.

Source§

impl SqliteStore

Source

pub fn open(path: &Path, machine_id: String) -> Result<Self>

Source

pub fn open_with_pool_size( path: &Path, machine_id: String, reader_pool_size: usize, ) -> Result<Self>

Same as open, with an explicit reader pool size instead of the parallelism-derived default. reader_pool_size is clamped to at least 1 so a caller passing 0 still gets a working store.

Source

pub fn in_memory(machine_id: String) -> Result<Self>

Source

pub fn machine_id(&self) -> &str

Source

pub fn tombstone_exists(&self, id: Uuid) -> Result<bool>

Source

pub fn memory_content_exists(&self, content_prefix: &str) -> Result<bool>

Source

pub fn max_sync_seq(&self) -> Result<u64>

Source

pub fn get_conversation(&self, id: Uuid) -> Result<Option<Conversation>>

Source§

impl SqliteStore

Source

pub fn index_watermark(&self) -> Result<i64>

Source

pub fn set_index_watermark(&self, seq: i64) -> Result<()>

Source

pub fn outbox_after(&self, seq: i64) -> Result<Vec<OutboxRow>>

Source

pub fn prune_outbox(&self, upto: i64) -> Result<()>

Source

pub fn max_outbox_seq(&self) -> Result<i64>

Source

pub fn vec_key(&self, memory_id: Uuid) -> Result<u64>

Stable u64 usearch key for a memory id, allocated on first use.

Source

pub fn existing_vec_key(&self, memory_id: Uuid) -> Result<Option<u64>>

Source

pub fn delete_vec_key(&self, memory_id: Uuid) -> Result<()>

Drops the key row once the vector is gone from usearch, so vec_keys only ever holds keys the index can still resolve.

Source

pub fn find_exact_duplicate( &self, content: &str, project_path: Option<&str>, ) -> Result<Option<Memory>>

The live memory content duplicates under remember’s rule: equal once surrounding whitespace is trimmed and both sides are lowercased. With project_path, only memories filed under that project count; among several matches the oldest wins. SQLite’s lower() folds ASCII only, so a stored memory that differs from content just in the case of a non-ASCII letter is not found, which the substring shortlist this replaces could not find either.

bm25_search’s match with a source/type/project scope inside the query, so limit counts only rows the scope admits instead of cutting before a caller’s filter could run. The match is a substring test over content on hot, unsuperseded rows, with no relevance score to order by (tantivy, not SQLite, holds the text index), so matches come back newest first. Case folding is SQLite’s lower(), which is ASCII only.

Source

pub fn scoped_count( &self, source: Option<&str>, memory_type: Option<MemoryType>, project_path: Option<&str>, ) -> Result<usize>

How many live memories a source/type/project scope holds. Answered by COUNT off the source, memory_type, and project_path indexes, so it stays O(1) in memory however large the scope grows.

Source

pub fn retain_in_scope( &self, ids: &[Uuid], source: Option<&str>, memory_type: Option<MemoryType>, project_path: Option<&str>, ) -> Result<HashSet<Uuid>>

Which of ids fall inside a source/type/project scope.

Asks the database about the candidates in hand rather than reading the scope out to test membership locally, so the cost tracks the caller’s list and not the scope. Batched because SQLite caps how many bound parameters one statement may carry.

Source

pub fn memory_id_for_vec_key(&self, key: u64) -> Result<Option<Uuid>>

Source

pub fn memory_for_indexing( &self, memory_id: Uuid, ) -> Result<Option<IndexableMemory>>

Source

pub fn all_indexable(&self) -> Result<Vec<(Uuid, Vec<f32>, String)>>

Source

pub fn vector_indexable_count(&self, dim: usize) -> Result<usize>

Hot rows whose stored embedding has exactly dim components: the only rows the vector index can hold, so a row still waiting for an embedding does not read as index drift.

Source

pub fn indexable_count(&self) -> Result<usize>

Source

pub fn supersede_memory(&self, memory_id: Uuid, by: Uuid) -> Result<()>

Zep-style soft invalidation: the row stays, the indexes drop it via the outbox, and peers learn of it through a sync Update carrying the row.

Source

pub fn archive_memory(&self, memory_id: Uuid) -> Result<()>

Local cold tier: the row keeps its content and embedding but leaves both retrieval indexes and every recall predicate. archived_segment stays NULL, which is what distinguishes this from mark_archived’s uploaded-segment stub. Replicated as an Update so peers retire the same row instead of keeping it hot beside the distillate.

Source

pub fn retire_memory( &self, memory_id: Uuid, reason: &str, ) -> Result<RetireOutcome>

forget’s default: archives the row as archive_memory does and records why, so the memory leaves recall on every peer while its content stays recoverable. Nothing is tombstoned, and an id this store does not hold changes nothing.

Source

pub fn mark_archived(&self, memory_id: Uuid, segment: &str) -> Result<()>

Cold-tier flip: content becomes a 200-char stub, the embedding is dropped (the full row lives in the archived segment), and the outbox evicts the memory from both indexes.

Trait Implementations§

Source§

impl Store for SqliteStore

Source§

fn traverse( &self, start_id: Uuid, depth: u32, ) -> Result<Vec<(Memory, Vec<Relation>)>>

Live memories within depth hops of start_id, newest first, capped at TRAVERSE_MAX_RESULTS. Ingest writes no memory-to-memory edges (only mentions, derived_from and distilled_from), so a hop is any of: a memory-to-memory relation in either direction, a shared mentioned entity, or a shared conversation.

Source§

fn store_memory(&self, memory: &Memory) -> Result<()>

Source§

fn get_memory(&self, id: Uuid) -> Result<Option<Memory>>

Source§

fn get_memory_with_embedding(&self, id: Uuid) -> Result<Option<Memory>>

Source§

fn delete_memory(&self, id: Uuid) -> Result<()>

Source§

fn remove_memory(&self, id: Uuid) -> Result<bool>

delete_memory that also reports whether a row was there to remove, so a forget of an unknown id can be answered as such instead of silently succeeding.
Source§

fn store_entity(&self, entity: &Entity) -> Result<()>

Source§

fn get_entity(&self, id: Uuid) -> Result<Option<Entity>>

Source§

fn find_entity_by_name(&self, name: &str) -> Result<Option<Entity>>

Source§

fn store_conversation(&self, conversation: &Conversation) -> Result<()>

Source§

fn store_relation(&self, relation: &Relation) -> Result<()>

Source§

fn get_relations( &self, node_id: Uuid, relation_type: Option<RelationType>, ) -> Result<Vec<Relation>>

Cosine search restricted to memories matching source, memory_type, and/or project_path, scanned exhaustively over that subset. Read more
Source§

fn memories_by_source(&self, source: &str) -> Result<Vec<Memory>>

Source§

fn source_admits(&self, memory: &Memory, source: &str) -> Result<bool>

Whether memory answers a recall scoped to source. Consolidation leaves the raws archived and the distillate tagged ‘consolidation’, so a store that tracks distillation must let a distillate answer for the sources of its raws, or a source filter finds nothing once its raws are consolidated. The default is the plain column comparison for stores that never consolidate.
Source§

fn memories_by_type(&self, memory_type: MemoryType) -> Result<Vec<Memory>>

Source§

fn update_memory(&self, memory: &Memory) -> Result<()>

Source§

fn record_access(&self, memory: &Memory) -> Result<()>

Source§

fn record_accesses(&self, ids: &[Uuid]) -> Result<()>

Count one access on each of ids and stamp them with the current time; confidence is left alone. A recall touches every result it returns, so an implementation should batch these into one write rather than take the writer lock per row. Ids the store does not hold are ignored.
Source§

fn memory_count(&self) -> Result<usize>

Source§

fn all_memory_ids(&self) -> Result<Vec<Uuid>>

Source§

fn all_relations(&self) -> Result<Vec<Relation>>

Source§

impl StoreAdmin for SqliteStore

Source§

fn register_machine(&self, identity: &MachineIdentity) -> Result<()>

Source§

fn get_all_machines(&self) -> Result<HashMap<String, String>>

Source§

fn diagnostic(&self) -> Result<String>

Source§

fn find_or_create_entity(&self, name: &str, entity_type: &str) -> Result<Entity>

Source§

fn memory_content_exists(&self, content_prefix: &str) -> Result<bool>

Source§

fn is_file_changed(&self, file_path: &str, file_hash: &str) -> Result<bool>

Source§

fn mark_ingested( &self, file_path: &str, file_hash: &str, memory_count: usize, source: &str, ) -> Result<()>

Source§

fn ingested_file_count(&self) -> Result<usize>

Source§

fn clear_ingest_log(&self) -> Result<usize>

Source§

fn delete_memories_by_source(&self, source: &str) -> Result<usize>

Source§

fn delete_memories_by_source_id(&self, source_id: &str) -> Result<usize>

Source§

fn rebuild_vector_index(&self) -> Result<()>

Source§

fn backfill_project_paths(&self) -> Result<u64>

Source§

fn backfill_sync_log(&self) -> Result<u64>

Source§

fn backfill_tombstones(&self) -> Result<u64>

Source§

fn prune_dangling_relations(&self) -> Result<u64>

Source§

fn compact_sync_log( &self, ops: &[SyncOp], dry_run: bool, ) -> Result<CompactStats>

Collapses re-logged duplicate sync_log rows down to one per (node_type, node_id, op), keeping the highest local_seq in each group since the counter is derived as MAX(local_seq)+1 on every write: dropping the row holding the true max would let the next write reuse a sequence a peer already considers seen.
Source§

fn memories_by_project_path(&self, project_path: &str) -> Result<Vec<Memory>>

Source§

fn distinct_project_paths(&self) -> Result<Vec<(String, u64)>>

Every distinct non-null project_path value with its live memory count, so a caller can discover the real slug recall’s project scope expects instead of guessing.
Source§

fn all_memories_with_embeddings(&self) -> Result<Vec<Memory>>

Source§

fn consolidation_stats(&self) -> Result<ConsolidationStats>

Source§

fn unconsolidated_memories(&self, limit: usize) -> Result<Vec<Memory>>

Hot, unsuperseded raws with no distillation_log row, ordered by (source_id, created_at) so a batch reads as conversations.
Source§

fn mark_consolidated( &self, raw_id: Uuid, distilled_id: Uuid, model: &str, ) -> Result<()>

Source§

fn retire_consolidated_raws(&self) -> Result<usize>

Moves every distilled or skip-marked raw that is still hot to the archive tier. Nothing is deleted: the raw keeps its content and embedding and only leaves recall, so a distillate can always be traced back to what it was distilled from.
Source§

fn memories_created_between( &self, start: &DateTime<Utc>, end: &DateTime<Utc>, ) -> Result<Vec<Memory>>

Source§

fn sync_log_since(&self, after_seq: u64) -> Result<Vec<SyncEntry>>

Source§

fn sync_log_page( &self, after_seq: u64, limit: Option<usize>, ) -> Result<Vec<SyncEntry>>

Source§

fn max_sync_seq(&self) -> Result<u64>

Source§

fn get_sync_state(&self, peer_id: &str) -> Result<Option<SyncState>>

Source§

fn set_sync_state(&self, state: &SyncState) -> Result<()>

Source§

fn get_all_sync_states(&self) -> Result<Vec<SyncState>>

Source§

fn tombstone_exists(&self, id: Uuid) -> Result<bool>

Source§

fn apply_record(&self, record: &SyncRecord) -> Result<ApplyOutcome>

Applies one remote sync record with LWW/tombstone conflict semantics, in its own transaction.
Source§

fn apply_batch(&self, records: &[SyncRecord]) -> Result<Vec<ApplyOutcome>>

Applies a batch atomically: either every record commits or none do, so a caller can fall back to per-record apply_record on error without double-applying a prefix.

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> Downcast for T
where T: Any,

Source§

fn into_any(self: Box<T>) -> Box<dyn Any>

Converts Box<dyn Trait> (where Trait: Downcast) to Box<dyn Any>, which can then be downcast into Box<dyn ConcreteType> where ConcreteType implements Trait.
Source§

fn into_any_rc(self: Rc<T>) -> Rc<dyn Any>

Converts Rc<Trait> (where Trait: Downcast) to Rc<Any>, which can then be further downcast into Rc<ConcreteType> where ConcreteType implements Trait.
Source§

fn as_any(&self) -> &(dyn Any + 'static)

Converts &Trait (where Trait: Downcast) to &Any. This is needed since Rust cannot generate &Any’s vtable from &Trait’s.
Source§

fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)

Converts &mut Trait (where Trait: Downcast) to &Any. This is needed since Rust cannot generate &mut Any’s vtable from &mut Trait’s.
Source§

impl<T> DowncastSend for T
where T: Any + Send,

Source§

fn into_any_send(self: Box<T>) -> Box<dyn Any + Send>

Converts Box<Trait> (where Trait: DowncastSend) to Box<dyn Any + Send>, which can then be downcast into Box<ConcreteType> where ConcreteType implements Trait.
Source§

impl<T> DowncastSync for T
where T: Any + Send + Sync,

Source§

fn into_any_sync(self: Box<T>) -> Box<dyn Any + Send + Sync>

Converts Box<Trait> (where Trait: DowncastSync) to Box<dyn Any + Send + Sync>, which can then be downcast into Box<ConcreteType> where ConcreteType implements Trait.
Source§

fn into_any_arc(self: Arc<T>) -> Arc<dyn Any + Send + Sync> ⓘ

Converts Arc<Trait> (where Trait: DowncastSync) to Arc<Any>, which can then be downcast into Arc<ConcreteType> where ConcreteType implements Trait.
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Fruit for T
where T: Send + Downcast,

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self> ⓘ

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self> ⓘ

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V

Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> ⓘ
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self> ⓘ

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more