Skip to main content

PersistentStoreConfig

Struct PersistentStoreConfig 

Source
pub struct PersistentStoreConfig {
Show 18 fields pub data_dir: PathBuf, pub read_only: bool, pub writer_lock_timeout_cap: Option<Duration>, pub segment_max_chunks: u32, pub min_finalize_chunks: u32, pub wal_checkpoint_interval: u32, pub enable_dense_search: bool, pub enable_hybrid_search: bool, pub enable_tiered_search: bool, pub hybrid_config: Option<HybridConfig>, pub embedding_model: EmbeddingModel, pub candle_model: CandleModel, pub enable_async_indexing: bool, pub async_index_batch_size: usize, pub async_index_poll_ms: u64, pub bounded_search_locks: bool, pub backfill_hnsw_on_startup: bool, pub backfill_canonical_text_on_startup: bool,
}
Expand description

Configuration for persistent store

Fields§

§data_dir: PathBuf

Base data directory

§read_only: bool

Open without writer privileges.

Read-only mode does not acquire the process-wide writer flock, avoids disk mutations outside SQLite metadata side files, makes mutating APIs return MemdError::ReadOnlyStore, and serves WAL-pending chunks from an in-memory overlay instead of replaying and truncating WAL.

§writer_lock_timeout_cap: Option<Duration>

Optional upper bound for writer-lock acquisition.

§segment_max_chunks: u32

Maximum chunks per segment before rotation

§min_finalize_chunks: u32

Minimum chunks in an active segment for graceful shutdown / Drop to finalize it. Active segments below this threshold are left unfinalized between invocations and recovered via WAL replay on the next startup; they continue to grow across runs until they cross either this threshold (on a graceful exit) or segment_max_chunks (on rotation). Prevents the “5,000 tiny segments” pathology on CLI workloads where each invocation wrote 1-2 chunks. The WAL durability barrier inside recover_from_wal ignores this gate so recovered chunks always land in a finalized segment before WAL truncation.

§wal_checkpoint_interval: u32

WAL checkpoint interval (chunks)

§enable_dense_search: bool

Enable dense vector search

§enable_hybrid_search: bool

Enable hybrid search (dense + sparse)

§enable_tiered_search: bool

Enable tiered search (cache/hot/warm fallback)

§hybrid_config: Option<HybridConfig>

Hybrid search configuration

§embedding_model: EmbeddingModel

Embedding model to use for dense search

Vestigial ONNX-era selector (retained for config compatibility); the live dense embedder is the Candle backend selected via candle_model.

§candle_model: CandleModel

Candle embedder model actually loaded for dense search. Set from the --embedding-model CLI choice.

§enable_async_indexing: bool

Enable async/background indexing of newly added chunks

§async_index_batch_size: usize

Max pending chunks processed per async indexer tick

§async_index_poll_ms: u64

Poll interval for async indexer in milliseconds

§bounded_search_locks: bool

Bound the dense search-path lock waits so a contended search fails fast with IndexBusy instead of parking its thread behind a long index write hold. Off by default: single-shot CLI processes should wait out in-process repairs/bulk inserts rather than error. The warm worker enables it (its event loop must never park) via apply_warm_worker_availability_defaults.

§backfill_hnsw_on_startup: bool

On startup, backfill the HNSW index for tenants whose in-memory dense state is colder than their metadata (observed when the previous daemon crashed or was killed before save_all() ran). When enabled, open() schedules a best-effort background task on the ambient Tokio runtime that re-indexes stranded chunks. Reads still work during the backfill (they go through metadata + segment files); only semantic search is degraded until the task completes.

§backfill_canonical_text_on_startup: bool

On startup, populate canonical_text for any chunk row whose value is NULL — pre-D2 production rows were inserted with canonical_text: None, so Track D’s idx_chunks_canonical partial index never sees them and memory.find_near_duplicates / exact-mode supersede_near_duplicates would silently miss them. The backfill reads each row’s text from its segment, canonicalises it via canonicalize_for_type(text, chunk_type), and writes the result back. Best-effort, single-pass.

Implementations§

Source§

impl PersistentStoreConfig

Source

pub fn apply_warm_worker_availability_defaults( &mut self, async_indexing_env: Option<&str>, )

Availability defaults for a warm-worker process.

The worker’s event loop and its command futures share one task, so a synchronous add-indexing path that parks on the dense index write lock (held by a repair or bulk insert for minutes) froze the whole worker — accept loop, ping, everything — until clients timed out. Async indexing acknowledges writes after WAL + metadata and lets the background indexer absorb the lock wait, so the worker defaults it on. An operator who explicitly set MEMD_ASYNC_INDEXING (either way) keeps that choice: pass the raw env value; None means unset. Explicitly disabling it on a worker is warned about, because the write path then indexes synchronously on the event-loop task again and a long index write hold can freeze the worker for its duration.

Trait Implementations§

Source§

impl Clone for PersistentStoreConfig

Source§

fn clone(&self) -> PersistentStoreConfig

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for PersistentStoreConfig

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Default for PersistentStoreConfig

Source§

fn default() -> Self

Returns the “default value” for a type. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> Downcast for T
where T: Any,

Source§

fn into_any(self: Box<T>) -> Box<dyn Any>

Converts Box<dyn Trait> (where Trait: Downcast) to Box<dyn Any>, which can then be downcast into Box<dyn ConcreteType> where ConcreteType implements Trait.
Source§

fn into_any_rc(self: Rc<T>) -> Rc<dyn Any>

Converts Rc<Trait> (where Trait: Downcast) to Rc<Any>, which can then be further downcast into Rc<ConcreteType> where ConcreteType implements Trait.
Source§

fn as_any(&self) -> &(dyn Any + 'static)

Converts &Trait (where Trait: Downcast) to &Any. This is needed since Rust cannot generate &Any’s vtable from &Trait’s.
Source§

fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)

Converts &mut Trait (where Trait: Downcast) to &Any. This is needed since Rust cannot generate &mut Any’s vtable from &mut Trait’s.
Source§

impl<T> DowncastSend for T
where T: Any + Send,

Source§

fn into_any_send(self: Box<T>) -> Box<dyn Any + Send>

Converts Box<Trait> (where Trait: DowncastSend) to Box<dyn Any + Send>, which can then be downcast into Box<ConcreteType> where ConcreteType implements Trait.
Source§

impl<T> DowncastSync for T
where T: Any + Send + Sync,

Source§

fn into_any_sync(self: Box<T>) -> Box<dyn Any + Sync + Send>

Converts Box<Trait> (where Trait: DowncastSync) to Box<dyn Any + Send + Sync>, which can then be downcast into Box<ConcreteType> where ConcreteType implements Trait.
Source§

fn into_any_arc(self: Arc<T>) -> Arc<dyn Any + Sync + Send> ⓘ

Converts Arc<Trait> (where Trait: DowncastSync) to Arc<Any>, which can then be downcast into Arc<ConcreteType> where ConcreteType implements Trait.
Source§

impl<T> ErasedDestructor for T
where T: 'static,

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Fruit for T
where T: Send + Downcast,

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V

Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more