pub struct PersistentStoreConfig {Show 18 fields
pub data_dir: PathBuf,
pub read_only: bool,
pub writer_lock_timeout_cap: Option<Duration>,
pub segment_max_chunks: u32,
pub min_finalize_chunks: u32,
pub wal_checkpoint_interval: u32,
pub enable_dense_search: bool,
pub enable_hybrid_search: bool,
pub enable_tiered_search: bool,
pub hybrid_config: Option<HybridConfig>,
pub embedding_model: EmbeddingModel,
pub candle_model: CandleModel,
pub enable_async_indexing: bool,
pub async_index_batch_size: usize,
pub async_index_poll_ms: u64,
pub bounded_search_locks: bool,
pub backfill_hnsw_on_startup: bool,
pub backfill_canonical_text_on_startup: bool,
}Expand description
Configuration for persistent store
Fields§
§data_dir: PathBufBase data directory
read_only: boolOpen without writer privileges.
Read-only mode does not acquire the process-wide writer flock,
avoids disk mutations outside SQLite metadata side files, makes
mutating APIs return MemdError::ReadOnlyStore, and serves
WAL-pending chunks from an in-memory overlay instead of replaying
and truncating WAL.
writer_lock_timeout_cap: Option<Duration>Optional upper bound for writer-lock acquisition.
segment_max_chunks: u32Maximum chunks per segment before rotation
min_finalize_chunks: u32Minimum chunks in an active segment for graceful shutdown / Drop
to finalize it. Active segments below this threshold are left
unfinalized between invocations and recovered via WAL replay on
the next startup; they continue to grow across runs until they
cross either this threshold (on a graceful exit) or
segment_max_chunks (on rotation). Prevents the “5,000 tiny
segments” pathology on CLI workloads where each invocation
wrote 1-2 chunks. The WAL durability barrier inside
recover_from_wal ignores this gate so recovered chunks always
land in a finalized segment before WAL truncation.
wal_checkpoint_interval: u32WAL checkpoint interval (chunks)
enable_dense_search: boolEnable dense vector search
enable_hybrid_search: boolEnable hybrid search (dense + sparse)
enable_tiered_search: boolEnable tiered search (cache/hot/warm fallback)
hybrid_config: Option<HybridConfig>Hybrid search configuration
embedding_model: EmbeddingModelEmbedding model to use for dense search
Vestigial ONNX-era selector (retained for config compatibility); the
live dense embedder is the Candle backend selected via candle_model.
candle_model: CandleModelCandle embedder model actually loaded for dense search. Set from the
--embedding-model CLI choice.
enable_async_indexing: boolEnable async/background indexing of newly added chunks
async_index_batch_size: usizeMax pending chunks processed per async indexer tick
async_index_poll_ms: u64Poll interval for async indexer in milliseconds
bounded_search_locks: boolBound the dense search-path lock waits so a contended search fails
fast with IndexBusy instead of parking its thread behind a long
index write hold. Off by default: single-shot CLI processes should
wait out in-process repairs/bulk inserts rather than error. The
warm worker enables it (its event loop must never park) via
apply_warm_worker_availability_defaults.
backfill_hnsw_on_startup: boolOn startup, backfill the HNSW index for tenants whose in-memory
dense state is colder than their metadata (observed when the
previous daemon crashed or was killed before save_all() ran).
When enabled, open() schedules a best-effort background task on
the ambient Tokio runtime that re-indexes stranded chunks. Reads
still work during the backfill (they go through metadata +
segment files); only semantic search is degraded until the task
completes.
backfill_canonical_text_on_startup: boolOn startup, populate canonical_text for any chunk row whose
value is NULL — pre-D2 production rows were inserted with
canonical_text: None, so Track D’s idx_chunks_canonical
partial index never sees them and memory.find_near_duplicates
/ exact-mode supersede_near_duplicates would silently miss
them. The backfill reads each row’s text from its segment,
canonicalises it via canonicalize_for_type(text, chunk_type),
and writes the result back. Best-effort, single-pass.
Implementations§
Source§impl PersistentStoreConfig
impl PersistentStoreConfig
Sourcepub fn apply_warm_worker_availability_defaults(
&mut self,
async_indexing_env: Option<&str>,
)
pub fn apply_warm_worker_availability_defaults( &mut self, async_indexing_env: Option<&str>, )
Availability defaults for a warm-worker process.
The worker’s event loop and its command futures share one task, so a
synchronous add-indexing path that parks on the dense index write
lock (held by a repair or bulk insert for minutes) froze the whole
worker — accept loop, ping, everything — until clients timed out.
Async indexing acknowledges writes after WAL + metadata and lets the
background indexer absorb the lock wait, so the worker defaults it
on. An operator who explicitly set MEMD_ASYNC_INDEXING (either
way) keeps that choice: pass the raw env value; None means unset.
Explicitly disabling it on a worker is warned about, because the
write path then indexes synchronously on the event-loop task again
and a long index write hold can freeze the worker for its duration.
Trait Implementations§
Source§impl Clone for PersistentStoreConfig
impl Clone for PersistentStoreConfig
Source§fn clone(&self) -> PersistentStoreConfig
fn clone(&self) -> PersistentStoreConfig
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for PersistentStoreConfig
impl Debug for PersistentStoreConfig
Auto Trait Implementations§
impl Freeze for PersistentStoreConfig
impl RefUnwindSafe for PersistentStoreConfig
impl Send for PersistentStoreConfig
impl Sync for PersistentStoreConfig
impl Unpin for PersistentStoreConfig
impl UnsafeUnpin for PersistentStoreConfig
impl UnwindSafe for PersistentStoreConfig
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> Downcast for Twhere
T: Any,
impl<T> Downcast for Twhere
T: Any,
Source§fn into_any(self: Box<T>) -> Box<dyn Any>
fn into_any(self: Box<T>) -> Box<dyn Any>
Box<dyn Trait> (where Trait: Downcast) to Box<dyn Any>, which can then be
downcast into Box<dyn ConcreteType> where ConcreteType implements Trait.Source§fn into_any_rc(self: Rc<T>) -> Rc<dyn Any>
fn into_any_rc(self: Rc<T>) -> Rc<dyn Any>
Rc<Trait> (where Trait: Downcast) to Rc<Any>, which can then be further
downcast into Rc<ConcreteType> where ConcreteType implements Trait.Source§fn as_any(&self) -> &(dyn Any + 'static)
fn as_any(&self) -> &(dyn Any + 'static)
&Trait (where Trait: Downcast) to &Any. This is needed since Rust cannot
generate &Any’s vtable from &Trait’s.Source§fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)
fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)
&mut Trait (where Trait: Downcast) to &Any. This is needed since Rust cannot
generate &mut Any’s vtable from &mut Trait’s.Source§impl<T> DowncastSend for T
impl<T> DowncastSend for T
Source§impl<T> DowncastSync for T
impl<T> DowncastSync for T
impl<T> ErasedDestructor for Twhere
T: 'static,
impl<T> Fruit for T
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more