Struct SideIndex 

pub struct SideIndex {
    pub ivf_drift: u64,
    /* private fields */
}

Fields

ivf_drift: u64

Count of vector inserts/removes since last fit. When dst-side drift exceeds IVF_DRIFT_REBUILD on an approximate rule, apply queues a RebuildRule second commit (fit resets this to zero).

Implementations

impl SideIndex

pub fn insert( &mut self, spec: &CandidateSpec<'_>, node: u32, get: &dyn Fn(&str) -> Option<Value>, )

pub fn insert_skipping( &mut self, spec: &CandidateSpec<'_>, node: u32, already: &BTreeSet<u32>, get: &dyn Fn(&str) -> Option<Value>, )

insert, but skip the HNSW graph for ids in already — the open-time scan’s version, where the adopted graph is the base and the scan only has to supply what the snapshot did not carry.

hnsw_tracked is still recorded for every node, adopted or not: it is the fallback candidate set and must cover the whole side.

pub fn insert_deferring_hnsw( &mut self, spec: &CandidateSpec<'_>, node: u32, get: &dyn Fn(&str) -> Option<Value>, )

insert, but the HNSW graph is left untouched — the sliced-build version, where SideIndex::insert_hnsw_only supplies the vectors a slice at a time.

Every other leg of spec (by-key buckets, IVF, ScanAll metadata) is filed exactly as insert files it, and hnsw_tracked is still recorded, so the rule’s non-vector state is whole from the moment it is created.

pub fn insert_hnsw_only( &mut self, spec: &CandidateSpec<'_>, node: u32, get: &dyn Fn(&str) -> Option<Value>, ) -> bool

Insert node into the HNSW graph only, leaving every other leg of spec alone — the second half of SideIndex::insert_deferring_hnsw.

Returns true when a vector actually went into a graph, which is how a build slice counts what it has done.

pub fn remove( &mut self, spec: &CandidateSpec<'_>, node: u32, get: &dyn Fn(&str) -> Option<Value>, )

pub fn vec_dim(&self, node: u32) -> Option<u32>

Cached vector dimension for a ScanAll member, if present.

pub fn vec_meta(&self, node: u32) -> Option<(u32, f64)>

Cached (dim, L2 norm) for tests / debug.

pub fn vec_ckpts(&self, node: u32) -> Option<&[f64; 8]>

Cached checkpoints for tests / debug.

pub fn candidates( &self, spec: &CandidateSpec<'_>, get: &dyn Fn(&str) -> Option<Value>, ) -> BTreeSet<u32>

pub fn fit_ivf_clusters(&mut self, rule_name: &str)

Fit (or re-fit) the IVF k-means index for this side using all currently stored raw vectors. Called by the engine after reindexing all nodes in create_rule and rebuild.

rule_name is hashed via FNV-1a to produce a stable seed, ensuring the same rule+data always yields the same clusters (WAL replay identity).

Clears all existing cluster assignments and by_key cluster entries, then assigns every non-zero vector (L2-normalized) to its nearest new centroid. Resets ivf_drift to zero.

pub fn ivf_k(&self) -> usize

Number of fitted centroids (0 = not yet fitted).

pub fn ivf_cluster_of(&self, node: u32) -> Option<usize>

Cluster assignment for a node (None if not fitted or node not in index).

pub fn export_ivf_state(&self) -> (Vec<Vec<f64>>, BTreeMap<u32, usize>, u64)

Export IVF state for snapshot persistence: (centroids, clusters, drift).

The caller stores this in the V4 snapshot and passes it back to load_ivf_state on the next open, avoiding a full k-means re-fit.

pub fn load_ivf_state( &mut self, centroids: Vec<Vec<f64>>, clusters: BTreeMap<u32, usize>, drift: u64, )

Restore IVF state from a V4 snapshot.

This must be called AFTER the normal insert() pass (which populates ivf_raw) but INSTEAD OF fit_ivf_clusters. It:

1. Removes any stale cluster-key entries from by_key.
2. Installs the persisted centroids and drift counter.
3. Rebuilds by_key cluster buckets from the persisted assignments.

Nodes present in ivf_raw but absent from clusters (e.g. inserted post-snapshot via WAL replay before this is called) are left unassigned; on_node_changed will assign them to the nearest centroid incrementally.

pub fn init_hnsw(&mut self, rule_name: &str)

Initialise the HNSW graph for this side, seeding it with FNV-1a(rule_name).

Must be called before inserting nodes via CandidateSpec::Hnsw. Idempotent: calling again with the same name replaces the existing graph.

pub fn export_hnsw_blob(&self, complete: bool) -> Vec<u8>

Export the HNSW graph as an opaque versioned blob.

Returns an empty Vec when the HNSW is not initialized.

complete is false when the rule’s sliced build still owes this side vectors; it rides in the blob so that a reader opening the snapshot knows the graph is a prefix and takes its exhaustive path rather than answering confidently about a fraction of the corpus. The engine reads it from pending_builds, which is not itself persisted.

pub fn load_hnsw_blob(&mut self, blob: &[u8])

Restore the HNSW graph from a previously exported blob.

The hnsw_tracked set is populated from the restored graph’s node ids so candidates/remove work correctly after restore. Silently ignores empty, corrupt, or unknown-version blobs (the HNSW stays uninitialized and the side keeps its full-scan fallback).

pub fn init_or_adopt_hnsw( &mut self, rule_name: &str, blob: &[u8], ) -> (BTreeSet<u32>, bool)

Initialise this side’s HNSW graph, adopting blob when it holds one.

Returns the node ids the adopted graph already contains, so an open-time scan can skip re-inserting them. An empty, corrupt, or unknown-version blob yields an empty graph and an empty set — exactly what init_hnsw gives today — and the scan then builds the graph as it always did.

true in the second slot means “this side was adopted, not built”, which is what the caller counts as a skipped build.

pub fn adopt_hnsw(&mut self, h: HnswIndex)

Install an already-deserialized HNSW graph, replacing any existing one.

hnsw_tracked is repopulated from the graph’s node ids so candidates and removal work against the installed graph rather than whatever the preceding node scan happened to record.

pub fn has_hnsw(&self) -> bool

True when the HNSW graph has been initialized and contains at least one node.

pub fn hnsw_ref(&self) -> Option<&HnswIndex>

Borrow the HNSW index, if initialized.

pub fn take_hnsw(&mut self) -> Option<HnswIndex>

Remove and return this side’s HNSW graph, leaving the side without one.

Lets a caller that is about to reset the whole SideIndex carry the graph across — the graph is the expensive part and is not always worth rebuilding.

Trait Implementations

impl Debug for SideIndex

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SideIndex

fn default() -> SideIndex

Returns the “default value” for a type.