# API Reference: fst_incremental
An updatable finite state set: an immutable `fst::Set` fronted by an in-memory mutation buffer, with amortized rebuilds.
## Contents
* [1. Primary Type](#1-primary-type)
* [IncrementalFstSet](#incrementalfstset)
* [2. Configuration](#2-configuration)
* [FstConfigOptions](#fstconfigoptions)
* [3. Change Reporting](#3-change-reporting)
* [FstMutationResult](#fstmutationresult)
* [FstChangeType](#fstchangetype)
* [4. Persistence](#4-persistence)
* [SerializableFstBuffers](#serializablefstbuffers)
* [CompactArenaSet](#compactarenaset)
* [5. Streaming](#5-streaming)
* [MergedSetStreamOwner](#mergedsetstreamowner)
* [6. Metrics](#6-metrics)
* [FstMetricsSnapshot](#fstmetricssnapshot)
* [7. Re-exports](#7-re-exports)
* [8. Error Handling](#8-error-handling)
* [IncrementalFstError](#incrementalfsterror)
* [Result](#result)
## 1. Primary Type
### IncrementalFstSet
A thread-safe set of byte strings supporting insertion and removal. All methods take `&self` and lock internally; the type is `Send + Sync`.
Constructors:
* `pub fn new(options: Option<FstConfigOptions>) -> Result<Self>` — creates an empty set.
* `pub fn from_data(fst_bytes: Option<Vec<u8>>, options: Option<FstConfigOptions>) -> Result<Self>` — opens a set over FST bytes held in memory. `None` starts empty. Returns `Fst` if the bytes are not a valid FST.
* `pub fn from_persisted_mmap(mmap_path: &Path, options: Option<FstConfigOptions>) -> Result<Self>` — memory-maps the FST at `mmap_path`. The file must not be modified while the set is alive. Returns `Io` if the file cannot be opened or mapped.
Mutation. Each returns the change it made; each may trigger a rebuild unless noted:
* `pub fn insert(&self, key: Vec<u8>) -> Result<FstMutationResult>` — adds `key`, resurrecting it if tombstoned. `NoChange` if already live.
* `pub fn remove(&self, key: &[u8]) -> Result<FstMutationResult>` — removes `key`. A key present only in the persisted component is tombstoned, not erased. `NoChange` if already absent.
* `pub fn bulk_insert<I: IntoIterator<Item = Vec<u8>>>(&self, iter: I) -> Result<FstMutationResult>` — adds every key under one write lock. Never rebuilds; returns `BuffersModified` if any key was added, `NoChange` otherwise.
* `pub fn finish_bulk_operations_and_rebuild_if_needed(&self) -> Result<FstMutationResult>` — ends a bulk sequence, rebuilding if the buffer holds anything or a rebuild was deferred.
* `pub fn force_rebuild(&self) -> Result<FstMutationResult>` — rebuilds regardless of the thresholds. A configured minimum rebuild interval still applies: inside the window this defers and returns `BuffersModified`.
Queries:
* `pub fn contains(&self, key: &[u8]) -> Result<bool>` — one buffer probe, then one FST lookup.
* `pub fn len(&self) -> Result<usize>` — number of live keys. O(n): walks the merged stream, as no live count is stored.
* `pub fn is_empty(&self) -> Result<bool>` — whether any live key exists. Stops at the first.
* `pub fn stream(&self) -> Result<MergedSetStreamOwner<'_>>` — every live key in lexicographic order. Holds a read lock until dropped.
* `pub fn search<A: Automaton + Clone>(&self, aut: A) -> Result<Vec<Vec<u8>>>` — every live key matching `aut`, sorted and deduplicated. Collects rather than streams. Takes the write lock, because it may build the buffer overlay FST.
Accessors:
* `pub fn persisted_fst_as_bytes(&self) -> Vec<u8>` — copy of the persisted FST bytes. Excludes the mutation buffer. Copies even when the source is memory-mapped.
* `pub fn buffers_snapshot(&self) -> SerializableFstBuffers` — copy of the pending mutations.
* `pub fn get_metrics(&self) -> Result<FstMetricsSnapshot>` — cumulative rebuild counters.
## 2. Configuration
### FstConfigOptions
Rebuild thresholds and initial buffer contents. Accepted by every constructor; `None` there means `Default::default()`. `#[non_exhaustive]`.
Fields, all `pub`. `None` on a threshold means the default applies:
* `rebuild_threshold_adds_count: Option<usize>`
* `rebuild_threshold_dels_ratio: Option<f32>`
* `min_rebuild_interval_ms: Option<u64>`
* `initial_mutation_buffer: Option<CompactArenaSet>`
Associated constants:
* `pub const DEFAULT_REBUILD_ADDS_COUNT: usize = 10000` — live buffer entries that trigger a rebuild.
* `pub const DEFAULT_REBUILD_DELS_RATIO: f32 = 0.3` — tombstones as a fraction of persisted keys that trigger a rebuild.
Builders:
* `pub fn new() -> Self` — same as `Default::default()`: both thresholds at their defaults, no minimum interval, empty buffer.
* `pub fn with_rebuild_adds_count(self, count: usize) -> Self`
* `pub fn with_rebuild_dels_ratio(self, ratio: f32) -> Self` — a fraction, not a percentage.
* `pub fn with_min_rebuild_interval_ms(self, interval_ms: u64) -> Self` — floor on the gap between rebuilds. A rebuild due inside the window is deferred, reported as `BuffersModified`, and run at the next call past the window. Applies to `force_rebuild` too.
* `pub fn with_initial_buffer(self, buffer: CompactArenaSet) -> Self` — seeds the set from a restored `SerializableFstBuffers`.
Implements `Debug`, `Clone`, `Default`.
## 3. Change Reporting
### FstMutationResult
Returned by every mutating call. `#[non_exhaustive]`.
* `pub change_type: FstChangeType`
Implements `Debug`, `Clone`, `Copy`, `PartialEq`, `Eq`.
### FstChangeType
What a mutation did. `#[non_exhaustive]`: a match on it requires a wildcard arm.
* `BuffersModified` — only the in-memory buffer changed. The persisted FST bytes are unchanged, so only the buffer needs rewriting.
* `FstRebuilt` — the persisted FST was rebuilt and the buffer was cleared into it. Both need rewriting.
* `NoChange` — the set already agreed with the requested state. Nothing needs rewriting.
Implements `Debug`, `Clone`, `Copy`, `PartialEq`, `Eq`.
## 4. Persistence
### SerializableFstBuffers
A point-in-time copy of the mutation buffer, produced by `buffers_snapshot` and consumed by `FstConfigOptions::with_initial_buffer`. Together with `persisted_fst_as_bytes` this is the complete state of a set. Constructible directly, since a caller restoring a buffer from their own storage needs to build one.
* `pub fn new(mutation_buffer: CompactArenaSet) -> Self`
* `pub fn mutation_buffer(&self) -> &CompactArenaSet` — borrows the buffer.
* `pub fn into_mutation_buffer(self) -> CompactArenaSet` — takes it, for handing to `FstConfigOptions::with_initial_buffer`.
* `impl From<CompactArenaSet> for SerializableFstBuffers`
The field is private. The serialized shape is unchanged: one key, `mutation_buffer`.
Implements `Debug`, `Clone`, `Default`, `PartialEq`, `Eq`, and with the `serde` feature `Serialize` and `Deserialize`.
### CompactArenaSet
The sorted mutation buffer: keys stored as spans into one contiguous byte vector, each carrying a tombstone flag. Exposed so it can be serialized and handed back; constructing one directly is not the normal path.
* `pub fn new() -> Self`
* `pub fn contains(&self, key: &[u8]) -> bool` — true only if present and not tombstoned.
* `pub fn insert(&mut self, key: &[u8]) -> bool` — true if the state changed, including resurrecting a tombstone.
* `pub fn remove(&mut self, key: &[u8]) -> bool` — true if the state changed. Inserts a tombstone if the key was absent.
* `pub fn get_status(&self, key: &[u8]) -> Option<bool>` — `Some(true)` live, `Some(false)` tombstoned, `None` absent.
* `pub fn len(&self) -> usize` — live entries, excluding tombstones. O(n).
* `pub fn len_raw(&self) -> usize` — all entries including tombstones. O(1).
* `pub fn is_empty(&self) -> bool` — whether any live entry exists. O(n).
* `pub fn clear(&mut self)` — drops all entries and releases the arena.
* `pub fn iter(&self) -> ArenaIter<'_>` — live keys in sorted order.
* `pub fn iter_raw(&self) -> ArenaRawIter<'_>` — every entry as `(&[u8], is_tombstone)` in sorted order.
* `pub fn size_in_bytes(&self) -> usize` — arena bytes plus span capacity. Excludes the struct itself.
Keys are limited to `u32::MAX` bytes of offset and length, since spans are `u32` pairs.
Implements `Debug`, `Clone`, `Default`, `PartialEq`, `Eq`, `FromIterator<T: AsRef<[u8]>>`, and with the `serde` feature `Serialize` and `Deserialize`.
## 5. Streaming
### MergedSetStreamOwner
A sorted, deduplicated view over the persisted component merged with the mutation buffer, returned by `IncrementalFstSet::stream`. Tombstoned keys are skipped; buffer entries win over persisted ones on a tie.
* `impl Streamer<'_> for MergedSetStreamOwner<'_>`, with `type Item = Vec<u8>` and `fn next(&mut self) -> Option<Vec<u8>>`.
`fst::Streamer` must be in scope to call `next`. This is not an `Iterator`. The stream holds a read lock on the set for as long as it lives, so writers block until it is dropped.
## 6. Metrics
### FstMetricsSnapshot
Cumulative rebuild counters, read via `get_metrics`. Every field is a running total over the life of the set, never reset. `#[non_exhaustive]`.
* `pub num_rebuilds: usize` — rebuilds completed. Deferred rebuilds are not counted until they run.
* `pub add_buffer_items_at_rebuild_sum: usize` — sum of live buffer entries observed at the start of each rebuild.
* `pub del_buffer_items_at_rebuild_sum: usize` — sum of tombstoned buffer entries observed at the start of each rebuild.
* `pub rebuild_duration_micros_sum: u64` — total wall time spent rebuilding, in microseconds.
* `pub persisted_keys_at_rebuild_sum: usize` — sum of persisted key counts observed at the start of each rebuild.
Divide any sum by `num_rebuilds` for a per-rebuild average. Implements `Debug`, `Clone`, `Copy`.
## 7. Re-exports
* `pub use fst;` — the `fst` crate, so callers can name `fst::Automaton` for `search` and bring `fst::Streamer` into scope for `stream` without declaring their own dependency. The version resolved here is the one this crate was built against.
## 8. Error Handling
### IncrementalFstError
`#[non_exhaustive]`: a match on it requires a wildcard arm.
* `Fst(fst::Error)` — bytes that do not parse as an FST, or a failure from the FST builder during a rebuild. `#[from] fst::Error`.
* `Io(std::io::Error)` — opening or mapping the FST file, or writing the temporary file a rebuild streams into. `#[from] std::io::Error`.
Implements `std::error::Error` and `Display`.
### Result
* `pub type Result<T> = std::result::Result<T, IncrementalFstError>;`