Crate plugmem_host

Expand description

Native host layer for the plugmem engine: file-backed storage with exclusive locking, a thread-safe database handle with a maintenance policy, and embedding providers.

This crate is the “point it at a file and go” Rust experience:

use plugmem_host::{Config, Database, RecallQuery, RememberInput};

let (db, _report) = Database::open("agent.plugmem", Config::default())?;
db.remember(RememberInput::text(1_784_000_000_000, "prefers tokio"))?;
let out = db.recall(RecallQuery::text(1_784_000_100_000, "runtime?"))?;
println!("{}", out.rendered);

Concurrency model: one file has one owning process — a second open is refused with HostError::Locked; within the process, clone the Database handle across threads and agents; different files are fully independent. Embedding calls run outside the database lock.

With the default config feature, the host also accepts the shared config.toml format through Settings::load and read_config. The SETTINGS.md file is the reference documentation for that format, not an input file. Disable the feature with default-features = false when a caller wants only programmatic Config construction.

Modules§

fact_flags: Bit flags of FactRecord::flags.

Structs§

Config: Full engine configuration with the defaults.
Database: A clonable, thread-safe handle to one database file. See the module docs for the concurrency model.
DatabaseBuilder: Tuning knobs of a Database. Construct through Database::builder.
EntityId: Identifier of an entity — a graph node created lazily on first mention.
ExportedFact: One exported fact — the human-readable, id-free shape Database::export dumps and an importer re-remembers. Internal ids and recorded_at are the engine’s bookkeeping and are not preserved across a round-trip; the knowledge itself (text, subject name, tags, validity start) is.
FactId: Identifier of a fact — the unit of memory.
FactRecord: The unit of memory: one fact (48-byte slot, Uniform arena).
FactSnapshot: An owned view of one fact — Memory::get returns borrows that cannot cross the lock, so the database hands out copies.
FileScratch: A host Scratch over a temp file (milestone H): sequential appends go through a buffered writer; freeze flushes and memory-maps the file, so the staged pool is read (randomly and sequentially) straight from the map instead of RAM. Dropping it unmaps and deletes the temp file.
FileStorage: File-backed Storage holding an advisory lock on its database.
LinkInput: Input of link.
MaintainReport: Report of a maintain pass.
MemScratch: In-memory Scratch: a growable buffer that “freezes” to a borrow of itself. The reference implementation — it backs tests, and because it drives the streaming path with no files it is the vehicle for the disk_first == in_memory property test. It stages in RAM, so it does not save RAM; a host Scratch over a temp file is what bounds RAM in practice.
NullEmbedder: The no-op embedder: dimension 0, never called by the database (a structural-only memory).
OpenAiCompatEmbedder: An /v1/embeddings client for any OpenAI-compatible server.
OpenReport: Report of an open: what the journal replay found.
ReadOnlyDatabase: A read-only database handle backed by a memory-mapped snapshot See the module docs. Send + Sync — share it across threads behind a reference or an Arc.
RecallQuery: A recall request. Default-like construction via RecallQuery::text plus field overrides.
RecallResult: A recall response. Reusable: pass to Memory::recall_into repeatedly and the buffers are recycled.
RecallScratch: Reusable recall scratch — caller-owned, so Memory::recall and Memory::recall_into take &self: many readers can recall the same engine at once, each threading its own scratch (the host wraps one per thread). Carries every buffer a recall mutates — the query-side term/score vectors, the fusion map, and its own tokenizer and name-normalization buffer — so a recall never touches the engine’s write-side scratches. Reused across calls it upholds the zero-alloc invariant.
RecalledEdge: One edge the graph source walked (: agents want the relations, not only the facts).
RecalledFact: One recalled fact.
RecoverReport: The outcome of a Database::recover salvage.
RememberInput: Input of remember and revise.
RememberOutcome: Result of remember/revise.
Scrub: A resumable, byte-level container scrub over a memory-mapped snapshot (— the ZFS-scrub model). Obtained from ReadOnlyDatabase::scrub.
ScrubProgress: Progress of an in-flight ScrubCursor scan.
SettingDoc: Documentation for one supported config.toml key.
Settings: Resolved runtime settings: the engine config, an optional embedder, and the maintenance policy. The wrapper-specific knobs (import batch size, server workers) are read separately from the same read_config table.
SettingsHelp: Runtime access to the complete config.toml help catalogue.
Similar: One similar-fact hint.
Stats: Engine size counters. Every field is an O(1) read; the struct is #[non_exhaustive] so later stages (database identity markers, HNSW state) can extend it without a breaking change.

Enums§

Error: Every way an engine call can fail.
FsyncPolicy: When journal appends reach the disk.
HostError: Every way the host layer can fail. Engine failures pass through as HostError::Engine; everything filesystem- or network-shaped is typed here.
SettingScope: Which runtime surface owns a setting.
SettingsError: A configuration error: malformed TOML, a bad [engine] value, or an [embedder] section missing a required field. Distinct from HostError (which covers opening the database once settings are resolved).
SimilarReason: Why a fact was flagged as similar.

Constants§

DEFAULT_SCRUB_BUDGET: Default number of bytes a ScrubCursor hashes per next() slice The scrub slice-size bench sweeps 64 KiB…64 MiB and finds throughput essentially flat (the scan is xxh3/memory-bandwidth-bound, not per-slice-overhead-bound), so the budget is a pacing quantum, not a throughput knob: 1 MiB is ~sub-millisecond per slice — fine-grained enough to pause, cancel or report progress smoothly, with negligible overhead.
VALID_TO_OPEN: valid_to value of an open fact (“true now”).

Traits§

Embedder: Turns texts into embedding vectors. Batched by design — providers price and perform far better on batches.
Scratch: An append-only staging area: build it sequentially with write, then freeze it into a readable byte view for random and sequential reads. Dropping it discards the storage.

Functions§

default_config_dir: The platform’s conventional per-user config directory.
default_config_path: The default config file path, when a platform user directory is available.
default_data_dir: The platform’s conventional per-user data directory.
default_database_path: The default persistent per-user database path.
read_config: Reads and parses config.toml, or Ok(None) if none applies. An explicit flag path must exist (a read error is a usage error); otherwise $PLUGMEM_CONFIG, then the platform path from crate::default_config_path, are read only if present. Wrappers call this once, then pass the table to Settings::from_table and also read their own keys (batch size, workers) from it.
settings_help: Returns the shared settings catalogue used by host, CLI, MCP and NAPI.