Skip to main content

Crate memfork_core

Crate memfork_core 

Source
Expand description

The MemFork engine: branchable in-memory state.

An embedded database for the memory an agent accumulates while it works. Store and recall values, then fork the whole database before a risky step, merge the fork if the attempt worked, discard it if it did not, and rewind to how things were at any earlier point.

Forking is constant time and constant space whatever the database holds, because branches share structure rather than copying it. Nothing here uses the operating system’s fork(), copy-on-write pages or a filesystem snapshot: branching is persistent data structures and nothing else, which is why it behaves identically on Windows, macOS and Linux.

This crate is the engine on its own — no I/O, no async, no network. The memfork crate builds an MCP server, a shared daemon and a command line on top of it. The project README has the wider picture: https://github.com/memforkdb/memfork.

§Example

use memfork_core::{Db, Value};

let db = Db::new();
db.put("main", "note:1", Value::new("the plan so far"))?;

// Fork before doing something that might not work out.
db.fork("main", "attempt")?;
db.put("attempt", "note:1", Value::new("a risky rewrite"))?;

// The parent branch cannot see the attempt.
let on_main = db.get("main", "note:1")?;
assert_eq!(on_main.map(|e| e.value.clone()), Some("the plan so far".into()));

// It did not work out: throw the whole attempt away.
db.discard("attempt")?;

Swap Db::discard for Db::merge and the change comes back to the parent instead, resolved key by key against the point the branches diverged.

§What it guarantees

  • Forking costs the same whatever it holds. Db::fork copies a pointer, and the two branches share everything they have in common until one of them changes.
  • A fork is invisible until it is merged. Writes on a branch cannot be seen from the branch it came from, and Db::discard leaves that branch exactly as it was.
  • You can read the past. Db::at reads a branch as of any retained commit, and Db::fork_at branches from there.
  • A dropped transaction leaves nothing behind. A Txn that is not committed moves no head, writes no commit and advances no counter.
  • The same operations give the same answers everywhere. Commit ids are blake3(parents ‖ message ‖ canonically encoded ops), and search breaks ties on the key, so ids and orderings match across machines and runs.

§Where the data goes

Nowhere, unless you ask. This crate keeps everything in memory and performs no I/O of its own; durability is a Journal the caller supplies, which is how the memfork binary writes a checksummed log without the engine ever touching a file. Long-lived memory also needs a limit, so Db::enforce_budget evicts by importance and recency together and records what it dropped as Op::Evict rather than as an ordinary delete.

The design is documented in docs/DESIGN.md.

Re-exports§

pub use journal::DecodeError;
pub use journal::Journal;
pub use journal::JournalError;
pub use journal::Record;

Modules§

journal
The write-ahead journal seam (DESIGN §4.5).

Structs§

BranchInfo
Summary of one branch (DESIGN §4.1).
Change
One entry of a diff, sorted by key with the rest of the diff.
Commit
One immutable point in a branch’s history.
CommitId
A BLAKE3-derived, content-addressed commit id (DESIGN §4.1).
Db
An embedded, in-memory database with Git semantics.
Discarded
A branch that was discarded: what is left to say about it once its commits are gone.
Entry
One stored record.
Evicted
A key chosen for eviction, and why.
EvictionConfig
When to evict, and what to keep.
LogEntry
One line of log output (DESIGN §4.2).
MergeOutcome
The result of a successful merge.
ReadView
A consistent, immutable view of a branch at one commit.
SearchHit
One search result.
Txn
A set of writes that become visible all at once, or not at all.
Value
The caller-supplied half of a write: everything except the logical clock.

Enums§

ChangeKind
How a key differs between two commits (DESIGN §4.2).
Error
Every fallible engine operation returns this error.
MergeKind
How a merge resolved.
MergePolicy
What to do with a key both branches changed (DESIGN §4.2).
Op
One change recorded in a commit. The ops of a commit are its change set and, taken together across history, the append-only event log (DESIGN §4.1).

Constants§

DEFAULT_BRANCH
The default branch name.
DEFAULT_IMPORTANCE
Default importance for an entry that does not specify one (DESIGN §4.1).
DISCARDS_REMEMBERED
How many discarded branches Db::discarded remembers.
MAX_KEY_BYTES
Longest permitted key, in bytes (DESIGN §4.1).
VERSION
The version of this crate.
WRITTEN_BY
The metadata key that records who wrote an entry (DESIGN §4.1).

Traits§

Store
A persistent ordered map from key to entry.

Functions§

cosine
Cosine similarity of two equal-length vectors.
entry_size
Approximate stored size of one entry, in bytes.
score
What an entry is worth keeping, per DESIGN §4.4: importance * 0.5^((head_seq - last_access_seq) / half_life).
validate_branch_name
Branch names are 1-255 bytes of printable, non-whitespace UTF-8.
validate_key
Validate a key against the rules in DESIGN §4.1.
with_txn
Convenience: run a closure in a transaction, retrying on optimistic conflicts, and commit it.

Type Aliases§

OnEvict
Called with the entries about to be evicted, before they go.
Result
Convenience alias.
Root
The persistent ordered map used by the engine.