claw-core

Embedded local storage engine for ClawDB. It provides an async, SQLite-backed core optimized for agent memory workloads: low-latency reads, FTS5 search, transactional consistency, snapshots, and migration-driven schema evolution.

Why claw-core

Local-first durability with SQLite WAL mode by default
Async API built for tokio applications
Full-text retrieval via SQLite FTS5 (memories_fts)
Indexed tag lookup using normalized memory_tags
Snapshot + restore support with checksum manifest
Explicit transactions with deferred staged writes
Built-in pagination primitives for large datasets

Installation

[dependencies]
claw-core = "0.1.1"

Optional encryption feature:

[dependencies]
claw-core = { version = "0.1.1", features = ["encryption"] }

Note: the encryption feature expects a SQLCipher-linked SQLite build.

Quick start

use claw_core::prelude::*;

#[tokio::main]
async fn main() -> claw_core::ClawResult<()> {
  let engine = ClawEngine::open_default().await?;

  let record = MemoryRecord::new(
    "The capital of France is Paris",
    MemoryType::Semantic,
    vec!["geography".to_string(), "europe".to_string()],
    None,
  );
  let id = engine.insert_memory(&record).await?;

  let found = engine.fts_search("France").await?;
  assert!(!found.is_empty());

  let fetched = engine.get_memory(id).await?;
  assert_eq!(fetched.content, record.content);

  engine.close().await;
  Ok(())
}

Core concepts

Memory records

Each MemoryRecord stores:

UUID primary key
textual content
memory type (Semantic, Episodic, Working, ...)
tags
optional TTL
timestamps (created_at, updated_at)

Sessions and tool outputs

Session lifecycle is first-class (start_session, end_session, list_sessions) and tool outputs are tied to sessions for traceability.

Transaction model

ClawTransaction stages writes first, then applies all staged operations on commit in one SQLite transaction. This keeps memories, memories_fts, and memory_tags consistent.

Search and retrieval

fts_search(query): FTS5 query across memory content
search_by_tag(tag): indexed join through memory_tags
get_memory(id): cache-aware retrieval path

Pagination

Use keyset pagination via ListOptions:

use claw_core::{ClawEngine, ListOptions, MemoryType};

# async fn demo(engine: &ClawEngine) -> claw_core::ClawResult<()> {
let (page1, cursor) = engine
  .list_memories_paginated(
    Some(MemoryType::Semantic),
    ListOptions { limit: 100, cursor: None },
  )
  .await?;

let (_page2, _next) = engine
  .list_memories_paginated(
    Some(MemoryType::Semantic),
    ListOptions { limit: 100, cursor },
  )
  .await?;
# Ok(())
# }

Snapshots and restore

snapshot_create() performs PRAGMA wal_checkpoint(FULL) before copy
metadata includes path, timestamp, size, and BLAKE3 checksum
manifest persisted at <snapshot_dir>/manifest.json
restore(path) validates SQLite magic bytes before replacement

Runtime stats

engine.stats() returns:

total memory count
rolling cache hit rate (last 1000 lookups)
last snapshot timestamp (in-process)
db and WAL file sizes

Configuration

use claw_core::{ClawConfig, JournalMode};

let config = ClawConfig::builder()
  .db_path("/var/lib/claw/claw.db")
  .max_connections(8)
  .cache_size_mb(128)
  .journal_mode(JournalMode::WAL)
  .snapshot_dir("/var/lib/claw/snapshots")
  .auto_migrate(true)
  .build()
  .expect("valid config");

Benchmarks

Included benchmark suites cover:

FTS5 search at 1k/10k/100k corpus sizes
tag index query latency
cache hit vs miss read paths
snapshot creation timing
concurrent insert throughput (4 writers)

Run with:

cargo bench

Development

cargo fmt --check
cargo clippy -- -D warnings
cargo test --all-features

License

Licensed under MIT. See LICENSE.

claw-core 0.1.1