Crate sochdb_core 

SochDB Core

TOON-native ACID database - fundamental types and data structures.

SochDB is to TOON what MongoDB is to JSON - a database that natively understands and stores TOON (Tabular Object-Oriented Notation) documents.

Core Components

• TOON Format: Native data format with schema support
• Transaction Manager: ACID transaction support via WAL
• Virtual Filesystem: POSIX-like VFS backed by WAL
• Schema Catalog: Table and index metadata management

Memory Allocation

Recommended: Enable jemalloc for production workloads:

sochdb-core = { version = "...", features = ["jemalloc"] }

jemalloc provides:

• Thread-local caching (no lock contention)
• Superior fragmentation handling
• Automatic memory return to OS
• Battle-tested in production (Firefox, Redis, RocksDB)

Note: The buddy_allocator module is deprecated. See its documentation for migration guidance.

Features

• jemalloc - Use jemalloc as the global allocator for better performance

Example

use sochdb_core::soch::{SochSchema, SochType, SochTable, SochRow, SochValue};

// Define a schema
let schema = SochSchema::new("users")
    .field("id", SochType::UInt)
    .field("name", SochType::Text)
    .field("email", SochType::Text)
    .primary_key("id");

// Create a table
let mut table = SochTable::new(schema);
table.push(SochRow::new(vec![
    SochValue::UInt(1),
    SochValue::Text("Alice".into()),
    SochValue::Text("alice@example.com".into()),
]));

// Format as TOON
println!("{}", table.format());
// Output: users[1]{id,name,email}:
//         1,Alice,alice@example.com

Re-exports

pub use block_storage::BlockCompression;

pub use block_storage::BlockRef;

pub use block_storage::BlockStore;

pub use block_storage::BlockStoreStats;

pub use block_storage::FileBlockManager;

pub use catalog::Catalog;

pub use catalog::CatalogEntry;

pub use catalog::CatalogEntryType;

pub use catalog::McpToolDescriptor;

pub use catalog::OperationImpl;

pub use columnar::ColumnChunk;

pub use columnar::ColumnStats;

pub use columnar::ColumnType as ColumnarColumnType;

pub use columnar::ColumnValue as ColumnarColumnValue;

pub use columnar::ColumnarStore;

pub use columnar::ColumnarTable;

pub use columnar::MemoryComparison;

pub use columnar::TypedColumn;

pub use columnar::ValidityBitmap;

pub use error::Result;

pub use error::SochDBError;

pub use key::CausalKey;

pub use key::TemporalKey;

pub use learned_index::LearnedSparseIndex;

pub use lockfree_interner::InternerStats;

pub use lockfree_interner::LockFreeInterner;

pub use lockfree_interner::Symbol;

pub use memory_schema::Entity;

pub use memory_schema::EntityFacts;

pub use memory_schema::EntityKind;

pub use memory_schema::EntitySearchResult;

pub use memory_schema::Episode;

pub use memory_schema::EpisodeSearchResult;

pub use memory_schema::EpisodeType;

pub use memory_schema::Event;

pub use memory_schema::EventMetrics;

pub use memory_schema::EventRole;

pub use memory_schema::MemoryStore;

pub use memory_schema::TableRole;

pub use memory_schema::TableSemanticMetadata;

pub use path_trie::ColumnGroupAffinity;

pub use path_trie::ColumnType as PathTrieColumnType;

pub use path_trie::PathTrie;

pub use path_trie::TrieNode;

pub use predefined_views::ViewDefinition;

pub use predefined_views::build_view_map;

pub use predefined_views::get_predefined_views;

pub use predefined_views::get_view;

pub use predefined_views::naming;

pub use soch::SochField;

pub use soch::SochIndex;

pub use soch::SochRow;

pub use soch::SochSchema;

pub use soch::SochTable;

pub use soch::SochType;

pub use soch::SochValue;

pub use soch_codec::SochDbBinaryCodec;

pub use soch_codec::SochDocument;

pub use soch_codec::SochParseError;

pub use soch_codec::SochTextEncoder;

pub use soch_codec::SochTextParser;

pub use soch_codec::SochTokenCounter;

pub use sochfs_metadata::DirEntryRow;

pub use sochfs_metadata::FsMetadataStore;

pub use sochfs_metadata::FsWalOp;

pub use sochfs_metadata::InodeRow;

pub use sochfs_metadata::SochFS;

pub use txn::AriesCheckpointData;

pub use txn::AriesDirtyPageEntry;

pub use txn::AriesTransactionEntry;

pub use txn::IsolationLevel;

pub use txn::Lsn;

pub use txn::PageId;

pub use txn::Transaction;

pub use txn::TransactionManager;

pub use txn::TxnId;

pub use txn::TxnState;

pub use txn::TxnStats;

pub use txn::TxnWalEntry;

pub use txn::TxnWrite;

pub use txn::WalRecordType;

pub use transaction_typestate::Transaction as TypestateTransaction;

pub use transaction_typestate::Active;

pub use transaction_typestate::Committed;

pub use transaction_typestate::Aborted;

pub use transaction_typestate::ReadOnly;

pub use transaction_typestate::ReadWrite;

pub use transaction_typestate::WriteOnly;

pub use transaction_typestate::TransactionStorage;

pub use transaction_typestate::TransactionMode;

pub use version_chain::MvccVersionChain;

pub use version_chain::MvccVersionChainMut;

pub use version_chain::VersionMeta;

pub use version_chain::VisibilityContext;

pub use version_chain::WriteConflictDetection;

pub use vfs::BlockId;

pub use vfs::DirEntry;

pub use vfs::Directory;

pub use vfs::FileStat;

pub use vfs::FileType;

pub use vfs::Inode;

pub use vfs::InodeId;

pub use vfs::Permissions;

pub use vfs::Superblock;

pub use vfs::VfsOp;

Modules

block_storage

File Data Block Storage (Task 12)

buddy_allocatorDeprecated

Buddy Allocator for SochDB Memory Management

catalog

Schema Catalog for SochDB

columnar

True Columnar Storage with Arrow-Compatible Layout

concurrency

Hierarchical Lock Architecture with Epoch-Based Reclamation

epoch_gc

Epoch-Based Garbage Collection for Version Cleanup

error

Error types for SochDB

format_migration

Block Format Migration - Backwards Compatible Format Versioning

key

Key types for SochDB indexing

learned_index

Learned Sparse Index (LSI)

lockfree_interner

Lock-Free String Interner with Chunked Append-Only Storage

memory_schema

Canonical Memory Schema for LLM Agents

path_trie

Path Trie for TOON Document Path Resolution (Task 4)

predefined_views

Predefined SochQL Views (Task 7: Stable Identifiers)

reclamation

Unified Memory Reclamation - Hazard Pointers + Epoch Hybrid

schema_bridge

Schema Bridge - TOON to Columnar Mapping

schema_evolution

Schema Evolution - Online Schema Changes

sharded_block_store

Sharded Block Store (Task 3/7)

soch

TOON (Tabular Object-Oriented Notation) - Native Data Format for SochDB

soch_codec

TOON Format Codec

sochfs_metadata

SochFS Metadata Layer (Task 11)

string_interner

String Interning for Path Segments

tbp

TOON Binary Protocol (TBP) - Zero-Copy Binary Wire Format

transaction_typestate

Type-State Transaction API

txn

Transaction Manager for ACID Transactions

version_chain

Unified MVCC Version Chain Interface

vfs

Virtual Filesystem Types for SochDB

zero_copy

Zero-Copy Iterators - mmap-based Scans

Constants

SCHEMA_VERSION

Current schema version

SOCHDB_MAGIC

Magic bytes for SochDB files

SOCHDB_VERSION

Database version