Crate netabase_store 

Netabase Store - Type-Safe Multi-Backend Key-Value Storage

netabase_store is a type-safe, multi-backend key-value storage library that provides a unified API across different database backends (Sled, Redb, IndexedDB). It uses procedural macros to generate type-safe schemas and supports primary and secondary key indexing.

Key Features

• Multi-Backend Support: Choose between Sled (embedded DB), Redb (ACID compliant), or IndexedDB (browser)
• Type-Safe Schema: Derive macros generate compile-time checked schemas
• Primary & Secondary Keys: Efficient indexing with automatic secondary key management
• Cross-Platform: Supports both native and WASM targets
• Zero-Copy Operations: Efficient serialization with bincode
• libp2p Integration: Optional integration for distributed systems

Quick Start

Define Your Data Models

use netabase_store::{netabase_definition_module, NetabaseModel};

// Define your schema with the definition module macro
#[netabase_definition_module(BlogDefinition, BlogKeys)]
mod blog {
    use super::*;
    use netabase_store::{netabase, NetabaseModel};

    /// User model with primary and secondary keys
    #[derive(
        NetabaseModel,
        Clone,
        Debug,
        PartialEq,
        bincode::Encode,
        bincode::Decode,
        serde::Serialize,
        serde::Deserialize,
    )]
    #[netabase(BlogDefinition)]
    pub struct User {
        #[primary_key]
        pub id: u64,
        pub username: String,
        #[secondary_key]
        pub email: String,
        pub age: u32,
    }

    /// Post model associated with the same definition
    #[derive(
        NetabaseModel,
        Clone,
        Debug,
        PartialEq,
        bincode::Encode,
        bincode::Decode,
        serde::Serialize,
        serde::Deserialize,
    )]
    #[netabase(BlogDefinition)]
    pub struct Post {
        #[primary_key]
        pub id: String,
        pub title: String,
        pub author_id: u64,
        #[secondary_key]
        pub published: bool,
    }
}  // end mod blog

use blog::*;

Using the Unified NetabaseStore (Recommended)

// Create a store with any backend - Sled example (using temp for doctest)
let store = NetabaseStore::<BlogDefinition, _>::temp()?;

// Or use Redb (in production):
// let store = NetabaseStore::<BlogDefinition, _>::redb("./my_db.redb")?;

// Open a tree for the User model - works with any backend!
let user_tree = store.open_tree::<User>();

// Standard operations work the same across all backends
let alice = User {
    id: 1,
    username: "alice".to_string(),
    email: "alice@example.com".to_string(),
    age: 30,
};

user_tree.put(alice.clone())?;
let retrieved = user_tree.get(alice.primary_key())?.unwrap();
assert_eq!(retrieved, alice);

// Backend-specific features still available
store.flush()?; // Sled-specific

Using Sled Backend (Direct)

use blog::*;

// Open a database
let store = SledStore::<BlogDefinition>::temp().unwrap();

// Get a type-safe tree for the User model
let user_tree = store.open_tree::<User>();

// Create a user
let alice = User {
    id: 1,
    username: "alice".to_string(),
    email: "alice@example.com".to_string(),
    age: 30,
};

// Insert the user
user_tree.put(alice.clone()).unwrap();

// Retrieve by primary key
let retrieved = user_tree.get(alice.primary_key()).unwrap().unwrap();
assert_eq!(retrieved, alice);

// Query by secondary key (email) using convenience function
use blog::AsUserEmail;
let users_by_email = user_tree
    .get_by_secondary_key("alice@example.com".as_user_email_key())
    .unwrap();
assert_eq!(users_by_email.len(), 1);

// Update the user
let mut alice_updated = alice.clone();
alice_updated.age = 31;
user_tree.put(alice_updated).unwrap();

// Remove the user
user_tree.remove(alice.primary_key()).unwrap();

Using Redb Backend (Native)

use netabase_store::{netabase_definition_module, NetabaseModel, netabase};


// Open a database with Redb backend (using temp file for doctest)
let temp_dir = tempfile::tempdir()?;
let db_path = temp_dir.path().join("test.redb");
let store = RedbStore::<BlogDefinition>::new(db_path)?;

// API is identical to SledStore
let user_tree = store.open_tree::<User>();

let user = User {
    id: 1,
    username: "bob".to_string(),
    email: "bob@example.com".to_string(),
    age: 25,
};
user_tree.put(user)?;

Using IndexedDB Backend (WASM)

IndexedDB backend provides persistent storage in web browsers. This example requires the wasm feature and wasm32 target:

async fn wasm_example() -> Result<(), Box<dyn std::error::Error>> {
    // Open a database in the browser (persists across page reloads)
    let store = IndexedDBStore::<AppDefinition>::new("my_app_db").await?;

    // Get an async tree - note WASM uses async API
    let user_tree = store.open_tree::<User>();

    // Create a user
    let alice = User {
        id: 1,
        username: "alice".into(),
        email: "alice@example.com".into()
    };

    // All operations are async in WASM
    user_tree.put(alice.clone()).await?;
    let retrieved = user_tree.get(alice.primary_key()).await?;

    // Query by secondary key using convenience function
    use app::AsUserEmail;
    let users_by_email = user_tree
        .get_by_secondary_key("alice@example.com".as_user_email_key())
        .await?;

    Ok(())
}

Using the Configuration

use netabase_store::traits::backend_store::BackendStore;
// Unified API across all backends via BackendStore trait
let temp_dir = tempfile::tempdir()?;
let config = FileConfig::new(temp_dir.path().join("my_store.db"));
let store = <SledStore<MyDef> as BackendStore<MyDef>>::new(config.clone())?;

// Or open existing
let store = <SledStore<MyDef> as BackendStore<MyDef>>::open(config)?;

// Temporary store
let temp_store = <SledStore<MyDef> as BackendStore<MyDef>>::temp()?;

Batch Operations

For high-performance bulk operations, use batch processing to reduce overhead. This example requires the native feature:

use netabase_store::{netabase_definition_module, NetabaseModel, netabase};
use netabase_store::databases::sled_store::SledStore;
use netabase_store::traits::batch::{Batchable, BatchOperations};
use netabase_store::traits::model::NetabaseModelTrait;

#[netabase_definition_module(AppDefinition, AppKeys)]
mod app {
    use super::*;
    use netabase_store::{NetabaseModel, netabase};

    #[derive(NetabaseModel, Clone, Debug,
             bincode::Encode, bincode::Decode,
             serde::Serialize, serde::Deserialize)]
    #[netabase(AppDefinition)]
    pub struct User {
        #[primary_key]
        pub id: u64,
        pub name: String,
    }
}
use app::*;

let store = SledStore::<AppDefinition>::temp()?;
let user_tree = store.open_tree::<User>();

// Prepare batch of users
let users: Vec<User> = (0..1000)
    .map(|i| User { id: i, name: format!("User {}", i) })
    .collect();

// Bulk insert - much faster than individual puts
user_tree.put_batch(users)?;

Batch operations are atomic and significantly faster than individual operations when working with many records.

Zero-Copy Redb Backend (High Performance)

For maximum performance with the Redb backend, use the zero-copy API that provides explicit transaction control and zero-copy reads.

Enabling Zero-Copy

Add both redb and redb-zerocopy features:

[dependencies]
netabase_store = { version = "*", features = ["redb", "redb-zerocopy"] }

Performance Characteristics

Operation	Regular API	Zero-Copy API	Speedup
Bulk insert (1000 items)	~50ms	~5ms	10x faster
Secondary key query	~5.4ms	~100μs	54x faster
Single read	~100ns	~100ns	Similar

Usage Example

// This example requires the `redb-zerocopy` feature
use netabase_store::{netabase_definition_module, NetabaseModel};
use netabase_store::databases::redb_zerocopy::RedbStoreZeroCopy;
use netabase_store::traits::model::NetabaseModelTrait;

#[netabase_definition_module(AppDef, AppKeys)]
mod app {
    use netabase_store::{NetabaseModel, netabase};
    #[derive(NetabaseModel, Clone, Debug, PartialEq,
             bincode::Encode, bincode::Decode,
             serde::Serialize, serde::Deserialize)]
    #[netabase(AppDef)]
    pub struct User {
        #[primary_key]
        pub id: u64,
        pub name: String,
        #[secondary_key]
        pub email: String,
    }
}
use app::*;

// Create a zero-copy store
let store = RedbStoreZeroCopy::<AppDef>::new(temp_dir.path().join("app.redb"))?;

// Write transaction - batch multiple operations
let mut write_txn = store.begin_write()?;
let mut tree = write_txn.open_tree::<User>()?;

// Batch insert 1000 users in one transaction
for i in 0..1000 {
    tree.put(User {
        id: i,
        name: format!("User {}", i),
        email: format!("user{}@example.com", i),
    })?;
}
drop(tree);
write_txn.commit()?;  // All 1000 inserts committed atomically

// Read transaction - efficient queries
let read_txn = store.begin_read()?;
let tree = read_txn.open_tree::<User>()?;
let user = tree.get(&UserPrimaryKey(42))?.unwrap();
assert_eq!(user.name, "User 42");

When to Use Zero-Copy API

Use zero-copy when:

• You need to batch multiple operations (bulk inserts/updates)
• Performance is critical
• You want explicit transaction control
• You’re doing many secondary key queries

Use regular API when:

• Simplicity is more important than performance
• Single-operation transactions are fine
• You want the simplest possible code

Available Types

When redb-zerocopy feature is enabled, these types are re-exported at the crate root:

• RedbStoreZeroCopy - The zero-copy store
• RedbWriteTransactionZC - Write transaction handle
• RedbReadTransactionZC - Read transaction handle
• RedbTreeMut - Mutable tree for write transactions
• RedbTree - Immutable tree for read transactions
• BorrowedGuard - Guard for zero-copy borrowed data
• BorrowedIter - Iterator for zero-copy iteration

Custom Backend Implementations

Netabase Store provides a unified API through traits, making it easy to implement custom storage backends:

• NetabaseTreeSync: For synchronous backends (native)
• NetabaseTreeAsync: For asynchronous backends (WASM, remote databases)
• OpenTree: For creating tree instances from stores
• Batchable: For batch operation support

Implement these traits to add support for new databases while maintaining compatibility with all existing code using netabase_store.

Architecture

Core Components

1. Definition Module: Groups related models into a schema

   • Created with #[netabase_definition_module] macro
   • Generates an enum containing all models
   • Generates a keys enum for type-safe queries

2. Models: Individual data structures

   • Derived with #[derive(NetabaseModel)]
   • Must have one #[primary_key]
   • Can have multiple #[secondary_key] fields

3. Storage Backends:

   • SledStore: Fast embedded database, native only
   • RedbStore: ACID-compliant embedded DB, native only
   • IndexedDBStore: Browser storage, WASM only

4. Traits:

   • NetabaseTreeSync: Synchronous CRUD operations (native)
   • NetabaseTreeAsync: Asynchronous CRUD operations (WASM)
   • NetabaseModelTrait: Core model trait
   • NetabaseDefinitionTrait: Schema trait

Secondary Key Queries

Secondary keys enable efficient querying by non-primary fields:

use netabase_store::{netabase_definition_module, NetabaseModel, netabase};
use netabase_store::databases::sled_store::SledStore;
use netabase_store::traits::tree::NetabaseTreeSync;
use netabase_store::traits::model::NetabaseModelTrait;

#[netabase_definition_module(BlogDefinition, BlogKeys)]
mod blog {
    use super::*;
    use netabase_store::{NetabaseModel, netabase};
    #[derive(NetabaseModel, Clone, Debug, PartialEq,
             bincode::Encode, bincode::Decode,
             serde::Serialize, serde::Deserialize)]
    #[netabase(BlogDefinition)]
    pub struct User {
        #[primary_key]
        pub id: u64,
        pub username: String,
        #[secondary_key]
        pub email: String,
    }
}
use blog::*;

let store = SledStore::<BlogDefinition>::temp().unwrap();
let user_tree = store.open_tree::<User>();

let user = User { id: 1, username: "alice".into(), email: "alice@example.com".into() };
user_tree.put(user.clone()).unwrap();

// Query by email (secondary key) using convenience function
use blog::AsUserEmail;
let users = user_tree
    .get_by_secondary_key("alice@example.com".as_user_email_key())
    .unwrap();

// Multiple users can have the same secondary key value
for user in users {
    println!("Found user: {}", user.username);
}

Iteration

Iterate over all records in a tree:

use netabase_store::{netabase_definition_module, NetabaseModel, netabase};
use netabase_store::databases::sled_store::SledStore;

#[netabase_definition_module(BlogDefinition, BlogKeys)]
mod blog {
    use super::*;
    use netabase_store::{NetabaseModel, netabase};
    #[derive(NetabaseModel, Clone, Debug, PartialEq,
             bincode::Encode, bincode::Decode,
             serde::Serialize, serde::Deserialize)]
    #[netabase(BlogDefinition)]
    pub struct User {
        #[primary_key]
        pub id: u64,
        pub username: String,
        #[secondary_key]
        pub email: String,
    }
}
use blog::*;

let store = SledStore::<BlogDefinition>::temp().unwrap();
let user_tree = store.open_tree::<User>();

// Iterate over all users
for result in user_tree.iter() {
    let (_key, user) = result.unwrap();
    println!("User: {} ({})", user.username, user.email);
}

Testing

Use temporary databases for testing:

use netabase_store::{netabase_definition_module, NetabaseModel, netabase};
use netabase_store::databases::sled_store::SledStore;
use netabase_store::traits::tree::NetabaseTreeSync;

#[netabase_definition_module(BlogDefinition, BlogKeys)]
mod blog {
    use super::*;
    use netabase_store::{NetabaseModel, netabase};
    #[derive(NetabaseModel, Clone, Debug, PartialEq,
             bincode::Encode, bincode::Decode,
             serde::Serialize, serde::Deserialize)]
    #[netabase(BlogDefinition)]
    pub struct User {
        #[primary_key]
        pub id: u64,
        pub username: String,
    }
}
use blog::*;

// Create an database in tmp database for testing
let store = SledStore::<BlogDefinition>::temp().unwrap();
let user_tree = store.open_tree::<User>();

// Perform test operations
let test_user = User { id: 1, username: "test".into() };
user_tree.put(test_user).unwrap();
// ... assertions

libp2p Integration

When using the libp2p feature, stores can be used as record stores for Kademlia DHT. This was designed to to be implementned with the netabase crate, which (eventually) should be a dht networking layer abstraction. The stores in the databases module implement RecordStore, which allow for the libp2p’s kademlia implementation.

Feature Flags

• sled - Enable Sled backend (default, native only)
• redb - Enable Redb backend (native only)
• wasm - Enable IndexedDB backend (WASM only)
• libp2p - Enable libp2p RecordStore integration

Error Handling

All operations return Result<T, NetabaseError>:

use netabase_store::{netabase_definition_module, NetabaseModel, netabase};
use netabase_store::databases::sled_store::SledStore;
use netabase_store::traits::tree::NetabaseTreeSync;
use netabase_store::traits::model::NetabaseModelTrait;

#[netabase_definition_module(BlogDefinition, BlogKeys)]
mod blog {
    use super::*;
    use netabase_store::{NetabaseModel, netabase};
    #[derive(NetabaseModel, Clone, Debug, PartialEq,
             bincode::Encode, bincode::Decode,
             serde::Serialize, serde::Deserialize)]
    #[netabase(BlogDefinition)]
    pub struct User {
        #[primary_key]
        pub id: u64,
        pub username: String,
    }
}
use blog::*;

let store = SledStore::<BlogDefinition>::temp().unwrap();
let user_tree = store.open_tree::<User>();

let user = User { id: 1, username: "alice".into() };
match user_tree.get(user.primary_key()) {
    Ok(Some(user)) => println!("Found: {}", user.username),
    Ok(None) => println!("User not found"),
    Err(e) => eprintln!("Database error: {}", e),
}

Re-exports

pub use store::NetabaseStore;

pub use subscription::subscription_tree::DefaultSubscriptionManager;

pub use traits::subscription::subscription_tree::ModelHash;

pub use transaction::ReadOnly;

pub use transaction::ReadWrite;

pub use transaction::TreeView;

pub use transaction::TxnGuard;

pub use utils::NetabaseDateTime;

pub use netabase_deps;

pub use utils::chrono;

pub use traits::*;

Modules

__private

Re-exports of all dependencies used by generated macro code. This ensures macro hygiene - users don’t need to manually import these dependencies.

config

Unified configuration system for all database backends.

databases

Database backend implementations for Netabase Store.

error

libp2p

libp2p is a modular peer-to-peer networking framework.

store

Unified store interface providing a single entry point for all storage backends.

subscription

Subscription system for tracking data changes and synchronization

traits

transaction

Transaction API with type-state pattern for compile-time safety.

uniffi

utils

Utility modules for netabase_store

Traits

DiscriminantBounds

MaybeSend

MaybeSync

Attribute Macros

netabase

Attribute macro that marks a struct as part of a netabase definition.

netabase_definition_module

Groups multiple models into a unified database schema (definition).

streams

Marker attribute for subscription streams functionality.

Derive Macros

NetabaseDiscriminant

Derive macro for NetabaseDiscriminant trait.

NetabaseModel

Derives the NetabaseModelTrait for a struct, enabling it to be stored in netabase.