Crate netabase_store

Crate netabase_store 

Source
Expand description

§Netabase Store - Type-Safe Multi-Backend Key-Value Storage

netabase_store is a type-safe, multi-backend key-value storage library that provides a unified API across different database backends (Sled, Redb, IndexedDB). It uses procedural macros to generate type-safe schemas and supports primary and secondary key indexing.

§Key Features

  • Multi-Backend Support: Choose between Sled (embedded DB), Redb (ACID compliant), or IndexedDB (browser)
  • Type-Safe Schema: Derive macros generate compile-time checked schemas
  • Primary & Secondary Keys: Efficient indexing with automatic secondary key management
  • Cross-Platform: Supports both native and WASM targets
  • Zero-Copy Operations: Efficient serialization with bincode
  • libp2p Integration: Optional integration for distributed systems

§Quick Start

§Define Your Data Models

use netabase_store::{netabase_definition_module, NetabaseModel};

// Define your schema with the definition module macro
#[netabase_definition_module(BlogDefinition, BlogKeys)]
mod blog {
    use super::*;
    use netabase_store::{netabase, NetabaseModel};

    /// User model with primary and secondary keys
    #[derive(
        NetabaseModel,
        Clone,
        Debug,
        PartialEq,
        bincode::Encode,
        bincode::Decode,
        serde::Serialize,
        serde::Deserialize,
    )]
    #[netabase(BlogDefinition)]
    pub struct User {
        #[primary_key]
        pub id: u64,
        pub username: String,
        #[secondary_key]
        pub email: String,
        pub age: u32,
    }

    /// Post model associated with the same definition
    #[derive(
        NetabaseModel,
        Clone,
        Debug,
        PartialEq,
        bincode::Encode,
        bincode::Decode,
        serde::Serialize,
        serde::Deserialize,
    )]
    #[netabase(BlogDefinition)]
    pub struct Post {
        #[primary_key]
        pub id: String,
        pub title: String,
        pub author_id: u64,
        #[secondary_key]
        pub published: bool,
    }
}  // end mod blog

use blog::*;

§Using Sled Backend (Native)

use netabase_store::{netabase_definition_module, NetabaseModel, netabase};
use netabase_store::databases::sled_store::SledStore;
use netabase_store::traits::tree::NetabaseTreeSync;
use netabase_store::traits::model::NetabaseModelTrait;

// Define schema for this example
#[netabase_definition_module(BlogDefinition, BlogKeys)]
mod blog {
    use super::*;
    use netabase_store::{NetabaseModel, netabase};

    #[derive(NetabaseModel, Clone, Debug, PartialEq,
             bincode::Encode, bincode::Decode,
             serde::Serialize, serde::Deserialize)]
    #[netabase(BlogDefinition)]
    pub struct User {
        #[primary_key]
        pub id: u64,
        pub username: String,
        #[secondary_key]
        pub email: String,
        pub age: u32,
    }
}
use blog::*;

// Open a database
let store = SledStore::<BlogDefinition>::temp().unwrap();

// Get a type-safe tree for the User model
let user_tree = store.open_tree::<User>();

// Create a user
let alice = User {
    id: 1,
    username: "alice".to_string(),
    email: "alice@example.com".to_string(),
    age: 30,
};

// Insert the user
user_tree.put(alice.clone()).unwrap();

// Retrieve by primary key
let retrieved = user_tree.get(alice.primary_key()).unwrap().unwrap();
assert_eq!(retrieved, alice);

// Query by secondary key (email)
let users_by_email = user_tree
    .get_by_secondary_key(alice.secondary_keys()[0].clone())
    .unwrap();
assert_eq!(users_by_email.len(), 1);

// Update the user
let mut alice_updated = alice.clone();
alice_updated.age = 31;
user_tree.put(alice_updated).unwrap();

// Remove the user
user_tree.remove(alice.primary_key()).unwrap();

§Using Redb Backend (Native)

use netabase_store::{netabase_definition_module, NetabaseModel, netabase};

// Define your schema with the definition module macro
#[netabase_definition_module(BlogDefinition, BlogKeys)]
mod blog {
    use super::*;
    use netabase_store::{netabase, NetabaseModel};

    /// User model with primary and secondary keys
    #[derive(
        NetabaseModel,
        Clone,
        Debug,
        PartialEq,
        bincode::Encode,
        bincode::Decode,
        serde::Serialize,
        serde::Deserialize,
    )]
    #[netabase(BlogDefinition)]
    pub struct User {
        #[primary_key]
        pub id: u64,
        pub username: String,
        #[secondary_key]
        pub email: String,
        pub age: u32,
    }

    /// Post model associated with the same definition
    #[derive(
        NetabaseModel,
        Clone,
        Debug,
        PartialEq,
        bincode::Encode,
        bincode::Decode,
        serde::Serialize,
        serde::Deserialize,
    )]
    #[netabase(BlogDefinition)]
    pub struct Post {
        #[primary_key]
        pub id: String,
        pub title: String,
        pub author_id: u64,
        #[secondary_key]
        pub published: bool,
    }
}  // end mod blog

use blog::*;
use netabase_store::databases::redb_store::RedbStore;
use netabase_store::traits::tree::NetabaseTreeSync;

// Open a database with Redb backend
let store = RedbStore::<BlogDefinition>::new("./my_redb_database").unwrap();

// API is identical to SledStore
let user_tree = store.open_tree::<User>();
// ... same operations as Sled

§Using IndexedDB Backend (WASM)

// Open a database in the browser
let store = IndexedDBStore::<BlogDefinition>::new("my_app_db").await.unwrap();

// Get an async tree (WASM uses async API)
let user_tree = store.open_tree::<User>().await.unwrap();

// Create a user
let alice = User { id: 1, username: "alice".into(), email: "alice@example.com".into() };

// Async operations
user_tree.put(alice.clone()).await.unwrap();
let retrieved = user_tree.get(alice.primary_key()).await.unwrap().unwrap();

§Architecture

§Core Components

  1. Definition Module: Groups related models into a schema

    • Created with #[netabase_definition_module] macro
    • Generates an enum containing all models
    • Generates a keys enum for type-safe queries

  2. Models: Individual data structures

    • Derived with #[derive(NetabaseModel)]
    • Must have one #[primary_key]
    • Can have multiple #[secondary_key] fields

  3. Storage Backends:

    • SledStore: Fast embedded database, native only
    • RedbStore: ACID-compliant embedded DB, native only
    • IndexedDBStore: Browser storage, WASM only

  4. Traits:

    • NetabaseTreeSync: Synchronous CRUD operations (native)
    • NetabaseTreeAsync: Asynchronous CRUD operations (WASM)
    • NetabaseModelTrait: Core model trait
    • NetabaseDefinitionTrait: Schema trait

§Secondary Key Queries

Secondary keys enable efficient querying by non-primary fields:

use netabase_store::{netabase_definition_module, NetabaseModel, netabase};
use netabase_store::databases::sled_store::SledStore;
use netabase_store::traits::tree::NetabaseTreeSync;
use netabase_store::traits::model::NetabaseModelTrait;

#[netabase_definition_module(BlogDefinition, BlogKeys)]
mod blog {
    use super::*;
    use netabase_store::{NetabaseModel, netabase};
    #[derive(NetabaseModel, Clone, Debug, PartialEq,
             bincode::Encode, bincode::Decode,
             serde::Serialize, serde::Deserialize)]
    #[netabase(BlogDefinition)]
    pub struct User {
        #[primary_key]
        pub id: u64,
        pub username: String,
        #[secondary_key]
        pub email: String,
    }
}
use blog::*;

let store = SledStore::<BlogDefinition>::temp().unwrap();
let user_tree = store.open_tree::<User>();

let user = User { id: 1, username: "alice".into(), email: "alice@example.com".into() };
user_tree.put(user.clone()).unwrap();

// Query by email (secondary key)
let users = user_tree
    .get_by_secondary_key(user.secondary_keys()[0].clone())
    .unwrap();

// Multiple users can have the same secondary key value
for user in users {
    println!("Found user: {}", user.username);
}

§Iteration

Iterate over all records in a tree:

use netabase_store::{netabase_definition_module, NetabaseModel, netabase};
use netabase_store::databases::sled_store::SledStore;

#[netabase_definition_module(BlogDefinition, BlogKeys)]
mod blog {
    use super::*;
    use netabase_store::{NetabaseModel, netabase};
    #[derive(NetabaseModel, Clone, Debug, PartialEq,
             bincode::Encode, bincode::Decode,
             serde::Serialize, serde::Deserialize)]
    #[netabase(BlogDefinition)]
    pub struct User {
        #[primary_key]
        pub id: u64,
        pub username: String,
        #[secondary_key]
        pub email: String,
    }
}
use blog::*;

let store = SledStore::<BlogDefinition>::temp().unwrap();
let user_tree = store.open_tree::<User>();

// Iterate over all users
for result in user_tree.iter() {
    let (_key, user) = result.unwrap();
    println!("User: {} ({})", user.username, user.email);
}

§Testing

Use temporary databases for testing:

use netabase_store::{netabase_definition_module, NetabaseModel, netabase};
use netabase_store::databases::sled_store::SledStore;
use netabase_store::traits::tree::NetabaseTreeSync;

#[netabase_definition_module(BlogDefinition, BlogKeys)]
mod blog {
    use super::*;
    use netabase_store::{NetabaseModel, netabase};
    #[derive(NetabaseModel, Clone, Debug, PartialEq,
             bincode::Encode, bincode::Decode,
             serde::Serialize, serde::Deserialize)]
    #[netabase(BlogDefinition)]
    pub struct User {
        #[primary_key]
        pub id: u64,
        pub username: String,
    }
}
use blog::*;

// Create an database in tmp database for testing
let store = SledStore::<BlogDefinition>::temp().unwrap();
let user_tree = store.open_tree::<User>();

// Perform test operations
let test_user = User { id: 1, username: "test".into() };
user_tree.put(test_user).unwrap();
// ... assertions

§libp2p Integration

When using the libp2p feature, stores can be used as record stores for Kademlia DHT. This was designed to to be implementned with the netabase crate, which (eventually) should be a dht networking layer abstraction. The stores in the databases module implement RecordStore, which allow for the libp2p’s kademlia implementation.

§Feature Flags

  • sled - Enable Sled backend (default, native only)
  • redb - Enable Redb backend (native only)
  • wasm - Enable IndexedDB backend (WASM only)
  • libp2p - Enable libp2p RecordStore integration

§Error Handling

All operations return Result<T, NetabaseError>:

use netabase_store::{netabase_definition_module, NetabaseModel, netabase};
use netabase_store::databases::sled_store::SledStore;
use netabase_store::traits::tree::NetabaseTreeSync;
use netabase_store::traits::model::NetabaseModelTrait;

#[netabase_definition_module(BlogDefinition, BlogKeys)]
mod blog {
    use super::*;
    use netabase_store::{NetabaseModel, netabase};
    #[derive(NetabaseModel, Clone, Debug, PartialEq,
             bincode::Encode, bincode::Decode,
             serde::Serialize, serde::Deserialize)]
    #[netabase(BlogDefinition)]
    pub struct User {
        #[primary_key]
        pub id: u64,
        pub username: String,
    }
}
use blog::*;

let store = SledStore::<BlogDefinition>::temp().unwrap();
let user_tree = store.open_tree::<User>();

let user = User { id: 1, username: "alice".into() };
match user_tree.get(user.primary_key()) {
    Ok(Some(user)) => println!("Found: {}", user.username),
    Ok(None) => println!("User not found"),
    Err(e) => eprintln!("Database error: {}", e),
}

Re-exports§

pub use netabase_deps;
pub use traits::*;

Modules§

__private
Re-exports of all dependencies used by generated macro code. This ensures macro hygiene - users don’t need to manually import these dependencies. Private re-exports for macro hygiene. Do not use directly.
databases
error
traits

Traits§

DiscriminantBounds
MaybeSend
MaybeSync

Attribute Macros§

netabase
Attribute macro that marks a struct as part of a netabase definition.
netabase_definition_module
Groups multiple models into a unified database schema (definition).

Derive Macros§

NetabaseModel
Derives the NetabaseModelTrait for a struct, enabling it to be stored in netabase.