netabase_store 0.0.3

![crates.io](https://img.shields.io/crates/v/netabase_store.svg)
![crates.io downloads](https://img.shields.io/crates/d/netabase_store.svg) ![docs.rs](https://docs.rs/netabase_store/badge.svg)

# Netabase Store

A type-safe, multi-backend key-value storage library for Rust with support for native (Sled, Redb) and WASM (IndexedDB) environments.

> ⚠️ **Early Development**: This crate is still in early development and will change frequently as it stabilizes. It is not advised to use this in a production environment until it stabilizes.

## Installation

Add to your `Cargo.toml`:

```toml
[dependencies]
netabase_store = "0.0.3"

# Required dependencies for macros to work
bincode = { version = "2.0", features = ["serde"] }
serde = { version = "1.0", features = ["derive"] }
strum = { version = "0.27.2", features = ["derive"] }
derive_more = { version = "2.0.1", features = ["from", "try_into", "into"] }
anyhow = "1.0"  # Optional, for error handling

# For WASM support
[target.'cfg(target_arch = "wasm32")'.dependencies]
netabase_store = { version = "0.0.2", default-features = false, features = ["wasm"] }
```

### Feature Flags

- `native` (default): Enable Sled and Redb backends
- `sled`: Enable Sled backend only
- `redb`: Enable Redb backend only
- `redb-zerocopy`: Enable zero-copy Redb backend (high-performance variant)
- `wasm`: Enable IndexedDB backend for WASM
- `libp2p`: Enable libp2p integration
- `record-store`: Enable RecordStore trait (requires `libp2p`)

## Quick Start

### 1. Define Your Schema

```rust
use netabase_store::netabase_definition_module;
use netabase_store::traits::model::NetabaseModelTrait;

#[netabase_definition_module(BlogDefinition, BlogKeys)]
pub mod blog_schema {
    use netabase_store::{NetabaseModel, netabase};

    #[derive(NetabaseModel, bincode::Encode, bincode::Decode, Clone, Debug)]
    #[netabase(BlogDefinition)]
    pub struct User {
        #[primary_key]
        pub id: u64,
        pub username: String,
        #[secondary_key]
        pub email: String,
    }

    #[derive(NetabaseModel, bincode::Encode, bincode::Decode, Clone, Debug)]
    #[netabase(BlogDefinition)]
    pub struct Post {
        #[primary_key]
        pub id: u64,
        pub title: String,
        pub content: String,
        #[secondary_key]
        pub author_id: u64,
    }
}

use blog_schema::*;
```

### 2. Create a Store with Unified Configuration API (Recommended)

The unified configuration system provides consistent backend initialization:

```rust
use netabase_store::config::FileConfig;
use netabase_store::traits::backend_store::BackendStore;
use netabase_store::databases::sled_store::SledStore;

fn main() -> anyhow::Result<()> {
    // Create configuration - same for all file-based backends!
    let config = FileConfig::builder()
        .path("my_app.db".into())
        .cache_size_mb(512)
        .build();

    // Initialize backend with BackendStore trait
    let store = <SledStore<BlogDefinition> as BackendStore<BlogDefinition>>::new(config)?;

    // Switch backends by changing only the type - same config works!
    // let store = <RedbStore<BlogDefinition> as BackendStore<BlogDefinition>>::new(config)?;
    // let store = <RedbStoreZeroCopy<BlogDefinition> as BackendStore<BlogDefinition>>::new(config)?;

    // Open a tree for users - works identically across all backends
    let user_tree = store.open_tree::<User>();

    // Insert a user
    let user = User {
        id: 1,
        username: "alice".to_string(),
        email: "alice@example.com".to_string(),
    };
    user_tree.put(user.clone())?;

    // Get by primary key
    let retrieved = user_tree.get(UserPrimaryKey(1))?.unwrap();
    assert_eq!(retrieved.username, "alice");

    // Query by secondary key
    let users_by_email = user_tree.get_by_secondary_key(
        UserSecondaryKeys::Email(UserEmailSecondaryKey("alice@example.com".to_string()))
    )?;
    assert_eq!(users_by_email.len(), 1);

    // Iterate over all users
    for result in user_tree.iter() {
        let (_key, user) = result?;
        println!("User: {} - {}", user.username, user.email);
    }

    // Access backend-specific features when needed
    store.flush()?; // Sled-specific method

    Ok(())
}
```

### 3. Alternative: Simple Configuration for Quick Setup

For simple use cases, use the convenience constructors:

```rust
use netabase_store::config::FileConfig;
use netabase_store::traits::backend_store::BackendStore;
use netabase_store::databases::sled_store::SledStore;

// Simple constructor with defaults
let config = FileConfig::new("my_app.db");
let store = <SledStore<BlogDefinition> as BackendStore<BlogDefinition>>::open(config)?;

// Or create a temporary database for testing
let store = <SledStore<BlogDefinition> as BackendStore<BlogDefinition>>::temp()?;
```

### 4. Use with IndexedDB (WASM)

```rust
use netabase_store::config::IndexedDBConfig;
use netabase_store::traits::backend_store::BackendStore;
use netabase_store::databases::indexeddb_store::IndexedDBStore;
use netabase_store::traits::tree::NetabaseTreeAsync;

#[cfg(target_arch = "wasm32")]
async fn wasm_example() -> Result<(), Box<dyn std::error::Error>> {
    // Create configuration
    let config = IndexedDBConfig::builder()
        .database_name("my_database".to_string())
        .version(1)
        .build();

    // Initialize IndexedDB store
    let store = <IndexedDBStore<BlogDefinition> as BackendStore<BlogDefinition>>::new(config).await?;

    // Note: WASM uses async API
    let user_tree = store.open_tree::<User>();

    let user = User {
        id: 1,
        username: "charlie".to_string(),
        email: "charlie@example.com".to_string(),
    };

    // All operations are async
    user_tree.put(user.clone()).await?;
    let retrieved = user_tree.get(user.primary_key()).await?;

    Ok(())
}
```

## Advanced Usage

### Configuration API

The new unified configuration system provides consistent backend initialization across all database types:

#### FileConfig - For File-Based Backends

```rust
use netabase_store::config::FileConfig;
use netabase_store::traits::backend_store::BackendStore;
use netabase_store::databases::sled_store::SledStore;

// Method 1: Builder pattern (recommended)
let config = FileConfig::builder()
    .path("app_data.db".into())
    .cache_size_mb(1024)
    .truncate(true)
    .build();

let store = <SledStore<BlogDefinition> as BackendStore<BlogDefinition>>::new(config)?;

// Method 2: Simple constructor
let config = FileConfig::new("app_data.db");
let store = <SledStore<BlogDefinition> as BackendStore<BlogDefinition>>::open(config)?;

// Method 3: Temporary database
let store = <SledStore<BlogDefinition> as BackendStore<BlogDefinition>>::temp()?;
```

#### Switching Backends with Same Config

The power of the configuration API is that you can switch backends without changing your code:

```rust
use netabase_store::config::FileConfig;
use netabase_store::traits::backend_store::BackendStore;

let config = FileConfig::builder()
    .path("my_app.db".into())
    .cache_size_mb(512)
    .build();

// Try different backends - same config!
#[cfg(feature = "sled")]
let store = <SledStore<BlogDefinition> as BackendStore<BlogDefinition>>::new(config.clone())?;

#[cfg(feature = "redb")]
let store = <RedbStore<BlogDefinition> as BackendStore<BlogDefinition>>::new(config.clone())?;

#[cfg(feature = "redb-zerocopy")]
let store = <RedbStoreZeroCopy<BlogDefinition> as BackendStore<BlogDefinition>>::new(config)?;

// All have the same API from this point on!
let user_tree = store.open_tree::<User>();
```

#### Configuration Options Reference

**FileConfig** (for Sled, Redb, RedbZeroCopy):
- `path: PathBuf` - Database file/directory path
- `cache_size_mb: usize` - Cache size in megabytes (default: 256)
- `create_if_missing: bool` - Create if doesn't exist (default: true)
- `truncate: bool` - Delete existing data (default: false)
- `read_only: bool` - Open read-only (default: false)
- `use_fsync: bool` - Fsync for durability (default: true)

**MemoryConfig** (for in-memory backend):
- `capacity: Option<usize>` - Optional capacity hint

**IndexedDBConfig** (for WASM):
- `database_name: String` - IndexedDB database name
- `version: u32` - Schema version (default: 1)

### Batch Operations & Bulk Methods

For high-performance bulk operations, use the convenient bulk methods:

```rust
use netabase_store::config::FileConfig;
use netabase_store::traits::backend_store::BackendStore;
use netabase_store::databases::sled_store::SledStore;

// Create store with temporary database
let store = <SledStore<BlogDefinition> as BackendStore<BlogDefinition>>::temp()?;
let user_tree = store.open_tree::<User>();

// Bulk insert - 8-9x faster than loop!
let users: Vec<User> = (0..1000)
    .map(|i| User {
        id: i,
        username: format!("user{}", i),
        email: format!("user{}@example.com", i),
    })
    .collect();

user_tree.put_many(users)?;  // Single transaction

// Bulk read
let keys: Vec<UserPrimaryKey> = (0..100).map(UserPrimaryKey).collect();
let users: Vec<Option<User>> = user_tree.get_many(keys)?;

// Bulk secondary key queries
let email_keys = vec![
    UserSecondaryKeys::Email(UserEmailSecondaryKey("alice@example.com".to_string())),
    UserSecondaryKeys::Email(UserEmailSecondaryKey("bob@example.com".to_string())),
];
let results: Vec<Vec<User>> = user_tree.get_many_by_secondary_keys(email_keys)?;
```

**Bulk Methods:**
- `put_many(Vec<M>)` - Insert multiple models in one transaction
- `get_many(Vec<M::Keys>)` - Read multiple models in one transaction
- `get_many_by_secondary_keys(Vec<SecondaryKey>)` - Query multiple secondary keys in one transaction

**Or use the batch API for more control:**

```rust
use netabase_store::traits::batch::Batchable;

// Create a batch
let mut batch = user_tree.create_batch()?;

// Add many operations
for i in 0..1000 {
    batch.put(User { /* ... */ })?;
}

// Commit atomically - all or nothing
batch.commit()?;
```


#### 3. Choose the Right API for Your Use Case

| Use Case | Recommended API | Reason |
|----------|----------------|--------|
| Simple CRUD, few operations | Standard wrapper | Simplest API, auto-commit |
| Bulk inserts/reads (100+ items) | Bulk methods | 8-9x faster than loops |
| Complex transactions | Explicit transactions | Full control, atomic commits |
| Read-heavy queries | ZeroCopy API | Up to 54x faster for secondary queries |


#### Serialization Overhead

The read-path overhead in Redb comes from type system limitations with Generic Associated Types (GATs). We prioritize safety over unsafe transmutes. For applications where this matters:
- Use bulk methods to amortize overhead
- Use explicit transactions for better performance
- Consider Sled backend for read-heavy workloads

See benchmark results and visualizations in `docs/benchmarks/` for detailed performance analysis.

### For 1.0.0

- [x] Transaction support across multiple operations (**COMPLETED**)
- [ ] Zero-copy reads for redb backend (via `redb-zerocopy` feature) - **Phase 1 Complete**
- [ ] Allow modules to define more than one definition for flexible organization
- [ ] Migration utilities for schema changes
- [ ] Query builder for complex queries
- [ ] Range queries on ordered keys
- [ ] Compression support
- [ ] Encryption at rest
- [ ] Improved documentation and examples

### Future Plans

- [ ] Distributed systems support with automatic sync
- [ ] CRDT-based conflict resolution
- [ ] WebRTC backend for peer-to-peer storage
- [ ] SQL-like query language
- [ ] GraphQL integration


## Acknowledgments

Built with:
- [Sled](https://github.com/spacejam/sled) - Embedded database
- [Redb](https://github.com/cberner/redb) - Embedded database
- [IndexedDB](https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API) - Browser storage
- [Bincode](https://github.com/bincode-org/bincode) - Binary serialization