wedb_embed 0.1.2

English | 中文

Embedded database engine providing Redis-compatible data structures and APIs, built on the fjall LSM-Tree storage engine.

Why an Embedded Redis Engine

Why an Embedded Redis Engine
Quickstart
- Installation
- Basic Usage
Performance & Resource Comparison
- macOS (Apple M2 Max)
Storage Architecture & Encoding Design
Runtime Architecture & Threading Model
- Thread-per-Core Architecture Design
- Pitfalls of Multi-Threaded Work-Stealing Runtimes
Multi-Tenant & Multi-DB Isolation
Data Structures & Operations
Tech Stack

In backend services, CLI tools, edge computing nodes, and desktop applications, developers frequently require composite data structures such as hash maps, sorted sets, message streams, bitmaps, and time-series collections. The conventional approach involves running an external Redis daemon and communicating over network or Unix domain sockets. In standalone and embedded environments, this architecture incurs specific performance and resource constraints:

IPC and Protocol Serialization Overhead: Every read or write operation traverses socket buffers, triggers OS context switches, and requires RESP protocol encoding and decoding within an event loop. Even on localhost, round-trip latency typically remains in the 20–50 microsecond range while consuming CPU cycles.
RAM Costs and Memory Limits: Redis keeps all datasets and pointer-heavy metadata resident in physical RAM. As dataset volume expands to tens of gigabytes, memory hardware costs escalate and remain strictly bounded by host RAM capacity. Background AOF/RDB persistence can further double memory usage via Copy-On-Write mechanisms.
Deployment and Operational Overhead: Managing external daemon processes requires process supervisors, port allocation, configuration syncing, and health monitoring.

wedb_embed integrates the storage engine directly into the application process, transforming the data access path:

In-Process Direct Invocation: Data operations execute directly via Rust function calls in memory, eliminating socket I/O, syscalls, and inter-process context switches. P95 latency for core commands is reduced from tens of microseconds to nanosecond and microsecond ranges.
LSM-Tree Disk Persistence with Tiered Storage: Built on an LSM-Tree engine with LZ4 block compression, hot data resides in memory buffers while full datasets persist compressed on disk. In a 2GB structured dataset benchmark, resident memory (RSS) dropped from 1951 MB (Redis) to 234 MB (an 88% reduction), while disk footprint was reduced by 38%.
Comprehensive Redis Data Models: Provides 16 composite data models built atop the key-value core (String, Hash with field-level TTL, List, Set, ZSet, Bitmap, JSON, Bloom/Cuckoo Filters, TimeSeries, Geo, HyperLogLog, TDigest, SortedInt, Stream, Full-Text Search, and HNSW Vector Retrieval).
Multi-Tenant and Multi-DB Isolation: Supports up to $2^{64}$ isolated tenants and databases. Tenant renaming updates only metadata mappings in $O(1)$ time without data rewrites.
Crash Consistency: Relies on Write-Ahead Logging (WAL) and cross-keyspace atomic write batches to maintain data integrity across crashes.

Quickstart

Installation

cargo add wedb_embed

Basic Usage

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    // 1. Open database
    let db = WeDb::open("./data/quickstart_db", [])?;

    // 2. String operations
    db.set(b"site", b"webc.site", &[])?;
    let val = db.get(b"site")?;
    assert_eq!(val.as_deref(), Some(&b"webc.site"[..]));

    // 3. Hash operations
    db.hset(b"user:1", &[(b"name", b"Alice"), (b"age", b"20")])?;
    let age = db.hget(b"user:1", b"age")?;
    assert_eq!(age.as_deref(), Some(&b"20"[..]));

    // 4. Sorted set operations
    db.zadd(b"rank", &[(100.0, b"player1"), (200.0, b"player2")], &[])?;
    let top = db.zrange(b"rank", 0, 10)?;
    assert_eq!(top.len(), 2);

    Ok(())
}

Click here for more examples

Why an Embedded Redis Engine
Quickstart
- Installation
- Basic Usage
Performance & Resource Comparison
Storage Architecture & Encoding Design
Runtime Architecture & Threading Model
- Thread-per-Core Architecture Design
- Pitfalls of Multi-Threaded Work-Stealing Runtimes
Multi-Tenant & Multi-DB Isolation
Data Structures & Operations
Tech Stack

Performance & Resource Comparison

macOS (Apple M2 Max)

Hardware & Test Environment

CPU: Apple M2 Max (12 cores) Memory: 64.0 GB OS: macOS 26.5.1 (Darwin 25.5.0) Rust: 1.98.0 (88d9e12ae 2026-08-18) Redis: v8.10.1

Physical Footprint & Memory Benchmark (5GB Dataset Scale)

Resource Metric	wedb_embed (Embedded LSM+LZ4)	Redis (v8.10.1 AOF Mode)	Resource Savings
Dataset Scale	5,000,000 Structured Items	5,000,000 Structured Items	All 14 Data Formats
Raw Uncompressed Payload	4377 MB	4377 MB	Structured Payload
Physical Disk Footprint	4791 MB	7791 MB	Saves 39%
Resident Memory (RSS)	508 MB	4918 MB	Saves 90%

wedb_embed vs Redis Core Command Benchmark

Command	wedb_embed P95 Latency	Redis P95 Latency	Speedup
`SET`	9.1 us	47.3 us	5.2x
`GET`	0.83 us	41.8 us	50.2x
`MSET`	52.2 us	54.6 us	1.0x
`MGET`	2.4 us	43.3 us	17.9x
`INCRBY`	0.58 us	48.1 us	82.8x
`DECRBY`	0.58 us	45.9 us	79.8x
`APPEND`	0.78 us	49.4 us	63.4x
`STRLEN`	0.24 us	40.9 us	168.1x
`GETDEL`	10.1 us	94.5 us	9.4x
`GETRANGE`	0.25 us	42.8 us	171.3x
`SETRANGE`	0.59 us	45.0 us	75.9x
`HSET`	3.0 us	48.4 us	16.0x
`HGET`	0.70 us	47.0 us	67.2x
`HMGET`	2.7 us	45.3 us	17.0x
`HEXISTS`	0.64 us	45.7 us	71.7x
`HLEN`	0.47 us	43.0 us	90.7x
`HDEL`	4.9 us	43.0 us	8.7x
`HGETALL`	3.2 us	44.1 us	13.9x
`HKEYS`	3.1 us	45.8 us	14.7x
`HVALS`	3.1 us	42.8 us	13.7x
`HINCRBY`	3.0 us	45.6 us	15.3x
`LPUSH`	2.9 us	44.1 us	15.0x
`RPUSH`	3.4 us	43.8 us	13.0x
`LPOP`	3.0 us	44.7 us	14.7x
`RPOP`	3.2 us	43.5 us	13.8x
`LLEN`	0.46 us	40.8 us	89.1x
`LRANGE`	2.6 us	42.4 us	16.2x
`LINDEX`	0.67 us	41.0 us	61.3x
`LSET`	2.2 us	44.0 us	19.9x
`LREM`	14.0 us	93.9 us	6.7x
`LTRIM`	2.8 us	43.1 us	15.4x
`SADD`	2.3 us	44.9 us	19.2x
`SREM`	4.8 us	46.5 us	9.7x
`SISMEMBER`	0.65 us	41.3 us	63.9x
`SCARD`	0.48 us	41.1 us	86.3x
`SMEMBERS`	3.2 us	41.8 us	13.0x
`SPOP`	9.2 us	88.4 us	9.6x
`SRANDMEMBER`	3.3 us	40.9 us	12.4x
`ZADD`	3.7 us	43.9 us	11.8x
`ZSCORE`	0.72 us	44.3 us	61.3x
`ZRANGE`	3.7 us	45.8 us	12.2x
`ZCARD`	0.49 us	40.8 us	82.5x
`ZCOUNT`	3.2 us	41.6 us	13.1x
`ZINCRBY`	3.9 us	44.6 us	11.4x
`ZRANK`	3.6 us	41.7 us	11.7x
`ZREVRANGE`	5.3 us	46.0 us	8.7x
`ZPOPMIN`	9.0 us	97.0 us	10.8x
`ZREM`	5.1 us	43.0 us	8.4x
`SETBIT`	9.5 us	55.4 us	5.8x
`GETBIT`	0.42 us	47.5 us	112.7x
`BITCOUNT`	0.44 us	46.0 us	104.3x
`BITPOS`	0.45 us	50.0 us	111.0x
`PFADD`	4.5 us	44.9 us	10.0x
`PFCOUNT`	32.3 us	41.5 us	1.3x
`GEOADD`	4.0 us	46.3 us	11.6x
`GEODIST`	0.86 us	41.7 us	48.6x
`GEOPOS`	0.65 us	41.3 us	63.2x
`GEOHASH`	0.70 us	41.1 us	58.9x
`XADD`	2.7 us	44.9 us	16.8x
`XLEN`	0.45 us	40.7 us	90.3x
`XRANGE`	3.8 us	47.8 us	12.5x
`XREAD`	3.7 us	49.1 us	13.1x
`XDEL`	5.3 us	88.3 us	16.6x
`DEL`	4.1 us	35.6 us	8.6x
`EXISTS`	0.20 us	35.1 us	172.9x
`EXPIRE`	2.0 us	42.0 us	21.0x
`TTL`	0.25 us	42.0 us	168.1x
`JSON.SET`	4.1 us	45.8 us	11.1x
`JSON.GET`	1.3 us	46.4 us	36.6x
`JSON.DEL`	11.3 us	101.5 us	9.0x
`JSON.NUMINCRBY`	4.2 us	48.2 us	11.4x
`JSON.ARRLEN`	1.1 us	46.4 us	42.1x
`JSON.TYPE`	1.1 us	42.5 us	37.8x
`BF.ADD`	20.6 us	48.2 us	2.3x
`BF.EXISTS`	0.63 us	37.4 us	59.7x
`BF.INFO`	0.44 us	48.6 us	110.8x
`CF.ADD`	4.1 us	39.7 us	9.7x
`CF.EXISTS`	0.66 us	35.5 us	53.6x
`CF.DEL`	7.6 us	84.7 us	11.1x
`TDIGEST.ADD`	3.3 us	41.2 us	12.5x
`TDIGEST.QUANTILE`	0.83 us	41.4 us	49.7x
`TDIGEST.BYRANK`	0.83 us	43.4 us	52.1x
`TDIGEST.CDF`	1.0 us	45.4 us	43.6x
`TS.ADD`	6.5 us	44.8 us	6.9x
`TS.GET`	2.2 us	43.4 us	19.6x
`TS.RANGE`	10.6 us	70.6 us	6.7x
`TS.INCRBY`	9.8 us	45.2 us	4.6x
`FT.SEARCH`	19.8 us	77.2 us	3.9x
`FT.TAG`	19.6 us	66.4 us	3.4x
`VECTOR.KNN`	2.2 us	64.8 us	28.8x

wedb_embed Performance Regression

Command	P95 Latency	Est. Throughput	vs Baseline (%)
`BITCOUNT`	437.2 ns	237 万/s	0%
`BITPOS`	1.135 µs	96 万/s	0%
`GETBIT`	468.4 ns	218 万/s	0%
`SETBIT`	117.7 µs	9 万/s	0%
`BF.ADD`	50.79 µs	3 万/s	0%
`BF.EXISTS`	713.2 ns	150 万/s	0%
`BF.INFO`	432 ns	233 万/s	0%
`CF.ADD`	55.83 µs	16 万/s	0%
`CF.DEL`	34.16 µs	9 万/s	0%
`CF.EXISTS`	812.2 ns	138 万/s	0%
`BATCH_COMMIT`	34.66 µs	16 万/s	0%
`DEL`	157.8 µs	5 万/s	0%
`EXISTS`	170.2 ns	601 万/s	0%
`EXPIRE`	6.041 µs	29 万/s	0%
`NAMESPACE`	11.45 µs	19 万/s	0%
`TTL`	227.5 ns	450 万/s	0%
`GEOADD`	227.4 µs	16 万/s	0%
`GEODIST`	1.244 µs	84 万/s	0%
`GEOHASH`	728.9 ns	139 万/s	0%
`GEOPOS`	984 ns	123 万/s	0%
`HDEL`	5.208 µs	26 万/s	0%
`HEXISTS`	640.3 ns	159 万/s	0%
`HGET`	728.8 ns	142 万/s	0%
`HGETALL`	30.7 µs	27 万/s	0%
`HINCRBY`	172.7 µs	19 万/s	0%
`HKEYS`	33.04 µs	28 万/s	0%
`HLEN`	471 ns	216 万/s	0%
`HMGET`	2.728 µs	40 万/s	0%
`HSET`	176.5 µs	19 万/s	0%
`HVALS`	25.29 µs	30 万/s	0%
`PFADD`	191.3 µs	6 万/s	0%
`PFCOUNT`	41.04 µs	3 万/s	0%
`PFMERGE`	40.95 µs	7 万/s	0%
`JSON.ARRLEN`	43.45 µs	71 万/s	0%
`JSON.DEL`	12.83 µs	11 万/s	0%
`JSON.GET`	26.49 µs	63 万/s	0%
`JSON.NUMINCRBY`	48.29 µs	18 万/s	0%
`JSON.SET`	64.58 µs	13 万/s	0%
`JSON.TYPE`	1.676 µs	87 万/s	0%
`LINDEX`	833 ns	135 万/s	0%
`LLEN`	484 ns	211 万/s	0%
`LPOP`	21.41 µs	6 万/s	0%
`LPUSH`	191.9 µs	20 万/s	0%
`LRANGE`	4.582 µs	32 万/s	0%
`LREM`	224 µs	5 万/s	0%
`LSET`	4.083 µs	32 万/s	0%
`LTRIM`	7.29 µs	33 万/s	0%
`RPOP`	8.416 µs	13 万/s	0%
`RPUSH`	132.6 µs	19 万/s	0%
`FT.SEARCH`	42.2 µs	4 万/s	0%
`FT.TAG`	27.66 µs	4 万/s	0%
`SADD`	150.6 µs	16 万/s	0%
`SCARD`	520.4 ns	202 万/s	0%
`SISMEMBER`	765.3 ns	138 万/s	0%
`SMEMBERS`	6.291 µs	30 万/s	0%
`SPOP`	17.95 µs	10 万/s	0%
`SRANDMEMBER`	6.249 µs	22 万/s	0%
`SREM`	4.916 µs	25 万/s	0%
`SI.ADD`	187 µs	19 万/s	0%
`SI.CARD`	585.6 ns	210 万/s	0%
`SI.EXISTS`	822.5 ns	123 万/s	0%
`SI.RANGE`	17.49 µs	15 万/s	0%
`SI.REM`	158.2 µs	6 万/s	0%
`APPEND`	609 ns	175 万/s	0%
`DECRBY`	666.4 ns	166 万/s	0%
`GET`	260.1 ns	418 万/s	0%
`GETDEL`	180.7 µs	7 万/s	0%
`GETRANGE`	372.1 ns	500 万/s	0%
`INCRBY`	775.7 ns	147 万/s	0%
`MGET`	2.666 µs	42 万/s	0%
`MSET`	172.2 µs	2 万/s	0%
`SET`	121 µs	8 万/s	0%
`SETRANGE`	872.1 ns	195 万/s	0%
`STRLEN`	197.6 ns	516 万/s	0%
`XADD`	176.1 µs	21 万/s	0%
`XDEL`	169.2 µs	10 万/s	0%
`XLEN`	486.7 ns	207 万/s	0%
`XRANGE`	12.33 µs	11 万/s	0%
`XREAD`	17.29 µs	11 万/s	0%
`TDIGEST.ADD`	9.791 µs	19 万/s	0%
`TDIGEST.BYRANK`	13.83 µs	89 万/s	0%
`TDIGEST.CDF`	10.91 µs	80 万/s	0%
`TDIGEST.QUANTILE`	12.49 µs	83 万/s	0%
`TS.ADD`	66.62 µs	12 万/s	0%
`TS.GET`	5.291 µs	43 万/s	0%
`TS.INCRBY`	53.62 µs	8 万/s	0%
`TS.RANGE`	53.87 µs	3 万/s	0%
`VECTOR.KNN`	8.165 µs	34 万/s	0%
`ZADD`	169.9 µs	18 万/s	0%
`ZCARD`	538.7 ns	226 万/s	0%
`ZCOUNT`	5.874 µs	31 万/s	0%
`ZINCRBY`	146.8 µs	19 万/s	0%
`ZPOPMIN`	14.79 µs	13 万/s	0%
`ZRANGE`	6.165 µs	26 万/s	0%
`ZRANK`	5.499 µs	29 万/s	0%
`ZREM`	5.499 µs	24 万/s	0%
`ZREVRANGE`	9.457 µs	20 万/s	0%
`ZSCORE`	723.7 ns	141 万/s	0%

Storage Architecture & Encoding Design

graph TD
  Client["Application Code"] --> WeDb["WeDb Instance"]
  WeDb --> NS["Namespace Handle<br/>(Zero-Heap Struct)"]

  subgraph KeyComposer["Key Composer & Fast Encoding"]
    Tag["1-Byte Fast KeyTag (#[repr(u8)])"]
    OPPV["OPPV Order-Preserving Varint (1~9B)"]
    DefaultBypass["Default NS 0-Prefix Bypass"]
    Slot["CRC16 Slot & {hashtag}"]
  end

  subgraph Engine["Storage Engine Layer"]
    Batch["Atomic WriteBatch (Cross-Keyspace WAL)"]
    Catalog["Catalog Compact Index"]
  end

  subgraph Storage["Fjall LSM Dual-Track 4-Partition"]
    subgraph DefaultTrack["Default NS Track (db0 / 0-Byte Prefix)"]
      DataKS["Data Keyspace<br/>(Default String & Subkeys / 64KB Blocks)"]
      MetaKS["Meta Keyspace<br/>(Default Composite Metadata / 4KB Blocks)"]
    end
    subgraph TenantTrack["Tenant Track (Multi-Tenant & Multi-DB)"]
      DataNsKS["Data NS Keyspace<br/>(Tenant String & Subkeys / 64KB Blocks)"]
      MetaNsKS["Meta NS Keyspace<br/>(Tenant Metadata & Catalog / 4KB Blocks)"]
    end
  end

  WeDb --> KeyComposer
  NS --> KeyComposer
  KeyComposer --> Engine
  Engine --> DataKS
  Engine --> MetaKS
  Engine --> DataNsKS
  Engine --> MetaNsKS

Dual-Track Storage Partitioning & Zero-Byte Prefix

Dual-Track 4-Partition Storage:
- Default Track (data / meta): Tailored for single-DB scenarios (default / db 0), storing keys directly as raw bytes with 0-byte prefix overhead. Composite metadata and subkeys are segregated using 1-Byte KeyTag.
- Tenant Track (data_ns / meta_ns): Dedicated to multi-tenant and multi-DB isolation. Tenant data resides in data_ns with encoded tenant prefixes, while tenant metadata and catalog directories reside in meta_ns, physically eliminating key collisions.
1-Byte Fast Tag (KeyTag): Composite metadata and subkey prefixes use #[repr(u8)] KeyTag encoding (\x01[key]), minimizing string prefix overhead.

Order-Preserving Prefix Varint (OPPV)

Database indices and tenant numerical IDs use OPPV (Order-Preserving Prefix Varint) encoding:
- Values $0 \sim 127$ occupy only 1 byte (significantly smaller than fixed 8-byte big-endian integers).
- Preserves lexicographical byte order equal to numeric value order: $\forall a < b \implies \text{encode}(a) < \text{encode}(b)$, allowing native range scans.

Numerical Tenant Mapping & Atomic Renaming

Tenant names map bi-directionally to numerical IDs (ns_id: u64) via global metadata.
Tenant Renaming (rename_namespace): Metadata mappings are atomically updated without rewriting underlying data keys.

Memory-Efficient Streaming Iterators

WeDb::iter(&self) -> Namespaces: Iterates distinct tenants via metadata prefixes with $O(1)$ auxiliary memory.
Namespace::iter(&self) -> Dbs: Decodes active database indices directly from Catalog directory entries.

Runtime Architecture & Threading Model

The underlying architecture of wedb_embed is co-designed specifically for Thread-per-Core (one thread per CPU core) asynchronous runtimes based on Linux io_uring (such as compio), maximizing hardware data locality, lock-free execution, and single-core CPU cache residency.

graph LR
  subgraph CompioModel["compio Thread Model (Thread-per-Core)"]
    direction TB
    C1["CPU Core 0 (Worker 0)<br/>Pinned Core"] --> S1["Local SmallKey Stack Buffer (128B)<br/>L1/L2 Cache Hit (Zero Invalidation)"]
    C2["CPU Core 1 (Worker 1)<br/>Pinned Core"] --> S2["Local SmallKey Stack Buffer (128B)<br/>L1/L2 Cache Hit (Zero Invalidation)"]
    S1 --> IO1["Direct Synchronous / io_uring<br/>No Work-Stealing ｜ Zero Syscall Overhead"]
    S2 --> IO2["Direct Synchronous / io_uring<br/>No Work-Stealing ｜ Zero Syscall Overhead"]
  end

  subgraph TokioModel["Tokio Work-Stealing Model"]
    direction TB
    T1["Worker Thread A"] <-->|"Cross-Core Work Stealing<br/>L1/L2 Cache Bouncing ｜ NUMA Migration"| T2["Worker Thread B"]
    T1 --> ST["Heap-Allocated State Machine (Send + 'static)<br/>Arc/Mutex Contention ｜ Stack Lifetime Lost"]
    T2 --> SB["spawn_blocking Pool Switch<br/>Thread Context Switches ｜ 5~10x Latency Amplification"]
  end

Thread-per-Core Architecture Design

Stack-Allocated Lifetimes & Zero-Heap Allocation: Physical key synthesis utilizes SmallKey 128-byte stack buffers and SubkeyComposer in-place prefix reuse. In a Thread-per-Core model, execution stays strictly within the current CPU core's stack frame without requesting heap allocations from global memory allocators (such as jemalloc or glibc malloc), eliminating allocator-level lock contention.
CPU Cache Line Locality & Zero Bouncing: Worker threads are statically pinned to physical CPU cores without cross-core task migration. Hot data structures (LSM-Tree memtable indices, bloom filter bitsets, and Catalog metadata caches) stay resident in L1/L2 CPU caches, preventing MESI cache coherency protocols from broadcasting invalidation traffic across cores (eliminating Cache Line Bouncing).
Lock-Free & Lightweight Metadata Access: Namespace and tenant directories are managed via papaya::HashMap lock-free concurrent hash maps, ensuring wait-free and lock-free read operations across all threads. Infrequent metadata updates use parking_lot adaptive spinlocks that resolve immediately within single-core execution without triggering OS-level futex sleeps or cross-core wakeup overhead.
Direct Synchronous Calls Without Cross-Thread Scheduling: All engine APIs are direct, synchronous in-memory/disk function calls. In a compio single-threaded event loop, microsecond and nanosecond lookups execute directly on the local thread and integrate seamlessly with completion-based io_uring I/O, avoiding the runtime overhead of wrapping futures into cross-thread state machines.

Pitfalls of Multi-Threaded Work-Stealing Runtimes

Employing wedb_embed inside general-purpose multi-threaded work-stealing runtimes based on epoll (such as tokio) introduces distinct physical bottlenecks and resource overhead:

Cross-Core Task Stealing & Cache Invalidation: Tokio's work-stealing scheduler migrates tasks across worker threads when idle. A single future resumed after an await point may run on a different CPU core or across NUMA nodes, causing L1/L2 data and instruction caches to invalidate completely and inducing tail-latency (P99/P999) jitter.
Broken Stack Lifetimes & Mandatory Heap Allocations: Tokio asynchronous tasks require Send + 'static bounds. Stack-allocated borrowed references (such as &[u8] slices and stack SmallKey buffers) cannot cross await boundaries without being promoted to heap memory (Box, Arc, or Vec<u8>), negating the zero-heap allocation advantages of embedded execution.
Event Loop Starvation & Blocking Thread Pool Overhead: In an epoll-based runtime, executing synchronous disk I/O or CPU-bound index lookups directly inside worker threads blocks the event loop, starving concurrent network connections. Offloading operations via tokio::task::spawn_blocking introduces thread context switches, cross-thread channel (IPC) transfers, and task reschedulings, amplifying microsecond operations by 5~10x.
Multi-Core Bus Locking & Mutex Contention: When multiple threads concurrently contend for shared instances, frequent atomic operations (CAS) and mutex locking across cores cause memory bus contention and lock storms, reducing peak concurrent throughput.

Multi-Tenant & Multi-DB Isolation

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/multitenant_demo", [])?;

    // 1. Tenant-isolated handles (zero-heap structs)
    let ns_apple = db.namespace("tenant_apple");
    let ns_google = db.namespace("tenant_google");

    ns_apple.set(b"config:theme", b"dark", &[])?;
    ns_google.set(b"config:theme", b"light", &[])?;

    // 2. Physically isolated lookups
    assert_eq!(ns_apple.get(b"config:theme")?.unwrap(), b"dark");
    assert_eq!(ns_google.get(b"config:theme")?.unwrap(), b"light");

    // 3. Multi-DB selection (e.g. SELECT 1)
    let db1 = db.select_db(1)?;
    db1.set(b"db1_key", b"value1", &[])?;

    // 4. Iterate active databases under tenant (streaming iterator)
    for db_idx in &ns_apple {
        println!("Active DB: {}", db_idx);
    }

    // 5. Atomic tenant renaming (zero data rewrite)
    db.rename_namespace("tenant_apple", "tenant_apple_v2")?;

    // 6. Keyspace management
    let count = ns_apple.key_count()?;
    let keys = ns_apple.keys("config:*")?;
    let exists = ns_apple.exists(&[b"config:theme"])?;
    ns_apple.del(&[b"config:theme"])?;
    ns_apple.clear()?; // Cascade clear all tenant data

    Ok(())
}

Data Structures & Operations

Initialization & Configuration

use wedb_embed::{Conf, PersistMode, Result, WeDb};

fn main() -> Result<()> {
    // Open with custom configuration enum list (LZ4 compression enabled by default)
    let db = WeDb::open(
        "./data/db_demo",
        [
            Conf::CacheSize(128 * 1024 * 1024), // 128MB LSM block cache
            Conf::ManualJournalPersist(false),  // Auto-flush WAL in background
            Conf::WorkerThreads(2),             // Background worker threads
        ],
    )?;

    // Persistence and active expiration tasks
    db.persist(PersistMode::SyncAll)?; // Synchronously persist WAL and dirty pages
    db.active_expire_cycle(100)?;       // Trigger active key expiration sampling

    Ok(())
}

Key-Value & Strings

use wedb_embed::{prelude::*, string::Set, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/kv_demo", [])?;

    // Basic read, write, and TTL expiration
    db.set(b"site", b"webc.site", &[])?;
    let val = db.get(b"site")?;
    db.setex(b"temp_token", b"xyz", 3600_000)?; // Millisecond TTL

    // Conditional write (NX / XX)
    db.set(b"lock", b"1", &[Set::Nx])?;

    // Numerical increment & decrement
    db.incr(b"counter")?;
    db.incrby(b"counter", 10)?;
    db.decrby(b"counter", 5)?;

    // Batch & atomic operations
    db.mset(&[(b"k1", b"v1"), (b"k2", b"v2")])?;
    let values = db.mget(&[b"k1", b"k2"])?;
    let len = db.strlen(b"site")?;
    let old_val = db.getset(b"site", b"new_site")?;
    let del_val = db.getdel(b"site")?;
    db.cas(b"site", b"new_site", b"final_site", 0)?; // Compare and Swap

    Ok(())
}

Hash Map & Field-Level TTL

use wedb_embed::{hash::HExpire, prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/hash_demo", [])?;

    // Field write & read
    db.hset(b"user:100", &[(b"name", b"Alice"), (b"age", b"20")])?;
    let name = db.hget(b"user:100", b"name")?;

    // Batch field operations
    db.hmset(b"user:100", &[(b"city", b"Beijing"), (b"role", b"Admin")])?;
    let fields = db.hmget(b"user:100", &[b"name", b"city"])?;

    // Counter increment and introspection
    db.hincrby(b"user:100", b"age", 1)?;
    let exists = db.hexists(b"user:100", b"name")?;
    let len = db.hlen(b"user:100")?;
    let all = db.hgetall(b"user:100")?;
    let keys = db.hkeys(b"user:100")?;
    let vals = db.hvals(b"user:100")?;
    db.hdel(b"user:100", &[b"role"])?;

    // Field-level independent TTL
    db.hexpire(b"user:100", &[b"name"], 3600, HExpire::None)?;
    let ttls = db.httl(b"user:100", &[b"name"])?;

    Ok(())
}

List

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/list_demo", [])?;

    // Push and pop operations
    db.lpush(b"tasks", &[b"task1", b"task2"])?;
    db.rpush(b"tasks", &[b"task3"])?;
    let first = db.lpop_one(b"tasks")?;
    let last = db.rpop_one(b"tasks")?;

    // Range, trim, and set
    let len = db.llen(b"tasks")?;
    let items = db.lrange(b"tasks", 0, -1)?;
    db.ltrim(b"tasks", 0, 10)?;
    db.lset(b"tasks", 0, b"updated_task")?;

    Ok(())
}

Set

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/set_demo", [])?;

    db.sadd(b"tags", &[b"rust", b"database", b"lsm"])?;
    let is_member = db.sismember(b"tags", b"rust")?;
    let count = db.scard(b"tags")?;
    let members = db.smembers(b"tags")?;
    db.srem(b"tags", &[b"lsm"])?;
    let popped = db.spop(b"tags", 1)?;

    // Set algebra operations
    db.sadd(b"tags_other", &[b"rust", b"storage"])?;
    let inter = db.sinter(&[b"tags", b"tags_other"])?;
    let union = db.sunion(&[b"tags", b"tags_other"])?;
    let diff = db.sdiff(&[b"tags", b"tags_other"])?;

    Ok(())
}

Sorted Set

use wedb_embed::{prelude::*, zset::RangeScoreSpec, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/zset_demo", [])?;

    // Add members and scores
    db.zadd(b"leaderboard", &[(100.0, b"player1"), (200.0, b"player2")], &[])?;
    db.zincrby(b"leaderboard", 50.0, b"player1")?;

    // Score and rank queries
    let score = db.zscore(b"leaderboard", b"player1")?;
    let rank = db.zrank(b"leaderboard", b"player1")?;
    let rev_rank = db.zrevrank(b"leaderboard", b"player1")?;

    // Range queries and count
    let top = db.zrange(b"leaderboard", 0, 10)?;
    let count = db.zcount(b"leaderboard", &RangeScoreSpec::new(100.0, 300.0))?;
    let card = db.zcard(b"leaderboard")?;
    db.zrem(b"leaderboard", &[b"player1"])?;

    Ok(())
}

Bitmap & Bitfield

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/bitmap_demo", [])?;

    // Bit manipulation
    db.setbit(b"online_users", 1001, 1)?;
    let is_online = db.getbit(b"online_users", 1001)?;
    let online_count = db.bitcount(b"online_users", None, None)?;
    let first_online = db.bitpos(b"online_users", 1, None, None)?;

    Ok(())
}

JSON & JSONPath

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/json_demo", [])?;

    // JSON write and JSONPath queries
    db.json_set(b"doc:1", "$", r#"{"user":{"name":"Alice","age":25,"roles":["admin"]}}"#)?;
    let name = db.json_get(b"doc:1", Some("$.user.name"))?;
    db.json_numincrby(b"doc:1", "$.user.age", "1.0")?;
    db.json_arrappend(b"doc:1", "$.user.roles", &[r#""editor""#])?;

    Ok(())
}

Bloom & Cuckoo Filters

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/filter_demo", [])?;

    // Bloom Filter
    db.bf_reserve(b"bf_filter", 0.01, 10000, 2)?;
    db.bf_add(b"bf_filter", b"item_1")?;
    let exists = db.bf_exists(b"bf_filter", b"item_1")?;

    // Cuckoo Filter (supports dynamic deletion)
    db.cf_reserve(b"cf_filter", 10000, 2, 500, 1)?;
    db.cf_add(b"cf_filter", b"item_2")?;
    let cf_exists = db.cf_exists(b"cf_filter", b"item_2")?;
    db.cf_del(b"cf_filter", b"item_2")?;

    Ok(())
}

TimeSeries & Aggregations

use wedb_embed::{
    prelude::*,
    timeseries::{AggregationType, DuplicatePolicy, TSRangeOption},
    Result, WeDb,
};

fn main() -> Result<()> {
    let db = WeDb::open("./data/timeseries_demo", [])?;

    // Create series
    db.ts_create(b"cpu:usage", 0, DuplicatePolicy::Last, &[("host", "server-1")])?;
    db.ts_add(b"cpu:usage", 1700000000000, 42.5)?;
    let latest = db.ts_get(b"cpu:usage")?;

    // Window downsampling query (Gorilla delta-of-delta compression)
    let samples = db.ts_range_opt(
        b"cpu:usage",
        &TSRangeOption {
            from_timestamp: 0,
            to_timestamp: 1800000000000,
            aggregation: Some((AggregationType::Avg, 60000)),
            ..Default::default()
        },
    )?;

    Ok(())
}

Geospatial

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/geo_demo", [])?;

    // Coordinate storage and distance calculation
    db.geoadd(
        b"cities",
        &[
            (116.4074, 39.9042, b"Beijing"),
            (121.4737, 31.2304, b"Shanghai"),
        ],
    )?;
    let dist_km = db.geodist(b"cities", b"Beijing", b"Shanghai", Some("km"))?;
    let hash = db.geohash(b"cities", &[b"Beijing"])?;

    Ok(())
}

HyperLogLog

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/hll_demo", [])?;

    db.pfadd(b"hll_uv", &[b"user_1", b"user_2", b"user_3"])?;
    let uv_count = db.pfcount(&[b"hll_uv"])?;
    db.pfmerge(b"hll_merged", &[b"hll_uv"])?;

    Ok(())
}

T-Digest

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/tdigest_demo", [])?;

    db.tdigest_create(b"latencies", 100.0)?;
    db.tdigest_add(b"latencies", &[12.5, 15.0, 18.2, 45.0, 99.9])?;
    let p95 = db.tdigest_quantile(b"latencies", &[0.95])?;
    let cdf = db.tdigest_cdf(b"latencies", &[20.0])?;

    Ok(())
}

SortedInt

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/sortedint_demo", [])?;

    // 64-bit compact integer set
    db.si_add(b"post:100:likes", &[1001, 1002, 1003, 1004])?;
    let has_liked = db.si_exists(b"post:100:likes", 1001)?;
    let count = db.si_card(b"post:100:likes")?;
    let top = db.si_range(b"post:100:likes", 0, 0, 10, false)?;

    Ok(())
}

Streams & Consumer Groups

use wedb_embed::{prelude::*, stream::StreamId, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/stream_demo", [])?;

    // Append entry
    let entry_id = db.xadd_simple(
        b"event_stream",
        None,
        &[(b"action", b"login"), (b"uid", b"1001")],
    )?;
    let len = db.xlen(b"event_stream")?;
    let entries = db.xrange(b"event_stream", StreamId::MIN, StreamId::MAX, Some(10))?;

    // Consumer groups & pending entries
    db.xgroup_create(b"event_stream", "group_workers", "0-0", false, None)?;
    let pending = db.xpending_summary(b"event_stream", "group_workers")?;

    Ok(())
}

Full-Text Search & Vector Retrieval

use wedb_embed::{
    prelude::*,
    search::{
        DistanceMetric, FtCreate, FtSearch, IndexField, IndexFieldType, IndexOnDataType,
        SearchIndexManager, VectorAlgorithm, VectorFieldMetadata, VectorType,
    },
    Result, WeDb,
};

fn main() -> Result<()> {
    let db = WeDb::open("./data/search_demo", [])?;
    let mut search_mgr = SearchIndexManager::new();

    // 1. Create multi-field schema (Inverted Text + HNSW Vector)
    let mut schema = FtCreate::new("idx_articles", IndexOnDataType::Json);
    schema.prefixes.push("article:".to_string());
    schema.fields.push(IndexField {
        name: "title".to_string(),
        alias: None,
        field_type: IndexFieldType::Text,
        weight: 1.0,
        sortable: false,
        noindex: false,
        separator: None,
        casesensitive: false,
        withsuffixtrie: false,
        unf: false,
        vector_meta: None,
    });
    schema.fields.push(IndexField {
        name: "vec".to_string(),
        alias: None,
        field_type: IndexFieldType::Vector,
        weight: 1.0,
        sortable: false,
        noindex: false,
        separator: None,
        casesensitive: false,
        withsuffixtrie: false,
        unf: false,
        vector_meta: Some(VectorFieldMetadata {
            algorithm: VectorAlgorithm::Hnsw,
            vector_type: VectorType::Float32,
            dim: 4,
            distance_metric: DistanceMetric::Cosine,
            m: 16,
            ef_construction: 200,
            ef_runtime: 10,
            block_size: 1024,
            initial_cap: 1000,
        }),
    });
    search_mgr.create_index_from_opts(schema)?;

    // 2. Execute text and vector queries
    let res = search_mgr.search("idx_articles", "@title:database", &FtSearch::default())?;
    println!("Found matching docs: {}", res.total);

    Ok(())
}

Tech Stack

Language: Rust Edition 2024
Storage Engine: fjall (LSM-Tree persistence engine)
JSON Engine: sonic-rs (SIMD-accelerated parser)
Non-Cryptographic Hash: rapidhash
Serialization: bitcode (compact binary encoding)
Strings & Memory: hipstr (compact string and zero-copy borrowing)
Concurrent Hash Map: papaya (lock-free concurrent dictionary)
Bitwise & Collections: roaring, memchr, crc32fast, fastrand
Timestamps: coarsetime, ts_
Number Formatter: zmij, itoa
Enum Derives: strum
Error Management: thiserror

嵌入式数据库引擎，提供 Redis 兼容数据结构与接口，基于 fjall LSM-Tree 存储引擎构建。

为什么需要嵌入式 Redis 引擎

为什么需要嵌入式 Redis 引擎
快速上手
- 添加依赖
- 基础读写示例
性能与资源实测对比
- macOS (Apple M2 Max)
存储架构与编码设计
运行时生态与线程模型设计
- 一线程一核心架构设计
- 传统多线程工作窃取运行时问题分析
多租户与分库隔离
数据结构与接口演示
技术堆栈

在后端服务、CLI 工具、边缘计算与桌面端应用中，开发者经常需要使用丰富的数据结构（例如哈希表、有序集合排行榜、消息队列、位图与时序序列）。传统的方案是部署独立的 Redis 进程，通过网络或本地套接字进行通信。这种架构在单机环境下存在以下物理瓶颈：

进程间通信与协议开销：每次读写都需要经过数据序列化、操作系统套接字缓冲区、进程上下文切换、RESP 协议解析与事件循环处理。即使在本地主机上，套接字往返延迟通常也在 20~50 微秒区间，并消耗额外的 CPU 周期。
物理内存成本与容量约束：Redis 将全量数据与内部指针结构常驻于物理内存中。当数据规模增长到数十 GB 时，内存硬件成本高昂，且受限于单机物理 RAM 容量。开启 AOF 或 RDB 持久化时，后台保存机制还会引发写时复制的额外内存开销。
部署与运维复杂度：独立进程需要额外的进程守护、端口监听、配置分发与健康检查逻辑，增加了软件交付与部署的维护成本。

wedb_embed 将存储引擎直接集成到应用程序的进程空间中，改变了数据存储与访问路径：

进程内直接调用：所有数据操作直接通过 Rust 函数调用完成，消除套接字通信、系统调用与跨进程上下文切换。在同等硬件下，核心指令的 P95 延迟从 Redis 的数十微秒降低至纳秒到微秒级。
LSM-Tree 磁盘持久化与冷热分层：基于 LSM-Tree 存储引擎与 LZ4 分块压缩，活跃数据保留在内存缓存中，冷数据与全量数据经过压缩持久化落盘。在 2GB 真实数据实测中，常驻内存 RSS 占用由 Redis 的 1951 MB 降低至 234 MB（减少 88%），物理落盘体积节省 38%。
全量 Redis 数据结构支持：在底层键值引擎之上，实现了 16 种复合数据模型（包含 String、Hash 字段级 TTL、List、Set、ZSet、Bitmap、JSON、Bloom/Cuckoo 过滤器、TimeSeries、Geo、HyperLogLog、TDigest、SortedInt、Stream、全文检索与 HNSW 向量检索）。
多租户与多库物理隔离：原生支持 $2^{64}$ 个独立租户与分库，租户重命名仅需原子修改元数据映射（$O(1)$ 时间复杂度），无需重写底层数据。
数据一致性保证：基于预写日志 WAL 与跨分区原子批处理 WriteBatch，保障断电与崩溃场景下的数据完整性。

快速上手

添加依赖

cargo add wedb_embed

基础读写示例

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    // 打开数据库
    let db = WeDb::open("./data/quickstart_db", [])?;

    // 字符串读写
    db.set(b"site", b"webc.site", &[])?;
    let val = db.get(b"site")?;
    assert_eq!(val.as_deref(), Some(&b"webc.site"[..]));

    // 哈希结构操作
    db.hset(b"user:1", &[(b"name", b"Alice"), (b"age", b"20")])?;
    let age = db.hget(b"user:1", b"age")?;
    assert_eq!(age.as_deref(), Some(&b"20"[..]));

    // 有序集合与排行榜
    db.zadd(b"rank", &[(100.0, b"player1"), (200.0, b"player2")], &[])?;
    let top = db.zrange(b"rank", 0, 10)?;
    assert_eq!(top.len(), 2);

    Ok(())
}

点此查看更多演示

为什么需要嵌入式 Redis 引擎
快速上手
- 添加依赖
- 基础读写示例
性能与资源实测对比
存储架构与编码设计
运行时生态与线程模型设计
- 一线程一核心架构设计
- 传统多线程工作窃取运行时问题分析
多租户与分库隔离
数据结构与接口演示
技术堆栈

性能与资源实测对比

macOS (Apple M2 Max)

硬件与测试环境

CPU: Apple M2 Max (12核) 内存: 64.0 GB 系统: macOS 26.5.1 (Darwin 25.5.0) Rust: 1.98.0 (88d9e12ae 2026-08-18) Redis: v8.10.1

真实物理落盘与内存占用实测 (5GB 数据规模)

资源维度	wedb_embed (嵌入式 LSM+LZ4)	Redis (v8.10.1 AOF持久化)	资源节省比例
测试数据规模	5,000,000 条全格式结构化数据	5,000,000 条全格式结构化数据	14 种数据格式等比实测
原始数据载荷	4377 MB	4377 MB	真实结构化载荷
实际物理落盘大小	4791 MB	7791 MB	节省 39%
进程常驻内存 (RSS)	508 MB	4918 MB	节省 90%

wedb_embed vs Redis 核心指令性能对比

指令	wedb_embed P95延迟	Redis P95延迟	性能领先
`SET`	9.1 us	47.3 us	5.2x
`GET`	0.83 us	41.8 us	50.2x
`MSET`	52.2 us	54.6 us	1.0x
`MGET`	2.4 us	43.3 us	17.9x
`INCRBY`	0.58 us	48.1 us	82.8x
`DECRBY`	0.58 us	45.9 us	79.8x
`APPEND`	0.78 us	49.4 us	63.4x
`STRLEN`	0.24 us	40.9 us	168.1x
`GETDEL`	10.1 us	94.5 us	9.4x
`GETRANGE`	0.25 us	42.8 us	171.3x
`SETRANGE`	0.59 us	45.0 us	75.9x
`HSET`	3.0 us	48.4 us	16.0x
`HGET`	0.70 us	47.0 us	67.2x
`HMGET`	2.7 us	45.3 us	17.0x
`HEXISTS`	0.64 us	45.7 us	71.7x
`HLEN`	0.47 us	43.0 us	90.7x
`HDEL`	4.9 us	43.0 us	8.7x
`HGETALL`	3.2 us	44.1 us	13.9x
`HKEYS`	3.1 us	45.8 us	14.7x
`HVALS`	3.1 us	42.8 us	13.7x
`HINCRBY`	3.0 us	45.6 us	15.3x
`LPUSH`	2.9 us	44.1 us	15.0x
`RPUSH`	3.4 us	43.8 us	13.0x
`LPOP`	3.0 us	44.7 us	14.7x
`RPOP`	3.2 us	43.5 us	13.8x
`LLEN`	0.46 us	40.8 us	89.1x
`LRANGE`	2.6 us	42.4 us	16.2x
`LINDEX`	0.67 us	41.0 us	61.3x
`LSET`	2.2 us	44.0 us	19.9x
`LREM`	14.0 us	93.9 us	6.7x
`LTRIM`	2.8 us	43.1 us	15.4x
`SADD`	2.3 us	44.9 us	19.2x
`SREM`	4.8 us	46.5 us	9.7x
`SISMEMBER`	0.65 us	41.3 us	63.9x
`SCARD`	0.48 us	41.1 us	86.3x
`SMEMBERS`	3.2 us	41.8 us	13.0x
`SPOP`	9.2 us	88.4 us	9.6x
`SRANDMEMBER`	3.3 us	40.9 us	12.4x
`ZADD`	3.7 us	43.9 us	11.8x
`ZSCORE`	0.72 us	44.3 us	61.3x
`ZRANGE`	3.7 us	45.8 us	12.2x
`ZCARD`	0.49 us	40.8 us	82.5x
`ZCOUNT`	3.2 us	41.6 us	13.1x
`ZINCRBY`	3.9 us	44.6 us	11.4x
`ZRANK`	3.6 us	41.7 us	11.7x
`ZREVRANGE`	5.3 us	46.0 us	8.7x
`ZPOPMIN`	9.0 us	97.0 us	10.8x
`ZREM`	5.1 us	43.0 us	8.4x
`SETBIT`	9.5 us	55.4 us	5.8x
`GETBIT`	0.42 us	47.5 us	112.7x
`BITCOUNT`	0.44 us	46.0 us	104.3x
`BITPOS`	0.45 us	50.0 us	111.0x
`PFADD`	4.5 us	44.9 us	10.0x
`PFCOUNT`	32.3 us	41.5 us	1.3x
`GEOADD`	4.0 us	46.3 us	11.6x
`GEODIST`	0.86 us	41.7 us	48.6x
`GEOPOS`	0.65 us	41.3 us	63.2x
`GEOHASH`	0.70 us	41.1 us	58.9x
`XADD`	2.7 us	44.9 us	16.8x
`XLEN`	0.45 us	40.7 us	90.3x
`XRANGE`	3.8 us	47.8 us	12.5x
`XREAD`	3.7 us	49.1 us	13.1x
`XDEL`	5.3 us	88.3 us	16.6x
`DEL`	4.1 us	35.6 us	8.6x
`EXISTS`	0.20 us	35.1 us	172.9x
`EXPIRE`	2.0 us	42.0 us	21.0x
`TTL`	0.25 us	42.0 us	168.1x
`JSON.SET`	4.1 us	45.8 us	11.1x
`JSON.GET`	1.3 us	46.4 us	36.6x
`JSON.DEL`	11.3 us	101.5 us	9.0x
`JSON.NUMINCRBY`	4.2 us	48.2 us	11.4x
`JSON.ARRLEN`	1.1 us	46.4 us	42.1x
`JSON.TYPE`	1.1 us	42.5 us	37.8x
`BF.ADD`	20.6 us	48.2 us	2.3x
`BF.EXISTS`	0.63 us	37.4 us	59.7x
`BF.INFO`	0.44 us	48.6 us	110.8x
`CF.ADD`	4.1 us	39.7 us	9.7x
`CF.EXISTS`	0.66 us	35.5 us	53.6x
`CF.DEL`	7.6 us	84.7 us	11.1x
`TDIGEST.ADD`	3.3 us	41.2 us	12.5x
`TDIGEST.QUANTILE`	0.83 us	41.4 us	49.7x
`TDIGEST.BYRANK`	0.83 us	43.4 us	52.1x
`TDIGEST.CDF`	1.0 us	45.4 us	43.6x
`TS.ADD`	6.5 us	44.8 us	6.9x
`TS.GET`	2.2 us	43.4 us	19.6x
`TS.RANGE`	10.6 us	70.6 us	6.7x
`TS.INCRBY`	9.8 us	45.2 us	4.6x
`FT.SEARCH`	19.8 us	77.2 us	3.9x
`FT.TAG`	19.6 us	66.4 us	3.4x
`VECTOR.KNN`	2.2 us	64.8 us	28.8x

wedb_embed 性能回归测试

命令	P95延时	估算吞吐量	较基线变化 (%)
`BITCOUNT`	437.2 ns	237 万/s	0%
`BITPOS`	1.135 µs	96 万/s	0%
`GETBIT`	468.4 ns	218 万/s	0%
`SETBIT`	117.7 µs	9 万/s	0%
`BF.ADD`	50.79 µs	3 万/s	0%
`BF.EXISTS`	713.2 ns	150 万/s	0%
`BF.INFO`	432 ns	233 万/s	0%
`CF.ADD`	55.83 µs	16 万/s	0%
`CF.DEL`	34.16 µs	9 万/s	0%
`CF.EXISTS`	812.2 ns	138 万/s	0%
`BATCH_COMMIT`	34.66 µs	16 万/s	0%
`DEL`	157.8 µs	5 万/s	0%
`EXISTS`	170.2 ns	601 万/s	0%
`EXPIRE`	6.041 µs	29 万/s	0%
`NAMESPACE`	11.45 µs	19 万/s	0%
`TTL`	227.5 ns	450 万/s	0%
`GEOADD`	227.4 µs	16 万/s	0%
`GEODIST`	1.244 µs	84 万/s	0%
`GEOHASH`	728.9 ns	139 万/s	0%
`GEOPOS`	984 ns	123 万/s	0%
`HDEL`	5.208 µs	26 万/s	0%
`HEXISTS`	640.3 ns	159 万/s	0%
`HGET`	728.8 ns	142 万/s	0%
`HGETALL`	30.7 µs	27 万/s	0%
`HINCRBY`	172.7 µs	19 万/s	0%
`HKEYS`	33.04 µs	28 万/s	0%
`HLEN`	471 ns	216 万/s	0%
`HMGET`	2.728 µs	40 万/s	0%
`HSET`	176.5 µs	19 万/s	0%
`HVALS`	25.29 µs	30 万/s	0%
`PFADD`	191.3 µs	6 万/s	0%
`PFCOUNT`	41.04 µs	3 万/s	0%
`PFMERGE`	40.95 µs	7 万/s	0%
`JSON.ARRLEN`	43.45 µs	71 万/s	0%
`JSON.DEL`	12.83 µs	11 万/s	0%
`JSON.GET`	26.49 µs	63 万/s	0%
`JSON.NUMINCRBY`	48.29 µs	18 万/s	0%
`JSON.SET`	64.58 µs	13 万/s	0%
`JSON.TYPE`	1.676 µs	87 万/s	0%
`LINDEX`	833 ns	135 万/s	0%
`LLEN`	484 ns	211 万/s	0%
`LPOP`	21.41 µs	6 万/s	0%
`LPUSH`	191.9 µs	20 万/s	0%
`LRANGE`	4.582 µs	32 万/s	0%
`LREM`	224 µs	5 万/s	0%
`LSET`	4.083 µs	32 万/s	0%
`LTRIM`	7.29 µs	33 万/s	0%
`RPOP`	8.416 µs	13 万/s	0%
`RPUSH`	132.6 µs	19 万/s	0%
`FT.SEARCH`	42.2 µs	4 万/s	0%
`FT.TAG`	27.66 µs	4 万/s	0%
`SADD`	150.6 µs	16 万/s	0%
`SCARD`	520.4 ns	202 万/s	0%
`SISMEMBER`	765.3 ns	138 万/s	0%
`SMEMBERS`	6.291 µs	30 万/s	0%
`SPOP`	17.95 µs	10 万/s	0%
`SRANDMEMBER`	6.249 µs	22 万/s	0%
`SREM`	4.916 µs	25 万/s	0%
`SI.ADD`	187 µs	19 万/s	0%
`SI.CARD`	585.6 ns	210 万/s	0%
`SI.EXISTS`	822.5 ns	123 万/s	0%
`SI.RANGE`	17.49 µs	15 万/s	0%
`SI.REM`	158.2 µs	6 万/s	0%
`APPEND`	609 ns	175 万/s	0%
`DECRBY`	666.4 ns	166 万/s	0%
`GET`	260.1 ns	418 万/s	0%
`GETDEL`	180.7 µs	7 万/s	0%
`GETRANGE`	372.1 ns	500 万/s	0%
`INCRBY`	775.7 ns	147 万/s	0%
`MGET`	2.666 µs	42 万/s	0%
`MSET`	172.2 µs	2 万/s	0%
`SET`	121 µs	8 万/s	0%
`SETRANGE`	872.1 ns	195 万/s	0%
`STRLEN`	197.6 ns	516 万/s	0%
`XADD`	176.1 µs	21 万/s	0%
`XDEL`	169.2 µs	10 万/s	0%
`XLEN`	486.7 ns	207 万/s	0%
`XRANGE`	12.33 µs	11 万/s	0%
`XREAD`	17.29 µs	11 万/s	0%
`TDIGEST.ADD`	9.791 µs	19 万/s	0%
`TDIGEST.BYRANK`	13.83 µs	89 万/s	0%
`TDIGEST.CDF`	10.91 µs	80 万/s	0%
`TDIGEST.QUANTILE`	12.49 µs	83 万/s	0%
`TS.ADD`	66.62 µs	12 万/s	0%
`TS.GET`	5.291 µs	43 万/s	0%
`TS.INCRBY`	53.62 µs	8 万/s	0%
`TS.RANGE`	53.87 µs	3 万/s	0%
`VECTOR.KNN`	8.165 µs	34 万/s	0%
`ZADD`	169.9 µs	18 万/s	0%
`ZCARD`	538.7 ns	226 万/s	0%
`ZCOUNT`	5.874 µs	31 万/s	0%
`ZINCRBY`	146.8 µs	19 万/s	0%
`ZPOPMIN`	14.79 µs	13 万/s	0%
`ZRANGE`	6.165 µs	26 万/s	0%
`ZRANK`	5.499 µs	29 万/s	0%
`ZREM`	5.499 µs	24 万/s	0%
`ZREVRANGE`	9.457 µs	20 万/s	0%
`ZSCORE`	723.7 ns	141 万/s	0%

存储架构与编码设计

graph TD
  Client["应用业务代码"] --> WeDb["WeDb 数据库实例"]
  WeDb --> NS["Namespace 句柄<br/>(零堆分配结构体)"]

  subgraph KeyComposer["键编排与紧凑编码"]
    Tag["1 字节键标签 (#[repr(u8)])"]
    OPPV["保序变长整型 (1~9 字节)"]
    DefaultBypass["默认库 0 字节前缀直通"]
    Slot["哈希槽与哈希标签"]
  end

  subgraph Engine["存储引擎与事务层"]
    Batch["原子写批处理 (跨分区 WAL)"]
    Catalog["紧凑元数据目录"]
  end

  subgraph Storage["Fjall LSM 双轨 4 分区存储"]
    subgraph DefaultTrack["默认库轨道 (db 0 / 0 字节物理前缀)"]
      DataKS["数据分区<br/>(字符串数据与复合子键 / 64KB 块)"]
      MetaKS["元数据分区<br/>(复合结构元数据 / 4KB 块)"]
    end
    subgraph TenantTrack["租户轨道 (多租户与多 DB 隔离)"]
      DataNsKS["租户数据分区<br/>(租户字符串与子键 / 64KB 块)"]
      MetaNsKS["租户元数据分区<br/>(租户元数据与目录 / 4KB 块)"]
    end
  end

  WeDb --> KeyComposer
  NS --> KeyComposer
  KeyComposer --> Engine
  Engine --> DataKS
  Engine --> MetaKS
  Engine --> DataNsKS
  Engine --> MetaNsKS

双轨存储分区与零前缀

双轨 4 分区存储结构：
- 默认库轨道 (data / meta)：面向单机单库（default / db 0）场景，数据键直接使用原始字节存储，物理前缀占用 0 字节；复合结构元数据与子键通过 1 字节键标签分离。
- 租户隔离轨道 (data_ns / meta_ns)：面向多租户与多 DB 场景，业务数据落入 data_ns 并携带租户编码前缀，租户元数据与 Catalog 目录落入 meta_ns，在物理存储层面杜绝键冲突。
1 字节紧凑标签 (KeyTag)：复合结构元数据与子键前缀采用 #[repr(u8)] KeyTag 编码（如 \x01[key]），减少字符串标签开销。

保序变长整型编码

数据库编号与租户数字 ID 采用 OPPV 保序变长整型 编码：
- 数值 $0 \sim 127$ 仅占用 1 字节（相比固定 8 字节大端序降低存储占用）。
- 编码后的字节序与原始数值大小顺序严格一致：$\forall a < b \implies \text{encode}(a) < \text{encode}(b)$，支持直接进行底层范围扫描。

租户数字映射与原子重命名

租户名称与数字 ID (ns_id: u64) 通过全局元数据维护双向映射。
租户重命名 (rename_namespace)：仅需原子更新元数据映射关系，无需重写底层业务键。

内存友好流式迭代

WeDb::iter(&self) -> Namespaces：基于元数据前缀流式扫描租户列表，辅助内存占用为 $O(1)$。
Namespace::iter(&self) -> Dbs：直接流式解析 Catalog 目录中编码的激活数据库编号。

运行时生态与线程模型设计

wedb_embed 的底层架构专门针对 compio 等基于 Linux io_uring 的一线程一核心异步运行时进行协同设计，全面发挥单核心独占、无共享状态与物理绑核的硬件局部性优势。

graph LR
  subgraph CompioModel["compio 线程模型（一线程一核心）"]
    direction TB
    C1["CPU 核心 0 (工作线程 0)<br/>绑定物理核心"] --> S1["本地栈缓冲 SmallKey (64B)<br/>L1/L2 缓存热命中 (0 跨核失效)"]
    C2["CPU 核心 1 (工作线程 1)<br/>绑定物理核心"] --> S2["本地栈缓冲 SmallKey (64B)<br/>L1/L2 缓存热命中 (0 跨核失效)"]
    S1 --> IO1["直接同步调用 / io_uring<br/>无工作窃取 ｜ 无上下文切换"]
    S2 --> IO2["直接同步调用 / io_uring<br/>无工作窃取 ｜ 无上下文切换"]
  end

  subgraph TokioModel["Tokio 传统工作窃取模型"]
    direction TB
    T1["工作线程 A"] <-->|"跨核任务窃取<br/>L1/L2 缓存颠簸 ｜ NUMA 节点跳转"| T2["工作线程 B"]
    T1 --> ST["堆分配状态机 (Send + 'static)<br/>互斥锁竞争 ｜ 破坏栈生命周期"]
    T2 --> SB["阻塞线程池切换<br/>线程上下文切换 ｜ 延迟放大 5~10x"]
  end

一线程一核心架构设计

栈上生命周期与零堆分配：物理键构建广泛采用 SmallKey 64 字节栈缓冲与 SubkeyComposer 前缀内存复用。在一线程一核心模型下，执行上下文严格局限在单个 CPU 核心的栈帧内，无需向全局堆内存分配器（如 jemalloc 或 glibc malloc）申请内存，彻底消除了多线程环境下的全局堆分配器互斥锁竞争。
CPU 缓存行局部性与零颠簸：工作线程与物理 CPU 核心静态绑定，任务执行过程中不发生跨核迁移。热点数据结构（LSM-Tree 内存表索引、布隆过滤器位图、Catalog 元数据缓存）常驻于当前 CPU 的 L1/L2 数据缓存中，避免了 MESI 缓存一致性协议在多核心之间广播无效化（Invalidate）消息引发的缓存行反弹。
无锁与轻量并发元数据：命名空间与租户目录采用 papaya::HashMap 无锁并发哈希表管理，读操作全链路无等待；极低频的元数据写操作采用 parking_lot 自适应自旋锁，在单核独占环境下自旋立即完成，不触发内核态 Futex 上下文挂起与跨核线程唤醒。
同步内嵌调用与无跨线程调度：所有存储引擎 API 均为纯同步内存/磁盘直接调用。在 compio 驱动的单线程事件循环中，微秒级与纳秒级的内存查找直接就地完成，与底层 io_uring 完成驱动的异步 I/O 驱动无缝协作，避免了将 Future 包装为跨线程状态机的运行时负担。

传统多线程工作窃取运行时问题分析

若在 tokio 等基于多线程工作窃取与 epoll 反应堆的通用异步运行时中使用，会导致以下性能瓶颈与物理损耗：

跨核心任务窃取导致缓存失效：调度器会在工作线程空闲时跨核窃取任务。同一个请求的 Future 在 await 恢复后可能被调度到不同的 CPU 核心或跨 NUMA 节点执行，导致 L1/L2 数据缓存和指令缓存全量失效，引发 CPU 访存延迟抖动与 P99 尾部延迟劣化。
破坏零堆分配约束与生命周期提升：通用异步任务要求满足 Send + 'static 约束，这意味着栈上分配的短期借用结构（如 &[u8] 切片、栈上 SmallKey）无法跨 await 点存活，被迫将键值与临时状态重新包装分配至堆内存（Box / Arc / Vec<u8>），破坏了进程内嵌入式调用的零堆分配优势。
事件循环阻塞与阻塞线程池切换开销：在基于 epoll 的多线程异步运行时中，直接在工作线程执行同步磁盘读取或 CPU 密集型索引查找会阻塞整个事件循环，导致当前线程承载的其他网络连接出现停顿；若通过任务调度将操作转发至阻塞线程池，则会引入线程上下文切换、跨线程通道传递与二次任务调度，导致微秒级进程内操作延迟放大 5~10 倍。
多核心内存总线争用与锁冲突：在多线程随机争抢共享实例时，跨核心的高频原子操作和互斥锁争用会导致 CPU 内存总线锁定与锁冲突，大幅降低并发吞吐上限。

多租户与分库隔离

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/multitenant_demo", [])?;

    // 获取租户隔离句柄
    let ns_apple = db.namespace("tenant_apple");
    let ns_google = db.namespace("tenant_google");

    ns_apple.set(b"config:theme", b"dark", &[])?;
    ns_google.set(b"config:theme", b"light", &[])?;

    // 物理隔离读取验证
    assert_eq!(ns_apple.get(b"config:theme")?.unwrap(), b"dark");
    assert_eq!(ns_google.get(b"config:theme")?.unwrap(), b"light");

    // 多分库选择
    let db1 = db.select_db(1)?;
    db1.set(b"db1_key", b"value1", &[])?;

    // 遍历租户下已激活的数据库编号
    for db_idx in &ns_apple {
        println!("Active DB: {}", db_idx);
    }

    // 租户重命名
    db.rename_namespace("tenant_apple", "tenant_apple_v2")?;

    // 键空间管理与清理
    let count = ns_apple.key_count()?;
    let keys = ns_apple.keys("config:*")?;
    let exists = ns_apple.exists(&[b"config:theme"])?;
    ns_apple.del(&[b"config:theme"])?;
    ns_apple.clear()?; // 级联清空整个租户数据

    Ok(())
}

数据结构与接口演示

初始化与配置

use wedb_embed::{Conf, PersistMode, Result, WeDb};

fn main() -> Result<()> {
    // 传入配置枚举列表打开数据库（默认开启 LZ4 压缩）
    let db = WeDb::open(
        "./data/db_demo",
        [
            Conf::CacheSize(128 * 1024 * 1024), // 128MB 块缓存
            Conf::ManualJournalPersist(false),  // 启用后台自动刷新预写日志
            Conf::WorkerThreads(2),             // 后台工作线程数
        ],
    )?;

    // 刷盘与周期任务
    db.persist(PersistMode::SyncAll)?; // 同步持久化预写日志与脏页
    db.active_expire_cycle(100)?;       // 触发主动过期采样清理

    Ok(())
}

键值与字符串

use wedb_embed::{prelude::*, string::Set, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/kv_demo", [])?;

    // 基础读写与过期时间
    db.set(b"site", b"webc.site", &[])?;
    let val = db.get(b"site")?;
    db.setex(b"temp_token", b"xyz", 3600_000)?; // 设置毫秒过期时间

    // 条件写入
    db.set(b"lock", b"1", &[Set::Nx])?;

    // 数值自增自减
    db.incr(b"counter")?;
    db.incrby(b"counter", 10)?;
    db.decrby(b"counter", 5)?;

    // 批量读写与原子操作
    db.mset(&[(b"k1", b"v1"), (b"k2", b"v2")])?;
    let values = db.mget(&[b"k1", b"k2"])?;
    let len = db.strlen(b"site")?;
    let old_val = db.getset(b"site", b"new_site")?;
    let del_val = db.getdel(b"site")?;
    db.cas(b"site", b"new_site", b"final_site", 0)?; // 比较并交换

    Ok(())
}

哈希与字段级过期

use wedb_embed::{hash::HExpire, prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/hash_demo", [])?;

    // 字段写入与读取
    db.hset(b"user:100", &[(b"name", b"Alice"), (b"age", b"20")])?;
    let name = db.hget(b"user:100", b"name")?;

    // 批量写入与获取
    db.hmset(b"user:100", &[(b"city", b"Beijing"), (b"role", b"Admin")])?;
    let fields = db.hmget(b"user:100", &[b"name", b"city"])?;

    // 数值自增与元数据
    db.hincrby(b"user:100", b"age", 1)?;
    let exists = db.hexists(b"user:100", b"name")?;
    let len = db.hlen(b"user:100")?;
    let all = db.hgetall(b"user:100")?;
    let keys = db.hkeys(b"user:100")?;
    let vals = db.hvals(b"user:100")?;
    db.hdel(b"user:100", &[b"role"])?;

    // 字段级独立过期时间
    db.hexpire(b"user:100", &[b"name"], 3600, HExpire::None)?;
    let ttls = db.httl(b"user:100", &[b"name"])?;

    Ok(())
}

列表

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/list_demo", [])?;

    // 双端推入与弹出
    db.lpush(b"tasks", &[b"task1", b"task2"])?;
    db.rpush(b"tasks", &[b"task3"])?;
    let first = db.lpop_one(b"tasks")?;
    let last = db.rpop_one(b"tasks")?;

    // 范围查询、修剪与修改
    let len = db.llen(b"tasks")?;
    let items = db.lrange(b"tasks", 0, -1)?;
    db.ltrim(b"tasks", 0, 10)?;
    db.lset(b"tasks", 0, b"updated_task")?;

    Ok(())
}

集合

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/set_demo", [])?;

    db.sadd(b"tags", &[b"rust", b"database", b"lsm"])?;
    let is_member = db.sismember(b"tags", b"rust")?;
    let count = db.scard(b"tags")?;
    let members = db.smembers(b"tags")?;
    db.srem(b"tags", &[b"lsm"])?;
    let popped = db.spop(b"tags", 1)?;

    // 集合代数运算
    db.sadd(b"tags_other", &[b"rust", b"storage"])?;
    let inter = db.sinter(&[b"tags", b"tags_other"])?;
    let union = db.sunion(&[b"tags", b"tags_other"])?;
    let diff = db.sdiff(&[b"tags", b"tags_other"])?;

    Ok(())
}

有序集合

use wedb_embed::{prelude::*, zset::RangeScoreSpec, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/zset_demo", [])?;

    // 添加成员与分数
    db.zadd(b"leaderboard", &[(100.0, b"player1"), (200.0, b"player2")], &[])?;
    db.zincrby(b"leaderboard", 50.0, b"player1")?;

    // 排名与分数查询
    let score = db.zscore(b"leaderboard", b"player1")?;
    let rank = db.zrank(b"leaderboard", b"player1")?;
    let rev_rank = db.zrevrank(b"leaderboard", b"player1")?;

    // 范围检索与计数
    let top = db.zrange(b"leaderboard", 0, 10)?;
    let count = db.zcount(b"leaderboard", &RangeScoreSpec::new(100.0, 300.0))?;
    let card = db.zcard(b"leaderboard")?;
    db.zrem(b"leaderboard", &[b"player1"])?;

    Ok(())
}

位图与位域

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/bitmap_demo", [])?;

    // 位设置与统计
    db.setbit(b"online_users", 1001, 1)?;
    let is_online = db.getbit(b"online_users", 1001)?;
    let online_count = db.bitcount(b"online_users", None, None)?;
    let first_online = db.bitpos(b"online_users", 1, None, None)?;

    Ok(())
}

JSON 与路径查询

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/json_demo", [])?;

    // JSON 写入与 JSONPath 表达式查询
    db.json_set(b"doc:1", "$", r#"{"user":{"name":"Alice","age":25,"roles":["admin"]}}"#)?;
    let name = db.json_get(b"doc:1", Some("$.user.name"))?;
    db.json_numincrby(b"doc:1", "$.user.age", "1.0")?;
    db.json_arrappend(b"doc:1", "$.user.roles", &[r#""editor""#])?;

    Ok(())
}

布隆与布谷鸟过滤器

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/filter_demo", [])?;

    // 布隆过滤器
    db.bf_reserve(b"bf_filter", 0.01, 10000, 2)?;
    db.bf_add(b"bf_filter", b"item_1")?;
    let exists = db.bf_exists(b"bf_filter", b"item_1")?;

    // 布谷鸟过滤器
    db.cf_reserve(b"cf_filter", 10000, 2, 500, 1)?;
    db.cf_add(b"cf_filter", b"item_2")?;
    let cf_exists = db.cf_exists(b"cf_filter", b"item_2")?;
    db.cf_del(b"cf_filter", b"item_2")?;

    Ok(())
}

时序数据与聚合

use wedb_embed::{
    prelude::*,
    timeseries::{AggregationType, DuplicatePolicy, TSRangeOption},
    Result, WeDb,
};

fn main() -> Result<()> {
    let db = WeDb::open("./data/timeseries_demo", [])?;

    // 创建时序序列
    db.ts_create(b"cpu:usage", 0, DuplicatePolicy::Last, &[("host", "server-1")])?;
    db.ts_add(b"cpu:usage", 1700000000000, 42.5)?;
    let latest = db.ts_get(b"cpu:usage")?;

    // 窗口降采样范围查询
    let samples = db.ts_range_opt(
        b"cpu:usage",
        &TSRangeOption {
            from_timestamp: 0,
            to_timestamp: 1800000000000,
            aggregation: Some((AggregationType::Avg, 60000)),
            ..Default::default()
        },
    )?;

    Ok(())
}

地理空间位置

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/geo_demo", [])?;

    // 坐标添加与距离计算
    db.geoadd(
        b"cities",
        &[
            (116.4074, 39.9042, b"Beijing"),
            (121.4737, 31.2304, b"Shanghai"),
        ],
    )?;
    let dist_km = db.geodist(b"cities", b"Beijing", b"Shanghai", Some("km"))?;
    let hash = db.geohash(b"cities", &[b"Beijing"])?;

    Ok(())
}

基数统计

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/hll_demo", [])?;

    db.pfadd(b"hll_uv", &[b"user_1", b"user_2", b"user_3"])?;
    let uv_count = db.pfcount(&[b"hll_uv"])?;
    db.pfmerge(b"hll_merged", &[b"hll_uv"])?;

    Ok(())
}

分位数统计

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/tdigest_demo", [])?;

    db.tdigest_create(b"latencies", 100.0)?;
    db.tdigest_add(b"latencies", &[12.5, 15.0, 18.2, 45.0, 99.9])?;
    let p95 = db.tdigest_quantile(b"latencies", &[0.95])?;
    let cdf = db.tdigest_cdf(b"latencies", &[20.0])?;

    Ok(())
}

紧凑整型集合

use wedb_embed::{prelude::*, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/sortedint_demo", [])?;

    // 64 位紧凑整型集合
    db.si_add(b"post:100:likes", &[1001, 1002, 1003, 1004])?;
    let has_liked = db.si_exists(b"post:100:likes", 1001)?;
    let like_count = db.si_card(b"post:100:likes")?;
    let top_likers = db.si_range(b"post:100:likes", 0, 0, 10, false)?;

    Ok(())
}

消息流与消费组

use wedb_embed::{prelude::*, stream::StreamId, Result, WeDb};

fn main() -> Result<()> {
    let db = WeDb::open("./data/stream_demo", [])?;

    // 追加消息
    let entry_id = db.xadd_simple(
        b"event_stream",
        None,
        &[(b"action", b"login"), (b"uid", b"1001")],
    )?;
    let len = db.xlen(b"event_stream")?;
    let entries = db.xrange(b"event_stream", StreamId::MIN, StreamId::MAX, Some(10))?;

    // 消费组与待处理条目统计
    db.xgroup_create(b"event_stream", "group_workers", "0-0", false, None)?;
    let pending = db.xpending_summary(b"event_stream", "group_workers")?;

    Ok(())
}

全文检索与向量检索

use wedb_embed::{
    prelude::*,
    search::{
        DistanceMetric, FtCreate, FtSearch, IndexField, IndexFieldType, IndexOnDataType,
        SearchIndexManager, VectorAlgorithm, VectorFieldMetadata, VectorType,
    },
    Result, WeDb,
};

fn main() -> Result<()> {
    let db = WeDb::open("./data/search_demo", [])?;
    let mut search_mgr = SearchIndexManager::new();

    // 创建多字段模式
    let mut schema = FtCreate::new("idx_articles", IndexOnDataType::Json);
    schema.prefixes.push("article:".to_string());
    schema.fields.push(IndexField {
        name: "title".to_string(),
        alias: None,
        field_type: IndexFieldType::Text,
        weight: 1.0,
        sortable: false,
        noindex: false,
        separator: None,
        casesensitive: false,
        withsuffixtrie: false,
        unf: false,
        vector_meta: None,
    });
    schema.fields.push(IndexField {
        name: "vec".to_string(),
        alias: None,
        field_type: IndexFieldType::Vector,
        weight: 1.0,
        sortable: false,
        noindex: false,
        separator: None,
        casesensitive: false,
        withsuffixtrie: false,
        unf: false,
        vector_meta: Some(VectorFieldMetadata {
            algorithm: VectorAlgorithm::Hnsw,
            vector_type: VectorType::Float32,
            dim: 4,
            distance_metric: DistanceMetric::Cosine,
            m: 16,
            ef_construction: 200,
            ef_runtime: 10,
            block_size: 1024,
            initial_cap: 1000,
        }),
    });
    search_mgr.create_index_from_opts(schema)?;

    // 文本搜索与向量近邻检索
    let res = search_mgr.search("idx_articles", "@title:database", &FtSearch::default())?;
    println!("Found matching docs: {}", res.total);

    Ok(())
}

技术堆栈

开发语言：Rust Edition 2024
存储引擎：fjall 分层存储持久化引擎
JSON 引擎：sonic-rs SIMD 指令集解析
非加密哈希：rapidhash 高效哈希算法
序列化编解码：bitcode 紧凑二进制序列化
字符串与内存：hipstr 紧凑字符串存储与零拷贝借用
并发哈希表：papaya 无锁并发哈希表
集合与位运算：roaring、memchr、crc32fast、fastrand
时间戳处理：coarsetime、ts_
数值与浮点序列化：zmij、itoa
枚举派生：strum
错误管理：thiserror