# Hero Redis
A high-performance, Redis-compatible server built on [redb](https://github.com/cberner/redb) with ChaCha20-Poly1305 encryption and dual authentication modes.
## Features
- **Redis Protocol Compatible** - Works with any Redis client including Redis Desktop Manager
- **Dual Authentication**:
- **Simple**: Secret-based auth with `AUTH <secret>` (recommended for most users)
- **Advanced**: Ed25519 signature-based auth (for high-security environments)
- **Multi-Database with ACL** - Per-database Read/Write/Admin permissions
- **Persistent Storage** - Data stored in redb (pure Rust embedded database)
- **Encryption** - All values encrypted with ChaCha20-Poly1305
- **Multiple Databases** - Up to 1000 databases with lazy loading
- **Low Memory** - ~3MB at startup, databases loaded on demand
- **Auto-cleanup** - Idle databases automatically closed after 5 minutes
- **Unix Socket & TCP** - Both connection methods supported
- **Vector Search** - HNSW-based similarity search with hannoy library
- **Cross-platform** - Linux (x86_64, aarch64) and macOS (x86_64, aarch64)
## Quick Start
### Build from Source
```bash
git clone https://github.com/herocode/hero_redis.git
cd hero_redis
cargo build --release
```
### Start Server (Simple Auth)
```bash
# Start with admin secret (simplest way)
./target/release/hero_redis \
--encryption-key "your-encryption-key" \
--admin-secret "your-admin-secret" \
--data-dir ~/.hero_redis
```
### Connect and Use
```bash
# Using redis-cli (default port is 6666)
redis-cli -p 6666
# Authenticate with your secret
> AUTH your-admin-secret
OK
# Now use any Redis command
> SET mykey myvalue
OK
> GET mykey
"myvalue"
```
### Using with Redis Desktop Manager
1. Start server with `--admin-secret`
2. In Redis Desktop Manager, configure connection:
- Host: 127.0.0.1
- Port: 6666
- Auth: your-admin-secret
3. Connect and use normally
## Authentication Modes
### 1. Simple Auth (Recommended)
Start the server with `--admin-secret`:
```bash
./target/release/hero_redis \
--encryption-key "encryption-key" \
--admin-secret "my-secret-password"
```
Connect with `AUTH`:
```
AUTH my-secret-password
```
Create users with secrets:
```
USER.CREATESECRET alice alice-secret-123
USER.GRANT 1 alice write
```
Users authenticate with:
```
AUTH alice-secret-123
```
### 2. Advanced Auth (Ed25519)
For high-security environments, use Ed25519 signature-based authentication:
```bash
# Generate keypair
./target/release/hero_redis_login --generate
# Start server with pubkey
./target/release/hero_redis \
--encryption-key "encryption-key" \
--admin-pubkey "a1b2c3d4..."
```
Authentication flow:
```
CHALLENGE → returns random challenge
TOKEN <pubkey> <signature> → returns session token
AUTH <token> → authenticates session
```
## Command Line Options
### Server (hero_redis)
| `-d, --data-dir` | `~/.hero_redis` | Database directory |
| `-s, --socket` | `~/.hero_redis/redis.sock` | Unix socket path |
| `-p, --port` | `6666` | TCP port (0 to disable) |
| `--encryption-key` | _required_ | Encryption key for DB 0 |
| `--admin-secret` | - | Admin secret for simple auth |
| `--admin-pubkey` | - | Admin public key for Ed25519 auth |
| `-v, --verbose` | false | Enable debug logging |
Note: At least one of `--admin-secret` or `--admin-pubkey` must be provided.
## User Management
### Create Users (Secret-based)
```
# As admin, create a user with a secret
USER.CREATESECRET <username> <secret>
# Example
USER.CREATESECRET alice mysecretpassword
```
### Create Users (Ed25519-based)
```
# As admin, register a user's public key
USER.CREATE <pubkey>
# Example
USER.CREATE a1b2c3d4e5f6...
```
### Grant Permissions
```
# Grant access to a database
USER.GRANT <db_number> <username_or_pubkey> <read|write|admin>
# Examples
USER.GRANT 1 alice write
USER.GRANT 2 bob read
USER.GRANT 1 a1b2c3d4e5f6... admin
```
### Revoke Permissions
```
USER.REVOKE <db_number> <username_or_pubkey>
# Example
USER.REVOKE 1 alice
```
## Permission Levels
| **read** | Yes | No | No | No | No |
| **write** | Yes | Yes | No | No | No |
| **admin** | Yes | Yes | Yes | Yes (same db) | No |
| **server admin** | Yes | Yes | Yes | Yes | Yes |
## Admin Commands
### Server Admin Commands
| `DATABASE.CREATE <encryption_key>` | Create new database, returns db number |
| `DATABASE.STATUS [db\|all]` | Show database info |
| `DATABASE.PUBLIC <db> <on\|off>` | Enable/disable public read-only access |
| `USER.CREATE <pubkey>` | Register Ed25519 user |
| `USER.CREATESECRET <username> <secret>` | Create secret-based user |
| `USER.DELETE <username_or_pubkey>` | Delete a user |
| `ADMIN.ADD <pubkey>` | Add server admin (Ed25519) |
| `ADMIN.REMOVE <pubkey>` | Remove server admin |
| `ADMIN.LIST` | List server admins |
### Public Read-Only Access
You can make a database publicly readable (no authentication required for reads):
```bash
# As admin, make database 1 public
DATABASE.PUBLIC 1 on
# Now anyone can read without AUTH
SELECT 1
GET publickey # Works without authentication
SET key value # Fails - writes still require auth
```
### Database Admin Commands
| `USER.GRANT <db> <user> <perm>` | Grant permission (read/write/admin) |
| `USER.REVOKE <db> <user>` | Revoke permission |
| `FLUSHDB` | Clear current database |
## Supported Redis Commands
### Authentication
`AUTH`, `SAUTH`, `CHALLENGE`, `TOKEN`
### String Commands
`GET`, `SET`, `MGET`, `MSET`, `DEL`, `EXISTS`, `EXPIRE`, `TTL`, `KEYS`, `SCAN`, `INCR`, `INCRBY`, `DECR`, `DECRBY`, `APPEND`, `STRLEN`, `GETRANGE`, `SETNX`, `SETEX`, `GETSET`, `TYPE`
### Hash Commands
`HSET`, `HGET`, `HMSET`, `HMGET`, `HGETALL`, `HDEL`, `HEXISTS`, `HLEN`, `HKEYS`, `HVALS`, `HINCRBY`, `HSETNX`, `HSCAN`
### List Commands
`LPUSH`, `RPUSH`, `LPOP`, `RPOP`, `LRANGE`, `LLEN`, `LINDEX`, `LSET`, `LREM`
### Set Commands
`SADD`, `SREM`, `SMEMBERS`, `SISMEMBER`, `SCARD`, `SPOP`, `SUNION`, `SINTER`, `SDIFF`, `SSCAN`
### Stream Commands
`XADD`, `XLEN`, `XRANGE`, `XREVRANGE`, `XREAD`, `XINFO`, `XTRIM`, `XGROUP`, `XREADGROUP`, `XACK`, `XPENDING`
### Vector Commands
`VECTOR.CREATE`, `VECTOR.ADD`, `VECTOR.SEARCH`, `VECTOR.SEARCHBYID`, `VECTOR.GET`, `VECTOR.DEL`, `VECTOR.BUILD`, `VECTOR.INFO`, `VECTOR.LIST`, `VECTOR.DROP`, `VECTOR.CLEAR`, `VECTOR.EXISTS`, `VECTOR.LEN`
### Connection Commands
`PING`, `AUTH`, `SAUTH`, `SELECT`, `QUIT`, `COMMAND`
### Management Commands
`COMPACT`, `SHUTDOWN`, `MEMORY USAGE`, `MEMORY STATS`, `DBSIZE`, `FLUSHDB`, `INFO`, `CONFIG`, `CLIENT`
## Vector Search
Hero Redis includes HNSW-based vector similarity search using the [hannoy](https://crates.io/crates/hannoy) library. Perfect for AI embeddings, semantic search, and RAG applications.
### Quick Start
```bash
# Create a vector index (128 dimensions, cosine similarity)
VECTOR.CREATE embeddings 128 METRIC cosine
# Add vectors (JSON format)
VECTOR.ADD embeddings 1 [0.1, 0.2, 0.3, ...]
VECTOR.ADD embeddings 2 [0.4, 0.5, 0.6, ...]
# Build the HNSW index (required before searching)
VECTOR.BUILD embeddings
# Search for similar vectors (returns top 10)
VECTOR.SEARCH embeddings [0.1, 0.2, 0.3, ...] 10
# Search by existing vector ID
VECTOR.SEARCHBYID embeddings 1 10
```
### High-Performance Bulk Insert
For bulk loading, use `VECTOR.ADDBATCH` with binary format - **~97x faster** than individual adds:
| Individual VECTOR.ADD | ~220 vec/s | ~45 sec |
| Binary VECTOR.ADDBATCH | ~21,000 vec/s | ~0.5 sec |
Binary format (little-endian):
```
Header (8 bytes):
- num_vectors: u32 (4 bytes)
- dimensions: u32 (4 bytes)
Per vector:
- id: u32 (4 bytes)
- vector: f32 × dimensions
```
### Rhai Script Example
```rhai
// Create index
vector_create(r, "embeddings", 128, "cosine");
// Batch add (fast!)
let batch = [];
for id in 1..=1000 {
batch.push([id, generate_embedding(id)]);
}
vector_add_batch(r, "embeddings", 128, batch);
// Build and search
vector_build(r, "embeddings");
let results = vector_search(r, "embeddings", query_vector, 10);
```
### Rust Client Example
```rust
use hero_redis::HeroRedisClient;
let mut client = HeroRedisClient::new("127.0.0.1", 6666, private_key)?;
client.select(1)?;
// Create index
client.vector_create("embeddings", 128, "cosine")?;
// Bulk add with binary format (fast!)
let items: Vec<(u32, Vec<f32>)> = embeddings.into_iter().enumerate()
.map(|(i, v)| (i as u32, v))
.collect();
client.vector_add_batch("embeddings", 128, &items)?;
// Build and search
client.vector_build("embeddings")?;
let results = client.vector_search("embeddings", &query, 10)?;
```
### Vector Commands Reference
| `VECTOR.CREATE <index> <dims> METRIC <cosine\|euclidean\|manhattan>` | Create index |
| `VECTOR.ADD <index> <id> <json_vector>` | Add single vector |
| `VECTOR.ADDBATCH <index> <binary_data>` | Bulk add (binary format, ~97x faster) |
| `VECTOR.BUILD <index>` | Build HNSW graph (required before search) |
| `VECTOR.SEARCH <index> <json_vector> <k>` | Find k nearest neighbors |
| `VECTOR.SEARCHBYID <index> <id> <k>` | Find neighbors of existing vector |
| `VECTOR.GET <index> <id>` | Get vector by ID |
| `VECTOR.DEL <index> <id>` | Delete vector |
| `VECTOR.INFO <index>` | Get index statistics |
| `VECTOR.LEN <index>` | Get vector count |
| `VECTOR.LIST` | List all indexes |
| `VECTOR.DROP <index>` | Delete index |
| `VECTOR.CLEAR <index>` | Remove all vectors |
### Distance Metrics
| `cosine` | Cosine similarity | Text embeddings (OpenAI, etc.) |
| `euclidean` | L2 distance | Image embeddings |
| `manhattan` | L1 distance | Sparse vectors |
### Common Embedding Dimensions
| OpenAI text-embedding-ada-002 | 1536 |
| OpenAI text-embedding-3-small | 1536 |
| Cohere embed-english-v3.0 | 1024 |
| all-MiniLM-L6-v2 | 384 |
| BGE-base-en | 768 |
See [docs/ai_embeddings.md](docs/ai_embeddings.md) for complete documentation.
## Using as a Library
Hero Redis supports feature flags to include only what you need, reducing binary size and compile times.
### Feature Flags
| `client` | Redis client with Ed25519 authentication | Yes |
| `server` | Redis-compatible server with redb backend | No |
| `rhai` | Rhai scripting support for the client | No |
| `vector` | Vector database with HNSW similarity search | No |
| `full` | All features enabled | No |
### Usage Examples
```toml
# Client only (default) - minimal dependencies
[dependencies]
hero_redis = "0.2"
# Server only (no client)
[dependencies]
hero_redis = { version = "0.2", default-features = false, features = ["server"] }
# Client with Rhai scripting
[dependencies]
hero_redis = { version = "0.2", features = ["rhai"] }
# Client with vector support
[dependencies]
hero_redis = { version = "0.2", features = ["vector"] }
# Everything
[dependencies]
hero_redis = { version = "0.2", features = ["full"] }
```
### Client Example
```rust
use hero_redis::hero_redis_client::HeroRedisClient;
// Connect with your private key
let mut client = HeroRedisClient::new("127.0.0.1", 6666, "your_private_key_hex")?;
client.select(1)?;
// String operations
client.set("name", "Alice")?;
let name = client.get("name")?.unwrap();
// Hashes
client.hset("user:1", "name", "Bob")?;
let user = client.hgetall("user:1")?;
// Lists
client.rpush("queue", "job1")?;
let job = client.lpop("queue")?;
// Sets
client.sadd("tags", "rust")?;
let tags = client.smembers("tags")?;
// Admin operations (create database, manage users)
let db_num = client.admin().create_database("my-encryption-key")?;
client.admin().grant_permission(db_num, &user_pubkey, Permission::Write)?;
```
### Rhai Scripting Example
With `features = ["rhai"]`:
```rust
use hero_redis::hero_redis_rhai::RhaiScriptRunner;
let runner = RhaiScriptRunner::builder()
.host("127.0.0.1")
.port(6666)
.private_key("your_private_key_hex")
.build()?;
runner.run_script("scripts/my_script.rhai")?;
```
Example Rhai script:
```rhai
// Select database
select(r, 1);
// Basic operations
set(r, "counter", "0");
incr(r, "counter");
let val = get(r, "counter");
print(`Counter: ${val}`);
// Hashes
hset(r, "user:1", "name", "Alice");
hset(r, "user:1", "email", "alice@example.com");
let user = hgetall(r, "user:1");
print(user);
```
## Data Storage
- Data directory: `~/.hero_redis/` (configurable)
- Database files: `db0.redb`, `db1.redb`, ... `db999.redb`
- DB 0 is reserved for admin metadata
- Each database has its own encryption key
- Values are encrypted with ChaCha20-Poly1305
## License
Apache 2.0