safebrowsing-db-redb

A persistent database backend for the Safe Browsing API using redb.

Overview

This crate provides a RedbDatabase implementation that stores Google Safe Browsing threat lists persistently on disk using the redb embedded database. Unlike the in-memory databases, this implementation:

Persists data between restarts - No need to re-download threat lists every time
Stores data in system cache directory - Uses platform-appropriate cache locations
Thread-safe with ACID transactions - Safe for concurrent access
Efficient storage - Compressed binary format with fast lookups

Features

Implements the Database trait from safebrowsing-db
Automatic cache directory management
Efficient hash prefix storage and lookup
Metadata tracking (last update time, initialization status, etc.)
Compatible with all Safe Browsing threat list types

Usage

Basic Usage

use safebrowsing_db_redb::RedbDatabase;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create database in default system cache location
    let mut db = RedbDatabase::default();

    // Or specify a custom path
    let mut db = RedbDatabase::new("/path/to/database.redb")?;

    // Use with Safe Browsing API
    // (Database trait methods available)
    Ok(())
}

With SafeBrowser

use safebrowsing::{Config, DatabaseType, SafeBrowser};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = Config {
        api_key: "your-api-key".to_string(),
        database_type: Some(DatabaseType::Redb),
        ..Default::default()
    };

    let mut sb = SafeBrowser::new(config).await?;
    // Database will be automatically stored in system cache
    Ok(())
}

Command Line Tool

The sblookup tool supports the redb backend:

# Use persistent redb database
sblookup --database-type redb --api-key YOUR_API_KEY example.com

# Database will be stored in:
# - Linux: ~/.cache/safebrowsing/database.redb
# - macOS: ~/Library/Caches/safebrowsing/database.redb
# - Windows: %LOCALAPPDATA%/safebrowsing/database.redb

Storage Location

By default, the database is stored in the system cache directory:

Linux: ~/.cache/safebrowsing/database.redb
macOS: ~/Library/Caches/safebrowsing/database.redb
Windows: %LOCALAPPDATA%\safebrowsing\database.redb

You can override this by providing a custom path to RedbDatabase::new().

Database Schema

The redb database uses three tables:

threat_lists: Stores serialized threat list entries with hash prefixes
metadata: Stores database metadata (last update time, initialization status, etc.)
hashes: Reserved for future use (direct hash lookups)

Performance

First run: Downloads and stores threat lists (may take a few minutes)
Subsequent runs: Fast startup using cached data
Updates: Incremental updates from Google's API
Lookups: Fast hash prefix lookups with in-memory caching

Thread Safety

The RedbDatabase is thread-safe and supports concurrent reads and writes through redb's ACID transaction system. Multiple processes can safely access the same database file.

Error Handling

The implementation maps redb errors to DatabaseError types:

DatabaseError::IoError - File system or permission errors
DatabaseError::DecodeError - Serialization/deserialization errors
DatabaseError::Stale - Database needs updating
DatabaseError::NotReady - Database not initialized

Requirements

Rust 1.70+
Tokio async runtime
Read/write access to cache directory

Dependencies

redb - Embedded database engine
dirs - System directory detection
serde - Serialization framework
tokio - Async runtime
safebrowsing-* - Safe Browsing API components

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option.

safebrowsing-db-redb 0.1.0