spam-db

Rust library for reading spam databases.

SPAM indexes Nix package closures and nixosOptionsDoc output into compressed, bucket-indexed databases. This crate lets you open those databases and run substring queries against them.

Usage

[dependencies]
spam-db = "0.2"

Query an options database

use spam_db::OptionsDb;

let db = OptionsDb::open("options.db")?;
for rec in db.query("services.nginx")? {
    println!("{}", rec.name);
    if let Some(summary) = rec.summary {
        println!("  {summary}");
    }
}

Query a packages database

use spam_db::PackagesDb;

let db = PackagesDb::open("files.db")?;
for rec in db.query("/bin/")? {
    println!("{} -> {}", rec.path, rec.packages.join(", "));
}

Auto-detect database kind

use spam_db::SpamDb;

match SpamDb::open("unknown.db")? {
    SpamDb::Options(db) => { /* ... */ }
    SpamDb::Packages(db) => { /* ... */ }
}

Database format

A spam database is a binary file with three sections:

# spam-db-v2\t{options|packages}\n
[256 x 16-byte index entries: (offset: u64le, length: u64le)]
[concatenated zstd-compressed bucket blobs]

Each line in the database is placed in every bucket corresponding to a unique byte in its search key. Queries decompress only the bucket for query[0], keeping lookup sublinear in the total database size.

Building spam databases

Use the spam CLI:

spam db build --manifest packages.json --output files.db
spam db build --manifest options.json --output options.db