spam-db 0.2.3

Parser and query library for SPAM databases
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89

# spam-db

Rust library for reading [spam](https://github.com/feel-co/spam) databases.

SPAM indexes Nix package closures and `nixosOptionsDoc` output into compressed,
bucket-indexed databases. This crate lets you open those databases and run
substring queries against them.

## Usage

```toml
[dependencies]
spam-db = "0.2"
```

### Query an options database

```rust
use spam_db::OptionsDb;

let db = OptionsDb::open("options.db")?;
for rec in db.query("services.nginx")? {
    println!("{}", rec.name);
    if let Some(summary) = rec.summary {
        println!("  {summary}");
    }
}
```

### Query a packages database

```rust
use spam_db::PackagesDb;

let db = PackagesDb::open("files.db")?;
for rec in db.query("/bin/")? {
    println!("{} -> {}", rec.path, rec.packages.join(", "));
}
```

### Auto-detect database kind

```rust
use spam_db::SpamDb;

match SpamDb::open("unknown.db")? {
    SpamDb::Options(db) => { /* ... */ }
    SpamDb::Packages(db) => { /* ... */ }
    SpamDb::Index(db) => { /* ... */ }
}
```

## Database format

`options` and `packages` databases are bucket-indexed binary files:

```plaintext
# spam-db-v3\t{options|packages}\n
[256 x 16-byte index entries: (offset: u64le, length: u64le)]
[concatenated zstd-compressed bucket blobs]
```

Each line in the database is placed in every bucket corresponding to a unique
byte in its search key. Queries decompress only the bucket for `query[0]`,
keeping lookup sublinear in the total database size.

`index` databases are compact package streams:

```plaintext
# spam-db-v3\tindex\n
[one zstd-compressed package stream]
```

The stream groups entries by package and prefix-delta encodes sorted paths to
avoid the path-record duplication used by the bucketed format.

The `packages` kind is produced by `spam db build` from local package manifests.
The `index` kind is produced by `spam index` from nixpkgs and binary-cache file
listings. `PackagesDb` can query both kinds, but consumers can distinguish them
with `SpamDb::kind()`.

## Building spam databases

Use the [spam CLI](https://github.com/feel-co/spam):

```bash
spam db build --manifest packages.json --output files.db
spam db build --manifest options.json --output options.db
```