matchy-format 2.0.0

Binary format and MMDB builder for matchy databases (internal)
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91

# matchy-format

Unified database format orchestrating IP tries, glob patterns, and literal string matching.

## Overview

This crate provides the binary format for matchy databases (.mxy files), combining three routing layers into a single memory-mapped file:
- **IP trie** (from matchy-ip-trie) - CIDR/IP → data offsets
- **Glob patterns** (from matchy-paraglob) - Pattern matching → data offsets  
- **Literal hash** (from matchy-literal-hash) - Exact strings → data offsets

## Architecture

```
.mxy File Format:
┌─────────────────────────────┐
│ IP Search Tree (binary trie)│  ← matchy-ip-trie
├─────────────────────────────┤
│ Data Section (deduplicated) │  ← matchy-data-format
├─────────────────────────────┤
│ MMDB_PATTERN separator      │
├─────────────────────────────┤
│ Paraglob Section (optional) │  ← matchy-paraglob
│   - AC automaton            │
│   - Pattern entries         │
│   - Glob segments           │
├─────────────────────────────┤
│ Literal Hash (optional)     │  ← matchy-literal-hash
│   - 96-bit hash table       │
│   - Pattern mappings        │
├─────────────────────────────┤
│ MMDB Metadata               │
└─────────────────────────────┘
```

## Features

- **Unified format**: Single file for all data types
- **Memory-mapped**: Zero-copy loading
- **MMDB compatible**: Backward compatible with MaxMind DB format
- **Builder API**: High-level interface for database construction
- **FormatError**: Proper error type for format operations

## Usage

```rust
use matchy_format::{MmdbBuilder, MatchMode};
use matchy_data_format::DataValue;
use std::collections::HashMap;

let mut builder = MmdbBuilder::new(MatchMode::CaseSensitive);

// Add IP entry
let mut data = HashMap::new();
data.insert("country".to_string(), DataValue::String("US".to_string()));
builder.add_entry("1.2.3.4", data)?;

// Add pattern entry
let mut data = HashMap::new();
data.insert("category".to_string(), DataValue::String("malware".to_string()));
builder.add_entry("*.evil.com", data)?;

// Build database
let db_bytes = builder.build()?;
std::fs::write("database.mxy", db_bytes)?;
```

## Entry Type Detection

The builder automatically detects entry types:
- **IP addresses**: `1.2.3.4`, `192.168.0.0/16`, `2001:db8::/32`
- **Literal strings**: `example.com` (exact match only)
- **Glob patterns**: `*.example.com`, `file?.txt` (wildcard matching)

## Components

- `mmdb_builder.rs` - High-level database builder
- `mmdb/` - MMDB format structures
- `offset_format.rs` - Binary format definitions
- `endian.rs` - Endianness handling
- `mmap.rs` - Memory-mapped file wrapper
- `error.rs` - FormatError type

## Dependencies

- `matchy-ip-trie` - IP address routing
- `matchy-paraglob` - Glob pattern matching
- `matchy-literal-hash` - Exact string matching
- `matchy-data-format` - Data encoding/decoding
- `matchy-glob` - Glob parsing
- `matchy-match-mode` - Shared MatchMode enum