Fail2Ban log parser

[!IMPORTANT] This library is still WIP.

Installation

cargo add fail2ban-log-parser-core

With serde support:

cargo add fail2ban-log-parser-core --features serde

With parallel parsing (multi-threaded via Rayon):

cargo add fail2ban-log-parser-core --features parallel

Usage

Parse a single line

use fail2ban_log_parser_core::parse;

let line = "2024-01-15 14:32:01,847 fail2ban.filter [12345] INFO [sshd] Found 192.168.1.1";
let log = parse(line).next().unwrap().unwrap();

assert_eq!(log.jail(), Some("sshd"));
assert_eq!(log.pid(), Some(12345));
println!("{:?} {:?} {:?}", log.event(), log.ip(), log.timestamp());

Parse a batch and handle errors

use fail2ban_log_parser_core::parse;

let input = "\
2024-01-15 14:32:01,847 fail2ban.filter [12345] INFO [sshd] Found 192.168.1.100
this line is not a valid log entry
2024-01-15 14:32:03,456 fail2ban.actions [12345] NOTICE [sshd] Ban 192.168.1.100";

for result in parse(input) {
    match result {
        Ok(log) => println!("{:?} {:?}", log.event(), log.ip()),
        Err(e) => eprintln!("parse error: {e}"),
    }
}

Filter specific events

use fail2ban_log_parser_core::{Fail2BanEvent, parse};

let input = "..."; // multi-line log
let bans: Vec<_> = parse(input)
    .filter_map(|r| r.ok())
    .filter(|log| log.event() == Some(&Fail2BanEvent::Ban))
    .collect();

Log structure

2024-01-15 14:32:01,847  fail2ban.filter  [12345]  INFO  [sshd] Found 1.2.3.4
|___________________|    |______________|  |____|   |__|  |____| |__| |______|
    timestamp              header            pid    level  jail  event  IP address

Supported formats

API

Features

Examples

cargo run -p fail2ban-log-parser-core --example parse_single
cargo run -p fail2ban-log-parser-core --example parse_batch
cargo run -p fail2ban-log-parser-core --example filter_bans

How it works

flowchart TD
    A["&str (multi-line log)"] --> B["parse()"]
    B --> C{parallel feature?}
    C -->|No| D["Split into lines (lazy)"]
    C -->|Yes| E["Collect lines + Rayon par_iter"]
    D --> F["parse_log_line() per line"]
    E --> F

    F --> T["timestamp"]
    F --> H["header"]
    F --> P["pid"]
    F --> L["level"]
    F --> J["jail"]
    F --> EV["event"]
    F --> I["ip"]

    T & H & P & L & J & EV & I --> R{Result}
    R -->|Ok| S["Fail2BanStructuredLog"]
    R -->|Err| ERR["ParseError"]

    S & ERR --> O["Iterator of Results"]

Benchmarks

Measured with Criterion.rs + dhat on Apple M4 Pro (12 cores) / 48 GB / rustc 1.94.0.

Performance evolution can be seen here.

Single-line parsing

Time to parse one log line in isolation.

Batch parsing

Sequential (default)

Single-threaded, lazy iterator. Each line is parsed on demand.

Parallel (--features parallel)

Multi-threaded via Rayon. All lines are parsed concurrently, then yielded in order.

Parallel overhead makes it slower than sequential for small inputs (<1,000 lines). The speedup scales with core count, expect different results on different hardware.

Collection strategies

Time to process 1,000 lines (sequential) with different consumption patterns.

Error handling

Memory usage (sequential, collect to Vec)

Heap allocations when collecting all parsed results into a Vec. Measured with dhat.

The low alloc count (19 for 1M lines) is because Fail2BanStructuredLog borrows from the input, only the Vec itself and its backing buffer are heap-allocated.

Reproduce

# Sequential benchmarks
cargo bench -p fail2ban-log-parser-core --bench parsing

# Parallel benchmarks (requires parallel feature)
cargo bench -p fail2ban-log-parser-core --bench parsing --features parallel