fail2ban-log-parser

Fast, zero-copy fail2ban log parser with bindings for Rust, Python, and JavaScript.

Parse fail2ban logs at ~142 ns/line with 19 heap allocations per million lines, Fail2BanStructuredLog borrows directly from the input, so only the output Vec and its backing buffer are heap-allocated. Enable the parallel feature for up to 5.4× speedup on large inputs via Rayon.

Installation

Rust

cargo add fail2ban-log-parser-core

With serde support:

cargo add fail2ban-log-parser-core --features serde

With parallel parsing (multi-threaded via Rayon):

cargo add fail2ban-log-parser-core --features parallel

Python

pip install fail2ban-log-parser

Requires Python 3.10+. Prebuilt wheels are available for Linux (x86_64, aarch64), macOS (aarch64), and Windows (x86_64).

Usage

Rust

Parse a single line

use fail2ban_log_parser_core::parse;

let line = "2024-01-15 14:32:01,847 fail2ban.filter [12345] INFO [sshd] Found 192.168.1.1";
let log = parse(line).next().unwrap().unwrap();

assert_eq!(log.jail(), Some("sshd"));
assert_eq!(log.pid(), Some(12345));
println!("{:?} {:?} {:?}", log.event(), log.ip(), log.timestamp());

Note: Parsed results borrow from the input &str, the input buffer must outlive the iterator and any collected results.

Parse a batch and handle errors

use fail2ban_log_parser_core::parse;

let input = "\
2024-01-15 14:32:01,847 fail2ban.filter [12345] INFO [sshd] Found 192.168.1.100
this line is not a valid log entry
2024-01-15 14:32:03,456 fail2ban.actions [12345] NOTICE [sshd] Ban 192.168.1.100";

for result in parse(input) {
    match result {
        Ok(log) => println!("{:?} {:?}", log.event(), log.ip()),
        Err(e) => eprintln!("parse error: {e}"),
    }
}

Filter specific events

use fail2ban_log_parser_core::{Fail2BanEvent, parse};

let input = "..."; // multi-line log
let bans: Vec<_> = parse(input)
    .filter_map(|r| r.ok())
    .filter(|log| log.event() == Some(&Fail2BanEvent::Ban))
    .collect();

Python

Parse a single line

from fail2ban_log_parser import parse, Event

line = "2024-01-15 14:32:01,847 fail2ban.filter [12345] INFO [sshd] Found 192.168.1.1"
logs, errors = parse(line)

log = logs[0]
print(log.timestamp)  # "2024-01-15T14:32:01.847+00:00"
print(log.jail)       # "sshd"
print(log.event)      # Event.Found
print(log.ip)         # "192.168.1.1"
print(log.pid)        # 12345

Parse a batch and handle errors

from fail2ban_log_parser import parse

input_text = """\
2024-01-15 14:32:01,847 fail2ban.filter [12345] INFO [sshd] Found 192.168.1.100
this line is not a valid log entry
2024-01-15 14:32:03,456 fail2ban.actions [12345] NOTICE [sshd] Ban 192.168.1.100"""

logs, errors = parse(input_text)

for log in logs:
    print(f"{log.event} {log.ip} in jail {log.jail}")

for err in errors:
    print(f"Line {err.line_number}: {err.line}")

Filter bans

from fail2ban_log_parser import parse, Event

logs, _ = parse(open("/var/log/fail2ban.log").read())
bans = [log for log in logs if log.event == Event.Ban]

for ban in bans:
    print(f"Banned {ban.ip} from {ban.jail} at {ban.timestamp}")

Group events by jail

from collections import defaultdict
from fail2ban_log_parser import parse

logs, _ = parse(open("/var/log/fail2ban.log").read())

by_jail = defaultdict(list)
for log in logs:
    if log.jail:
        by_jail[log.jail].append(log)

for jail, entries in by_jail.items():
    print(f"{jail}: {len(entries)} events")

Log structure

2024-01-15 14:32:01,847  fail2ban.filter  [12345]  INFO  [sshd] Found 1.2.3.4
|___________________|    |______________|  |____|   |__|  |____| |__| |______|
    timestamp              header            pid    level  jail  event  IP

Supported formats

API

Rust

Python

Features

How it works

flowchart TD
    A["&str (multi-line log)"] --> B["parse()"]
    B --> C{parallel feature?}
    C -->|No| D["Split into lines (lazy)"]
    C -->|Yes| E["Collect lines + Rayon par_iter"]
    D --> F["parse_log_line() per line"]
    E --> F

    F --> T["timestamp"]
    F --> H["header"]
    F --> P["pid"]
    F --> L["level"]
    F --> J["jail"]
    F --> EV["event"]
    F --> I["ip"]

    T & H & P & L & J & EV & I --> R{Result}
    R -->|Ok| S["Fail2BanStructuredLog"]
    R -->|Err| ERR["ParseError"]

    S & ERR --> O["Iterator of Results"]

The parser is built with winnow, a fast parser combinator library. Each log line is parsed in a single pass with no intermediate allocations, Fail2BanStructuredLog holds &str slices into the original input rather than owned strings.

Benchmarks

Measured with Criterion.rs + dhat on Apple M4 Pro (12 cores) / 48 GB / rustc 1.94.0.

📈 View full benchmark history

Single-line parsing

Batch parsing — sequential (default)

Single-threaded, lazy iterator. Each line is parsed on demand.

Batch parsing — parallel (--features parallel)

Multi-threaded via Rayon. Lines are parsed concurrently, then yielded in order.

Parallel overhead makes it slower than sequential for inputs under ~1,000 lines. Speedup scales with core count.

Collection strategies (1,000 lines, sequential)

Error handling

Memory usage (sequential, collect to Vec)

Heap allocations measured with dhat. The low allocation count reflects the zero-copy design, only the Vec itself and its backing buffer are heap-allocated; all string fields borrow from the input.

Reproduce

# Sequential benchmarks
cargo bench -p fail2ban-log-parser-core --bench parsing

# Parallel benchmarks
cargo bench -p fail2ban-log-parser-core --bench parsing --features parallel

Examples

cargo run -p fail2ban-log-parser-core --example parse_single
cargo run -p fail2ban-log-parser-core --example parse_batch
cargo run -p fail2ban-log-parser-core --example filter_bans

License

Apache-2.0, see LICENSE.