parex

Blazing-fast parallel search engine — generic, embeddable, zero opinions.

parex is a Rust library that owns the parallel walk engine, the trait contracts, and the error type. It does not own filesystem logic, output formatting, or built-in matchers — those belong to the caller.

Built to power ldx — a parallel file search CLI.

Features

• Parallel traversal via a clean Source trait — search files, databases, memory, anything
• Custom matching via a Matcher trait — substring, regex, fuzzy, metadata, ML scoring
• Typed error handling with is_recoverable() / is_fatal() — callers decide what to skip vs halt
• Opt-in path and error collection — zero allocation overhead when unused
• Results are explicitly unordered — parallel traversal does not guarantee output order
• #![forbid(unsafe_code)]

Install

cargo add parex

Quick Start

Implement Source for whatever you want to search:

use parex::{Source, Entry, EntryKind, ParexError};
use parex::engine::WalkConfig;

struct VecSource(Vec<&'static str>);

impl Source for VecSource {
    fn walk(&self, _config: &WalkConfig) -> Box<dyn Iterator<Item = Result<Entry, ParexError>>> {
        let entries = self.0.iter().map(|name| Ok(Entry {
            path:     name.into(),
            kind:     EntryKind::File,
            depth:    0,
            metadata: None,
        })).collect::<Vec<_>>();
        Box::new(entries.into_iter())
    }
}

Run a search:

let results = parex::search()
    .source(VecSource(vec!["invoice_jan.txt", "invoice_feb.txt", "report.txt"]))
    .matching("invoice")
    .limit(50)
    .threads(8)
    .collect_paths(true)
    .collect_errors(true)
    .run()?;

println!("Found {} matches in {}ms",
    results.matches,
    results.stats.duration.as_millis()
);

for path in &results.paths {
    println!("  {}", path.display());
}

Custom Matchers

use parex::{Matcher, Entry};

struct ExtensionMatcher(String);

impl Matcher for ExtensionMatcher {
    fn is_match(&self, entry: &Entry) -> bool {
        entry.path
            .extension()
            .map(|e| e.eq_ignore_ascii_case(&self.0))
            .unwrap_or(false)
    }
}

let results = parex::search()
    .source(my_source)
    .with_matcher(ExtensionMatcher("rs".into()))
    .collect_paths(true)
    .run()?;

Builder API

Error Handling

for err in &results.errors {
    if let Some(path) = err.path() {
        eprintln!("Error at: {}", path.display());
    }
    if err.is_recoverable() {
        // permission denied, not found, symlink loop — safe to skip
    }
    if err.is_fatal() {
        // thread pool failure, invalid source — halt immediately
    }
}

Design

parex owns the walk engine, trait contracts, error type, and builder API. It does not own filesystem logic, output formatting, or concrete matchers — those live in the tool built on top.

Source and Matcher are the extension points. A caller wanting to search a database, an API, or a pre-built index just implements Source — the engine handles threading, result collection, and early exit transparently.

For filesystem traversal, parawalk is the recommended Source implementation — a minimal parallel directory walker designed to pair with parex.

See DOCS.md for the full architecture guide, custom source examples, and embedding parex in your own project.

License

MIT — see LICENSE