Weavatrix Scan

weavatrix-scan is a deterministic, read-only repository scanner for static analysis, code intelligence, indexing, and AI tooling.

It does more than walk a directory. A scan produces a stable manifest with normalized paths, file sizes, optional content hashes, an aggregate revision, and explicit evidence explaining why files were skipped. The default build has zero runtime dependencies.

Why another repository walker?

walkdir and jwalk are excellent traversal libraries. ignore adds mature Git-style filtering. weavatrix-scan targets the next layer: the repeatable, auditable source manifest an analyzer needs before parsing begins.

Choose a traversal crate when you only need paths. Choose weavatrix-scan when downstream results must be reproducible and explainable.

Install

[dependencies]
weavatrix-scan = "0.1"

Enable serialization only when needed:

[dependencies]
weavatrix-scan = { version = "0.1", features = ["serde"] }

Quick start

use weavatrix_scan::{ScanOptions, Scanner};

let options = ScanOptions::default()
    .with_extensions(["rs", "go", "ts", "py"])
    .with_parallelism(0);

let report = Scanner::new(".").options(options).scan()?;

println!("revision: {}", report.revision);
for file in &report.files {
    println!(
        "{}: {} bytes, hash={}",
        file.relative,
        file.bytes,
        file.content_hash.as_deref().unwrap_or("disabled")
    );
}
for skipped in &report.skipped {
    println!("skipped {}: {:?}", skipped.relative, skipped.kind);
}
# Ok::<(), weavatrix_scan::Error>(())

For the fastest path-only discovery, disable content reads:

use weavatrix_scan::{ScanOptions, Scanner};

let report = Scanner::new(".")
    .options(
        ScanOptions::default()
            .with_extensions(["rs", "go", "ts"])
            .metadata_only(),
    )
    .scan()?;

assert!(report.files.iter().all(|file| file.content_hash.is_none()));
# Ok::<(), weavatrix_scan::Error>(())

Scan modes

The same scanner supports three useful cost levels:

Content inspection uses available CPU parallelism by default. Set .with_parallelism(1) for a serial run or pass a fixed worker count when a host application owns the wider scheduling policy.

Output contract

ScanReport contains:

• root: canonical absolute repository root;
• files: stable, lexicographically sorted ScannedFile values;
• skipped: stable, sorted evidence for excluded entries;
• warnings: non-fatal ignore-file diagnostics;
• revision: FNV-1a digest over selected relative paths and optional content hashes.

Each ScannedFile contains an absolute path, slash-normalized repository path, byte size, and optional content hash. Default hashes are deterministic FNV-1a digests intended for change detection, not cryptographic verification.

SkipKind distinguishes:

• Binary
• Extension
• Ignored
• Oversized
• PathEscape
• StandardDirectory
• Symlink

This distinction matters to analyzers: "not selected by policy" is different from "unreadable" or "outside the repository."

Configuration

ScanOptions exposes:

The standard directory policy skips:

.git .hg .svn .venv __pycache__ build coverage dist
node_modules target vendor

Disable it when another layer owns generated-directory policy:

use weavatrix_scan::{ScanOptions, StandardSkips};

let mut options = ScanOptions::default();
options.standard_skips = StandardSkips::Disabled;

Ignore semantics

Ignore files are loaded hierarchically. Later matching rules win. Supported Git-style constructs include:

• comments and escaped leading # / !;
• negation with !;
• root-anchored patterns;
• directory-only patterns;
• *, **, and ?;
• character classes, negated classes, and ranges;
• escaped literals and escaped trailing spaces.

The scanner intentionally does not read global Git configuration or .git/info/exclude. Repository-local selection therefore stays portable across machines. Differential tests compare exact selected path sets against the ignore crate for anchored, nested, negated, wildcard, and character-class fixtures.

Safety model

• never executes repository code;
• never starts subprocesses or accesses the network;
• canonicalizes and validates the root before traversal;
• does not follow symlink entries;
• rejects paths outside the canonical root;
• caps selected file size before content reads;
• forbids unsafe Rust.

The scanner is read-only. As with ordinary filesystem walkers, callers should avoid concurrently replacing directories while a scan is running.

Benchmarks

Run all included benchmarks:

cargo bench --locked

Run the competitor comparison:

cargo bench --locked --bench compare_competitors

Run exact selected-path parity on a real repository:

$env:WEAVATRIX_BENCH_ROOT = "C:\path\to\repository"
cargo bench --locked --bench real_repository

The synthetic comparison uses 6,000 source files across Rust, Go, and TypeScript. It runs two warmups and 11 interleaved measured samples, then reports the median. Comparable walkers must produce the same fully sorted manifest of normalized relative paths and byte sizes, not merely the same count.

Sample result on Windows 11, Rust 1.97.1, warm filesystem cache:

This is an output-equivalent Windows manifest benchmark. jwalk parallelizes directory reads very effectively, but its per-entry metadata path is expensive on this Windows corpus; it remains a strong choice for path-only traversal and this table must not be used to claim otherwise. The rich-manifest row has no direct equivalent in the walkers: it also reads content, detects binaries, hashes sources, records typed evidence, and computes a deterministic revision.

Source review explains the remaining differences:

• walkdir streams unsorted directory entries and bounds open descriptors;
• jwalk schedules read_dir work through Rayon and restores ordered output;
• ignore compiles patterns into GlobSet matchers and shares inherited matchers;
• Weavatrix Scan now streams the no-ignore fast path, reuses inherited rules, indexes exact literals, specializes prefix/suffix globs, prefilters complex patterns, and sorts only the final report.

Exact-path real-repository sample:

Timing varies by filesystem, cache, antivirus, and CPU. Treat the table as a reproducible sample, not a universal constant.

Correctness checks

The test suite covers:

• deterministic results and revisions;
• ignore-rule precedence and nested ignore files;
• parity with ignore on representative Git-style patterns;
• binary, oversized, extension, generated-directory, and symlink policies;
• serial/parallel content-inspection equivalence;
• optional Serde support.

The real-repository benchmark compares the complete normalized selected-path set against ignore. Its comparison policy disables Weavatrix's file-size cap so an oversized file cannot masquerade as an ignore-rule mismatch.

Development

cargo fmt --all -- --check
cargo test --locked --all-features
cargo clippy --locked --all-targets --all-features -- -D warnings
cargo doc --locked --no-deps --all-features
cargo bench --locked
cargo publish --locked --dry-run

The MSRV is Rust 1.88. CI checks stable Rust plus Rust 1.88 on Linux and Windows.

Relationship to Weavatrix

weavatrix-scan owns repository discovery. It does not parse languages or build graphs. weavatrix-graph owns typed graph primitives. Higher-level Weavatrix crates can compose both without coupling either library to MCP, a CLI, or language-specific parsers.

License

MIT © 2026 Sergii Ziborov.