Crate scribe_scanner 

Scribe Scanner

High-performance file system scanning and indexing capabilities for the Scribe library. This crate provides efficient tools for discovering, filtering, and analyzing files in large codebases with git integration and parallel processing.

Features

• Fast Repository Traversal: Efficient file discovery using walkdir and ignore
• Git Integration: Prefer git ls-files when available, with fallback to filesystem walk
• Language Detection: Automatic detection for 25+ programming languages
• Content Analysis: Extract imports, documentation structure, and metadata
• Parallel Processing: Memory-efficient parallel file processing using Rayon
• Binary Detection: Smart binary file detection using content analysis

Usage

use scribe_scanner::{Scanner, ScanOptions};
use std::path::Path;

let scanner = Scanner::new();
let options = ScanOptions::default()
    .with_git_integration(true)
    .with_parallel_processing(true);

let results = scanner.scan(Path::new("."), options).await?;
println!("Scanned {} files", results.len());

Re-exports

pub use scanner::Scanner;

pub use scanner::ScanOptions;

pub use scanner::ScanResult;

pub use scanner::ScanProgress;

pub use metadata::FileMetadata;

pub use metadata::MetadataExtractor;

pub use metadata::SizeStats;

pub use content::ContentAnalyzer;

pub use content::ImportInfo;

pub use content::DocumentationInfo;

pub use content::ContentStats;

pub use git_integration::GitIntegrator;

pub use git_integration::GitFileInfo;

pub use git_integration::GitCommitInfo;

pub use language_detection::LanguageDetector;

pub use language_detection::LanguageHints;

pub use language_detection::DetectionStrategy;

Modules

content

Content analysis for extracting imports, documentation structure, and code metrics.

git_integration

Git integration for enhanced file discovery and status tracking.

language_detection

Advanced programming language detection for 25+ languages.

metadata

File metadata extraction and analysis.

scanner

Core scanning functionality for efficient file system traversal.

Structs

FileScanner

High-level scanner facade providing convenient access to all scanning functionality

ScannerStats

Statistics about the scanning process

Constants

VERSION

Current version of the scanner crate