# ADR-001: Regex Pre-compilation for Performance
**Status**: Accepted
**Date**: 2026-06-20
**Author**: codesearch team
## Context
The codebase uses regular expressions extensively for:
- Language parsing (Rust, Python, JavaScript, Go, Java)
- Security pattern scanning
- Code metrics calculation (class/function/field counting)
- Duplicate detection (comment stripping, normalization)
Profiling revealed that `regex::Regex::new()` was being called repeatedly in hot loops:
- `scan_security_patterns()` compiled 7 patterns per file scanned
- `count_classes()`/`count_functions()` compiled patterns per file analyzed
- `normalize_code()` compiled normalization patterns on every call
Regex compilation is expensive. Re-compiling the same patterns thousands of times during a codebase scan dominated execution time.
## Decision
Pre-compile all fixed regex patterns at program startup using `lazy_static` and store them in static variables. Use `regex::RegexSet` where multiple patterns are matched against the same text simultaneously.
### Implementation
1. **Security scanner** (`src/security.rs`): Compile 7 security patterns into a `regex::RegexSet` with parallel `SecurityKind`/`Severity` arrays.
2. **Code metrics** (`src/codemetrics/helpers.rs`): Store per-language pattern vectors in `lazy_static` `HashMap`s.
3. **Duplicate normalization** (`src/duplicates/normalize.rs`): Store comment-stripping and whitespace normalization patterns in `lazy_static` arrays.
## Consequences
### Positive
- **Eliminates regex compilation from hot loops**: Patterns compiled once at startup.
- **RegexSet matching is SIMD-accelerated**: Multiple patterns matched against a line in a single pass.
- **Simpler call sites**: No more `if let Ok(re) = Regex::new(...)` error handling in loops.
- **Thread-safe**: `lazy_static` handles synchronization; no per-thread duplication.
### Negative
- **Startup cost**: All patterns compiled at first access, adding ~1-2ms to cold-start.
- **Memory**: Compiled regexes resident for program lifetime (acceptable for CLI tool).
- **Reduced flexibility**: Dynamic patterns still require runtime compilation.
### Risks
- **Pattern validation errors become fatal**: `unwrap()` in `lazy_static` panics on invalid regex. Mitigated by testing all patterns in CI.
- **RegexSet doesn't support capture groups**: Security scanner only needed `is_match`, so no impact. Extractors that need captures still use individual `Regex` instances.
## Alternatives Considered
1. **Compile per-thread caches**: More complex, no benefit for single-threaded CLI usage.
2. **Use `once_cell` instead of `lazy_static`**: Would work, but `lazy_static` is already a dependency.
3. **Replace regex with hand-written parsers**: Too much effort for marginal gain; regex is sufficient for the current use cases.
## References
- `src/security.rs` — `SECURITY_PATTERNS` RegexSet
- `src/codemetrics/helpers.rs` — `CLASS_PATTERNS`, `FUNCTION_PATTERNS`, `FIELD_PATTERNS`
- `src/duplicates/normalize.rs` — `SINGLE_LINE_COMMENT_RE`, `MULTI_LINE_COMMENT_RE`, etc.