# Forge - Production-Ready File Watcher Library# Forge - Ultra-Fast CRDT Version Control
Ultra-fast file watcher library with dual-mode event system optimized for DX tools.Operation-level version control powered by CRDTs. Forge tracks file edits as fine-grained operations, persists them in a local DeltaDB, and keeps peers in sync over WebSockets.
## ✨ Features## ⚡⚡ Dual-Watcher Architecture
- **⚡ Rapid Events**: <35µs ultra-fast change notifications (typically 1-2µs)Forge uses a **dual-watcher system** for maximum performance and quality:
- **📊 Quality Events**: <60µs full operation detection with line numbers and diffs
- **🚀 Production Ready**: Zero environment variables, optimal hardcoded settings### 🚀 Mode 1: RAPID Detection (<20µs)
- **🔧 CRDT-based**: Conflict-free replicated data types for distributed sync
- **💾 Memory-mapped I/O**: Leverages OS page cache for sub-microsecond reads- **Zero syscalls** - Uses atomic sequence counter (no time calls!)
- **🎯 DX-focused**: Built specifically as a base for developer experience tools- **No file operations** - Skips metadata, mtime, and content reads
- **Instant feedback** - Ultra-fast change logging
## 🚀 Quick Start- **Target: <20µs** - Ultra-fast notification system
### As a Library Dependency### 📊 Mode 2: QUALITY Detection (<60µs)
Add to your `Cargo.toml`:- **Full file analysis** - Complete operation detection with line numbers
- **Rich metadata** - Diffs, timestamps, and sync details
```toml- **Background execution** - Runs after rapid mode
[dependencies]- **Target: <60µs** - Fast detailed analysis
forge = "1.0"
tokio = { version = "1.48", features = ["full"] }Both modes run sequentially for every file change, providing instant feedback (rapid) followed by complete details (quality).
```
## 🎯 Performance Targets
### Basic Usage
- **RAPID mode**: <20µs change detection ✅ **ACHIEVED: 3-20µs**
```rust- **QUALITY mode**: <100µs operation detection ⚠️ **CURRENT: ~60-300µs**
use forge::{ForgeWatcher, ForgeEvent};- **Total latency**: <320µs for complete processing
- **Debounce**: 1ms ultra-fast mode
#[tokio::main]- **Inspired by**: dx-style project's <100µs techniques
async fn main() -> anyhow::Result<()> {
// Create watcher for current directory### Current Performance
let watcher = ForgeWatcher::new(".", false, vec![]).await?;
```
```bash
// Run the watcher# Small appends (cached, best case)
watcher.run().await?;⚡ [RAPID 3µs] test.txt changed
✨ [QUALITY 58µs | total 61µs] test.txt - 1 ops
Ok(())
}# Regular edits (typical case)
```⚡ [RAPID 20µs] test.txt changed
🐢 [QUALITY 301µs | total 321µs] test.txt - 1 ops
### Running Examples```
```bash**RAPID mode**: ✅ Target exceeded (3µs is 6x faster than 20µs goal!)
## Simple watcher example**QUALITY mode**: ⚠️ 58-301µs (varies by edit type - appends are fast, full diffs slower)
cargo run --release --example simple
## Quick Start
## Full CLI with all features
cargo run --release --bin forge```bash
```# Default mode (dual-watcher enabled)
cargo run --release
## 🎯 Dual-Event System
# Enable profiling to see timings
Forge emits **two types of events** for every file change:DX_WATCH_PROFILE=1 cargo run --release
### 1. ⚡ Rapid Event (<35µs)# Disable rapid mode (quality only, for testing)
DX_DISABLE_RAPID_MODE=1 cargo run --release
Ultra-fast notification using zero syscalls:
- **Timing**: Typically 1-2µs, max 35µs# Example output:
- **Purpose**: Instant UI feedback for formatters/linters# ⚡ [RAPID 8µs] README.md changed
- **Method**: Atomic sequence counter (no file I/O)# ✨ [QUALITY 52µs | total 60µs] README.md - 1 ops
- **Data**: File path + timing only```
### 2. ✨ Quality Event (<60µs) ## Configuration
Complete operation detection with details:### Environment Variables
- **Timing**: Typically <60µs
- **Purpose**: Full analysis for quality tools- `DX_WATCH_PROFILE=1` - Show detailed timing for both modes
- **Method**: Memory-mapped I/O + SIMD diffs- `DX_DISABLE_RAPID_MODE=1` - Disable rapid mode (quality only)
- **Data**: Operations, line numbers, content changes- `DX_DEBOUNCE_MS=1` - Debounce interval (default: 1ms)
## 📊 Performance Benchmarks### Performance Markers
```- ⚡ RAPID mode ≤20µs (target achieved)
Rapid Mode (Change Detection):- 🐌 RAPID mode >20µs (needs optimization)
⚡ Best case: 1-2µs (cached, atomic only)- ✨ QUALITY mode ≤60µs (target achieved)
⚡ Typical: 8-20µs (95th percentile)- 🐢 QUALITY mode >60µs (needs optimization)
🎯 Target: <35µs ✅ ACHIEVED
**Clean output - only shows when there are changes!**
Quality Mode (Full Analysis):
✨ Best case: 58µs (simple append)Testing no-op detection...
✨ Typical: 60µs (typical edits)
🐢 Worst case: 301µs (complex diffs)
🎯 Target: <60µs ⚠️ MOSTLY ACHIEVED
```
### Example Output
```text
⚡ [RAPID 8µs] test.txt changed
✨ [QUALITY 52µs | total 60µs]
- test.txt @ 1:1
Hello, Forge!
```
## 🏗️ Architecture
### Core Components
- **Watcher**: File system monitoring with 1ms debounce
- **CRDT**: Automerge + Yrs for conflict-free merging
- **Storage**: SQLite + LZ4 compression for operation log
- **Sync**: WebSocket-based real-time synchronization
- **Cache**: Memory-mapped file pool + OS page cache warming
### Event Flow
```text
File Change
↓
[1ms Debounce]
↓
⚡ Rapid Detection (1-2µs)
↓ atomic counter
↓ emit rapid event
↓
✨ Quality Detection (60µs)
↓ mmap read
↓ SIMD diff
↓ emit quality event
```
## 🔧 Configuration
### Production Settings (Hardcoded)
```rust
const RAPID_MODE_ENABLED: bool = true; // Always on
const DEBOUNCE_MS: u64 = 1; // 1ms optimal
const RAPID_TARGET_US: u128 = 35; // <35µs rapid
const QUALITY_TARGET_US: u128 = 60; // <60µs quality
```
No environment variables needed - everything is optimized out of the box!
## 📦 Use Cases
Perfect for building DX tools that need:
1. **Formatters**: Rapid events trigger instant formatting
2. **Linters**: Quality events provide detailed analysis
3. **Hot Reload**: Rapid events for instant dev server refresh
4. **Build Tools**: Quality events for incremental compilation
5. **Test Runners**: Rapid events trigger test re-runs
## 🎨 Example: Custom Event Handler
```rust
use forge::{ForgeWatcher, Operation, OperationType};
## [tokio::main]
async fn main() -> anyhow::Result<()> {
let watcher = ForgeWatcher::new("./src", false, vec![]).await?;
// The watcher automatically handles events internally
// and prints detailed logs for rapid + quality events
watcher.run().await?;
Ok(())
}
```
## 🚀 Advanced Features
### CRDT Synchronization
```rust
use forge::ForgeWatcher;
## [tokio::main]
async fn main() -> anyhow::Result<()> {
// Enable sync with remote peer
let watcher = ForgeWatcher::new(
".",
true, // enable_sync
vec!["ws://localhost:3000/ws".to_string()]
).await?;
watcher.run().await?;
Ok(())
}
```
### Operation Log Access
```rust
use forge::{Database, OperationLog};
use std::sync::Arc;
let db = Database::new(".dx/forge")?;
let oplog = Arc::new(OperationLog::new(Arc::new(db)));
// Query operations for a file
// oplog.get_file_operations("src/main.rs")?;
```
## 🎯 Performance Optimization Techniques
1. **Zero-copy I/O**: Memory-mapped files via `memmap2`
2. **SIMD acceleration**: Fast byte comparison via `memchr`
3. **Atomic operations**: Lock-free sequence counters
4. **OS cache warming**: Pre-load all files at startup
5. **Lazy allocation**: Defer expensive operations until needed
6. **Parallel processing**: `rayon` for large file diffs
## 📝 License
MIT OR Apache-2.0
## 🙏 Credits
Performance techniques inspired by:
- [dx-style](https://github.com/dx-style/dx-style) - Sub-100µs code generation
- [notify](https://github.com/notify-rs/notify) - File system events
- [automerge](https://github.com/automerge/automerge) - CRDT implementation
## 🔗 Links
- **Repository**: <https://github.com/najmus-sakib-hossain/version-control>
- **Documentation**: <https://docs.rs/forge>
- **Crates.io**: <https://crates.io/crates/forge>
---
**Built with ❤️ for the DX community**