# RoboSync 🚀
**Lightning-fast file synchronization that just works.**
[](https://opensource.org/licenses/MIT)
[](https://www.rust-lang.org)
[](#platform-support)
RoboSync combines the battle-tested reliability of RoboCopy and rsync with modern Rust performance. Sync terabytes with confidence using smart strategies that adapt to your workload.
## 🎯 Why RoboSync?
- **🏎️ Blazing Fast**: Multi-threaded parallel processing that saturates your storage bandwidth
- **🧠 Smart Defaults**: Automatically picks the best strategy for your files - no PhD required
- **🔒 Rock Solid**: Zero panics, comprehensive error handling, and automatic error reports
- **🌍 Cross-Platform**: One tool that works everywhere - Linux, macOS, and Windows
- **📊 Real Feedback**: Know exactly what's happening with progress bars, ETAs, and detailed logging
## ✨ Features
### ⚡ Performance That Scales
- **Concurrent Mixed Processing** - Different strategies for different file sizes, all running in parallel:
- Small files (< 1MB): Lightning-fast parallel copies
- Medium files (1-100MB): Platform-optimized APIs
- Large files (> 100MB): Delta transfer with 64KB blocks
- **Delta-Transfer Algorithm** - Only copy what changed in large files
- **Smart Compression** - Zstandard and LZ4 for optimal network transfers
- **Platform Optimizations** - io_uring on Linux, native APIs everywhere
### 🛡️ Enterprise-Ready Reliability
- **Automatic Error Reports** - Never lose track of what failed
- **Retry Logic** - Configurable retries with exponential backoff
- **Metadata Preservation** - Timestamps, permissions, ownership, all preserved
- **Symlink Support** - Handle links your way: copy, follow, or skip
- **Network Support** - Works with mounted network drives (SMB/NFS/SSHFS)
### 🎮 Developer-Friendly Interface
- **RoboCopy Compatible** - Your muscle memory still works
- **Multi-Level Verbosity** - From silent to full debug output
- **Dry Run Mode** - See what would happen before it does
- **Interactive Confirmation** - Double-check before big operations
## 🚀 Quick Start
### Prerequisites
- **Rust 1.70+** (for building from source or cargo install)
- **Zstandard library** (libzstd) - for compression support
### Installation
#### From Package Managers
```bash
# Arch Linux (AUR) - using yay
yay -S robosync
# Arch Linux (AUR) - using paru
paru -S robosync
# Using Rust's Cargo
cargo install robosync
# Update to latest version
cargo install --force robosync
```
#### Build from Source
```bash
# Clone and build
git clone https://github.com/roethlar/robosync.git
cd robosync
cargo build --release
# Copy to your PATH
sudo cp target/release/robosync /usr/local/bin/
```
### Your First Sync
```bash
# Simple copy
robosync /source /destination -e
# Mirror with confirmation
robosync /source /dest --mir --confirm
# Sync to network mount with compression
robosync /local/path /mnt/network/path -e -z
# See what would happen
robosync /source /dest -e -n
```
## 📚 Common Use Cases
### 🗄️ Backup Your Home Directory
```bash
robosync ~/ /backup/home/ -e \
--xd ".cache" --xd ".local/share/Trash" \
--xf "*.tmp" --xf ".DS_Store" \
-v --log backup.log
```
### 🔄 Keep Servers in Sync
```bash
robosync /var/www/ /mnt/server/var/www/ --mir \
--compress \
--retry 3 --wait 5 \
--mt 16
```
### 📸 Organize Photos by Date
```bash
robosync /camera/DCIM/ /photos/2024/ -e \
--min 1048576 \ # Skip files < 1MB
--copy DATSOU \ # Preserve all metadata
--no-report-errors # Photos are already backed up
```
### 🎮 Sync Game Saves
```bash
robosync "C:\Users\Gamer\SavedGames" "D:\Backup\Saves" \
--mir \
--xf "*.log" \
--confirm
```
## 🎛️ Command Reference
### Essential Options
| `-e` | Copy all subdirectories (even empty ones) |
| `-n` | Dry run - preview without changes |
| `-v` | Verbose - show what's happening |
| `-z` | Compress during transfer |
| `--mir` | Mirror mode - make dest exactly like source |
### File Selection
| `--xf` | Exclude files | `--xf "*.tmp" --xf "*.cache"` |
| `--xd` | Exclude directories | `--xd "node_modules" --xd ".git"` |
| `--min` | Minimum file size | `--min 1024` (skip < 1KB) |
| `--max` | Maximum file size | `--max 1073741824` (skip > 1GB) |
### Performance Tuning
| `--mt` | Thread count | CPU cores |
| `-b` | Block size for delta | 1024 bytes |
| `--strategy` | Force specific strategy | `mixed` |
### File Size Categories (shown in -v mode)
| Small | < 256KB | Parallel batch processing | Green |
| Medium | 256KB - 10MB | Platform-optimized APIs | Yellow |
| Large | 10MB - 100MB | Standard copy | Red |
| Delta | > 100MB | Delta transfer algorithm | Cyan |
### Safety & Control
| `--confirm` | Ask before starting |
| `--no-report-errors` | Don't create error report |
| `-r` | Retry count on failures |
| `-w` | Wait seconds between retries |
## 📊 Performance Benchmarks
Real-world performance on commodity hardware:
| Small files (< 1MB) | 100,000 | 12 GB | 45s | 267 MB/s |
| Large files (> 100MB) | 50 | 200 GB | 62s | 3.2 GB/s |
| Mixed workload | 10,000 | 50 GB | 35s | 1.4 GB/s |
| Delta update (10% changed) | 1 | 100 GB | 18s | 556 MB/s |
*Tested on NVMe SSD with 16 threads*
## 🔧 Advanced Usage
### Force Specific Strategies
```bash
# Best for many small files
robosync /source /dest --strategy parallel
# Best for large file updates
robosync /source /dest --strategy delta
# Use native tools (rsync/robocopy)
robosync /source /dest --strategy rsync
```
### Platform-Specific Optimizations
```bash
# Linux: Enable io_uring for maximum performance
robosync /source /dest --strategy io_uring
# Linux: Optimize for many small files
robosync /source /dest --linux-optimized
# Windows: Use native robocopy
robosync C:\source D:\dest --strategy robocopy
```
### Error Handling
RoboSync's error handling adapts to your verbosity level:
```bash
# Default: Errors only in report file
robosync /source /dest -e
# -v: Errors on console + report file
robosync /source /dest -e -v
# -vv: Everything on console + report
robosync /source /dest -e -vv
# Disable error reports
robosync /source /dest -e --no-report-errors
```
## 📝 Important Notes
### Network Transfers
RoboSync operates on local and mounted filesystems including network mounts like NFS, SMB/CIFS, and SSHFS. Mount your network drives first, then use RoboSync for blazing fast synchronized transfers.
## 🏗️ Architecture
RoboSync uses a sophisticated strategy selection system:
```
┌─────────────────┐
│ File Analysis │ ← Scan source and destination
└────────┬────────┘
│
┌────────▼────────┐
│ Strategy Select │ ← Choose based on file sizes
└────────┬────────┘
│
┌────────▼────────┐
│ Mixed Executor │ ← Run multiple strategies concurrently
└────────┬────────┘
│
┌────┴────┬─────────┬──────────┐
▼ ▼ ▼ ▼
[Parallel] [Platform] [Delta] [Native]
```
## 🌍 Platform Support
### Linux 🐧
- **io_uring** support for bleeding-edge performance
- Adaptive thread limits based on system resources
- Native rsync integration when beneficial
### macOS 🍎
- Optimized for APFS and HFS+
- Conservative threading for system stability
- Full metadata preservation including extended attributes
### Windows 🪟
- Native Win32 APIs for maximum compatibility
- RoboCopy fallback for complex scenarios
- Full NTFS metadata support
## 🤝 Contributing
We love contributions! Whether it's:
- 🐛 Bug reports
- 💡 Feature requests
- 📖 Documentation improvements
- 🚀 Performance optimizations
- 🧪 Test coverage
Check out our [Contributing Guide](CONTRIBUTING.md) to get started.
## 📜 License
MIT License - see [LICENSE](LICENSE) for details.
## 🙏 Acknowledgments
Standing on the shoulders of giants:
- **rsync** - The grandfather of smart syncing
- **RoboCopy** - Windows' robust file copier
- **Rust Community** - For amazing crates like tokio, rayon, and blake3
- **You** - For choosing RoboSync!
---
**Ready to sync at the speed of light?** 🚀
See the [Installation](#installation) section above to get started!