Crate titor

Titor - Time-traveling directory snapshots

A high-performance checkpointing library that enables time-travel through directory states with efficient storage and fast operations.

Overview

Titor provides a Git-like versioning system for entire directory trees, allowing you to:

• Create immutable snapshots (checkpoints) of directory states
• Restore directories to any previous checkpoint
• Fork from existing checkpoints to create alternate timelines
• Efficiently store only changed files using content-addressable storage
• Verify checkpoint integrity using Merkle trees
• Navigate through checkpoint history with a timeline interface
• Compare checkpoints with line-level differences like git diff

Architecture

Titor uses several key components to achieve high performance:

• Content-Addressable Storage: Files are stored by their content hash, enabling automatic deduplication across checkpoints
• Merkle Trees: Each checkpoint includes a Merkle tree for cryptographic verification of file integrity
• Compression: Multiple compression strategies (Fast, Balanced, Best) to optimize storage vs performance tradeoffs
• Parallel Processing: File scanning and storage operations use parallel processing for maximum throughput
• Reference Counting: Efficient garbage collection tracks which objects are still referenced by active checkpoints

Quick Start

use titor::{Titor, TitorBuilder, CompressionStrategy};
use std::path::PathBuf;

// Initialize a new Titor instance
let mut titor = Titor::init(
    PathBuf::from("./my_project"),    // Directory to track
    PathBuf::from("./.titor_storage") // Where to store checkpoints
)?;

// Create a checkpoint
let checkpoint = titor.checkpoint(Some("Initial state".to_string()))?;
println!("Created checkpoint: {}", checkpoint.id);

// Make some changes to your files...

// Create another checkpoint
let checkpoint2 = titor.checkpoint(Some("Added new feature".to_string()))?;

// Restore to the first checkpoint
let result = titor.restore(&checkpoint.id)?;
println!("Restored {} files", result.files_restored);

Advanced Usage

Using TitorBuilder for Custom Configuration

use titor::{TitorBuilder, CompressionStrategy};
use std::path::PathBuf;

let titor = TitorBuilder::new()
    .compression_strategy(CompressionStrategy::Fast)
    .ignore_patterns(vec![
        "*.tmp".to_string(),
        "node_modules/**".to_string(),
        ".git/**".to_string()
    ])
    .max_file_size(100 * 1024 * 1024) // Skip files larger than 100MB
    .parallel_workers(8)
    .follow_symlinks(false)
    .build(
        PathBuf::from("./project"),
        PathBuf::from("./storage")
    )?;

Forking Timelines

// Fork from an existing checkpoint to create an alternate timeline
let fork = titor.fork("checkpoint_id", Some("Experimental branch".to_string()))?;

Checkpoint Verification

// Verify integrity of a specific checkpoint
let report = titor.verify_checkpoint("checkpoint_id")?;
if report.is_valid() {
    println!("Checkpoint is valid!");
} else {
    let invalid_files = report.file_checks.iter().filter(|f| !f.is_valid()).count();
    println!("Found {} invalid files", invalid_files);
}

// Verify entire timeline
let timeline_report = titor.verify_timeline()?;
println!("Timeline verification: {:?}", timeline_report);

Line-Level Diffs

// Get detailed diff with line-level changes
let options = DiffOptions {
    context_lines: 3,
    ignore_whitespace: false,
    show_line_numbers: true,
    max_file_size: 10 * 1024 * 1024, // 10MB
};

let detailed_diff = titor.diff_detailed("checkpoint1", "checkpoint2", options)?;

println!("Files changed: {}, +{} -{} lines",
    detailed_diff.basic_diff.stats.total_operations(),
    detailed_diff.total_lines_added,
    detailed_diff.total_lines_deleted
);

// Print unified diff for each file
for file_diff in &detailed_diff.file_diffs {
    if !file_diff.is_binary {
        println!("\n--- a/{}", file_diff.path.display());
        println!("+++ b/{}", file_diff.path.display());
         
        for hunk in &file_diff.hunks {
            println!("@@ -{},{} +{},{} @@",
                hunk.from_line, hunk.from_count,
                hunk.to_line, hunk.to_count
            );
        }
    }
}

Key Concepts

Checkpoints

A checkpoint is an immutable snapshot of a directory at a specific point in time. Each checkpoint contains:

• Unique identifier (UUID)
• Parent checkpoint reference (for timeline tracking)
• Timestamp and description
• Merkle root hash for verification
• Metadata about files and changes

Timeline

The timeline tracks the relationship between checkpoints, forming a directed acyclic graph (DAG). This allows for:

• Linear history tracking
• Branching and forking
• Finding common ancestors
• Navigating checkpoint relationships

Content-Addressable Storage

Files are stored using their SHA-256 hash as the key. This provides:

• Automatic deduplication
• Integrity verification
• Efficient storage of similar files
• Immutable object store

Merkle Trees

Each checkpoint builds a Merkle tree from all tracked files, enabling:

• Efficient verification of checkpoint integrity
• Cryptographic proof of file inclusion
• Fast comparison between checkpoints

Performance Considerations

• Parallel Scanning: Directory scanning uses parallel workers for large directories
• Compression: Choose compression strategy based on your needs:
  • Fast: Best for frequently changing files
  • Balanced: Good general-purpose compression
  • Best: Maximum compression for archival
• Memory Usage: Large directories may require significant memory during scanning
• Storage Growth: Use gc() periodically to clean up unreferenced objects

Security Considerations

• Checkpoints are immutable once created
• File integrity is verified using SHA-256 hashes
• Merkle trees provide cryptographic verification
• Symlinks are stored as references (be cautious with external targets)
• File permissions are preserved but ownership is not

Error Handling

All operations return Result<T, TitorError> where TitorError provides detailed error information including:

• Error category (Storage, Checkpoint, Timeline, etc.)
• Descriptive message
• Optional source error for debugging

Module Organization

• checkpoint: Checkpoint creation and metadata management
• compression: Compression strategies and engine
• timeline: Timeline structure and navigation
• storage: Content-addressable storage backend (internal module)
• verification: Integrity checking and verification
• types: Common types and data structures
• error: Error types and handling
• diff: File content comparison and diffing

Re-exports

pub use checkpoint::Checkpoint;

pub use checkpoint::CheckpointMetadata;

pub use compression::CompressionEngine;

pub use compression::CompressionStrategy;

pub use error::Result;

pub use error::TitorError;

pub use storage::Storage;

pub use titor::Titor;

pub use titor::TitorBuilder;

pub use timeline::Timeline;

pub use verification::CheckpointVerifier;

pub use verification::TimelineVerificationReport;

pub use verification::VerificationReport;

pub use types::*;

Modules

checkpoint

Checkpoint definitions and operations

compression

High-performance LZ4 compression engine for Titor

diff

Line-level diff computation for text files

error

Error types for the Titor library

storage

Content-addressable storage implementation for Titor

timeline

Timeline management for checkpoint navigation

titor

Main Titor implementation

types

Core data types used throughout the Titor library

verification

Checkpoint verification and integrity checking