# fs_interface
Provides a filesystem-like API over Heroforge repositories with staging directory support.
## Overview
The `fs` module provides two filesystem interfaces:
1. **`FsInterface`** (recommended) - New staging-based interface with background commits
2. **`FileSystem`** (legacy) - Direct database operations with immediate commits
## Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ FsInterface (sync API) │
│ - RwLock<StagingState> │
│ - Author name (set at initialization) │
│ - All read/write operations acquire appropriate locks │
└─────────────────────────────────────────────────────────────────┘
│
┌───────────────────┴───────────────────┐
│ │
▼ ▼
┌─────────────────────┐ ┌─────────────────────────┐
│ Staging Directory │ │ Commit Timer │
│ │ │ (background) │
│ - All writes go │ │ │
│ here first │ │ - Runs every 1 minute │
│ - Files < 2MB │ │ - Signals commit due │
│ - Frequent updates │ │ - Actual commit is │
│ allowed │ │ synchronous │
└─────────────────────┘ └─────────────────────────┘
│ │
└───────────────────┬───────────────────┘
│
▼
┌─────────────────────┐
│ .forge Database │
│ (SQLite) │
└─────────────────────┘
```
## Key Types
### FsInterface
Main entry point for staging-based filesystem operations.
```rust
use heroforge::Repository;
use heroforge::fs::FsInterface;
use std::sync::Arc;
let repo = Arc::new(Repository::open_rw("project.forge")?);
let fs = FsInterface::new(repo, "developer@example.com")?;
// Writes go to staging (fast)
fs.write_file("config.json", b"{}")?;
// Reads check staging first, then database
let content = fs.read_file("config.json")?;
// Partial updates use read-modify-write pattern
fs.write_at("data.bin", 100, b"updated")?;
// Force immediate commit (normally auto-commits every 1 minute)
fs.commit()?;
```
### StagingState
Tracks uncommitted changes in the staging directory.
- Holds metadata about staged files
- Manages file promotion from database to staging
- Tracks deletions as markers
### CommitTimer
Background timer that signals when commits are due.
- Runs in a separate thread
- Checks every 100ms if interval has elapsed
- Signals but doesn't perform commits (due to SQLite threading)
## Module Structure
```
src/fs/
├── mod.rs # Module exports and documentation
├── README.md # This file
├── errors.rs # FsError and FsResult types
├── staging.rs # StagingState and Staging wrapper
├── commit_thread.rs # CommitTimer and commit helpers
├── fs_interface.rs # Main FsInterface implementation
├── interface.rs # Legacy FileSystem implementation
├── operations.rs # FileMetadata, FilePermissions, etc.
├── transaction.rs # Transaction support
├── find.rs # Find files with glob patterns
└── ops.rs # High-level ops (copy, move, delete, chmod, symlinks)
```
## Dependencies
- `core` (via `crate::error`): Error types
- `repo`: Repository and commit operations
- Standard library: `std::sync::RwLock`, `std::thread`, `std::fs`
## Thread Safety
Uses `RwLock<StagingState>` for concurrent access:
| `exists()` | Read | Nothing |
| `read_file()` | Read | Nothing |
| `list_dir()` | Read | Nothing |
| `stat()` | Read | Nothing |
| `write_file()` | Write | Other writes |
| `delete_file()` | Write | Other writes |
| `commit()` | Write | Other writes |
## Read Path (Layered Lookup)
When reading a file:
1. **Staging Directory** → If file exists here, return it (most recent)
2. **`.forge` Database** → Query SQLite for committed version
3. **Return Error** → File does not exist
## Write Path
All writes follow this flow:
1. Validate path and content size (< 2MB)
2. Acquire write lock (RwLock)
3. Write file to staging directory
4. Update staging state metadata
5. Release lock
6. Return success (file is NOT yet in database)
## Partial File Updates (Read-Modify-Write)
When modifying part of a file:
**File NOT in staging:**
1. Read full file from database
2. Copy entire file to staging (promotion)
3. Apply modification at offset
4. Mark file as modified
**File already in staging:**
1. Seek to offset in staging file
2. Write data directly
3. Update size if file grew
## Auto-Commit Behavior
- Timer checks every 100ms if 1-minute interval elapsed
- When due, next operation will trigger commit
- Forced commits happen on:
- Branch switch
- `commit()` call
- Interface drop (cleanup)
## File Size Limits
**Maximum: 2 MB per file**
```rust
// This will return FsError::FileTooLarge
fs.write_file("large_video.mp4", &large_content)?;
```
## Error Types
```rust
pub enum FsError {
NotFound(String),
NotAFile(String),
NotADirectory(String),
InvalidPath(String),
FileTooLarge { path: String, size: u64, max: u64 },
LockTimeout,
CommitInProgress,
DatabaseError(String),
StagingError(String),
// ... and more
}
```
## Example: Full Workflow
```rust
use heroforge::Repository;
use heroforge::fs::FsInterface;
use std::sync::Arc;
fn main() -> heroforge::Result<()> {
// 1. Open repository
let repo = Arc::new(Repository::open_rw("project.forge")?);
// 2. Create FsInterface with author
let fs = FsInterface::new(repo, "developer@example.com")?;
// 3. Write files (goes to staging)
fs.write_file("README.md", b"# Project")?;
fs.write_file("src/main.rs", b"fn main() {}")?;
// 4. Read files (checks staging first)
let readme = fs.read_file_string("README.md")?;
// 5. Partial update
fs.write_at("README.md", 2, b"My ")?; // "# My Project"
// 6. Copy/move operations
fs.copy_file("README.md", "docs/README.md")?;
fs.move_file("old.txt", "archive/old.txt")?;
// 7. Delete operations
fs.delete_file("temp.log")?;
fs.delete_dir("cache")?;
// 8. Find files
let rust_files = fs.find("**/*.rs")?;
println!("Found {} Rust files", rust_files.count);
// 9. Force commit (or wait for auto-commit)
let hash = fs.commit()?;
println!("Committed: {}", hash);
// 10. Switch branch (auto-commits first)
// fs.switch_branch("feature")?;
Ok(())
}
```
## Find Files
Use `Find` to search for files with glob patterns:
```rust
use heroforge::Repository;
use heroforge::fs::Find;
let repo = Repository::open("project.forge")?;
// Find all Rust files
let rust_files = Find::new(&repo)
.pattern("**/*.rs")
.paths()?;
// Find with filters
let filtered = Find::new(&repo)
.pattern("**/*.rs")
.ignore("target/**")
.ignore_hidden()
.max_depth(5)
.in_dir("src")
.paths()?;
// Convenience functions
use heroforge::fs::{find, count, exists, is_dir, stat, du};
let files = find(&repo, "**/*.rs")?;
let num = count(&repo, "**/*.md")?;
let has_readme = exists(&repo, "README.md")?;
let is_src_dir = is_dir(&repo, "src")?;
let info = stat(&repo, "Cargo.toml")?;
let total_size = du(&repo, "**/*")?;
```
## Modify Files (Copy, Move, Delete, Chmod)
Use `Modify` for batch file operations:
```rust
use heroforge::Repository;
use heroforge::fs::Modify;
let repo = Repository::open_rw("project.forge")?;
// Batch operations with atomic commit
Modify::new(&repo)
.message("Reorganize project")
.author("developer")
.copy_file("README.md", "docs/README.md")
.copy_dir("src", "src_backup")
.move_file("old.rs", "archive/old.rs")
.move_dir("scripts", "tools")
.delete_file("temp.log")
.delete_dir("cache")
.delete_matching("**/*.bak")
.chmod("script.sh", 0o755)
.make_executable("bin/run")
.symlink("latest", "releases/v1.0.0")
.write_str("VERSION", "1.0.0")
.touch(".gitkeep")
.execute()?;
// Preview without committing
let preview = Modify::new(&repo)
.copy_file("a.txt", "b.txt")
.preview()?;
for desc in preview.describe() {
println!("{}", desc);
}
```
## Upload/Download (OS Filesystem <-> .forge Database)
Transfer files between your local filesystem and the repository:
### Upload (OS -> Repository)
```rust
use heroforge::Repository;
use heroforge::fs::{upload, upload_dir};
let repo = Repository::open_rw("project.forge")?;
// Upload a single file
upload(&repo, "/home/user/config.json", "config.json", "developer", None)?;
// Upload with custom message
upload(&repo, "/tmp/data.bin", "data/file.bin", "admin", Some("Import data file"))?;
// Upload entire directory
upload_dir(&repo, "/home/user/project/src", "src", "developer", None)?;
```
### Download (Repository -> OS)
```rust
use heroforge::Repository;
use heroforge::fs::{download, download_dir, download_matching};
let repo = Repository::open("project.forge")?;
// Download a single file
download(&repo, "config.json", "/home/user/config.json")?;
// Download entire directory
let count = download_dir(&repo, "src", "/tmp/src_export")?;
println!("Downloaded {} files", count);
// Download entire repository
download_dir(&repo, "", "/tmp/full_export")?;
// Download files matching a pattern
let count = download_matching(&repo, "**/*.rs", "/tmp/rust_files")?;
println!("Downloaded {} Rust files", count);
// Download from specific branch
use heroforge::fs::download_from_branch;
download_from_branch(&repo, "feature", "config.json", "/tmp/config.json")?;
```
## See Also
- `docs/FS_IMPLEMENTATION_SUMMARY.md` - Full design specification
- `src/fs/mod.rs` - Module-level documentation
- `examples/fs_operations.rs` - Working examples