heroforge-core

Overview

heroforge-core is the core database library for Heroforge, providing a complete API for interacting with Fossil SCM repositories programmatically, without requiring the Fossil CLI to be installed. It supports both reading from existing repositories and creating new ones from scratch.

Features

Read Operations

• Open and read existing Fossil repositories
• Browse repository history and check-ins
• List and read files at any check-in
• Find files using glob patterns
• Navigate directory structures
• Access branch and tag information

Write Operations

• Create new repositories from scratch
• Commit files with full manifest generation
• Create and manage branches
• Add tags to check-ins
• Manage users and permissions

Filesystem Operations

• Copy, move, rename files and directories
• Delete files and directories
• Change permissions (chmod)
• Create symbolic links
• Advanced find with ignore patterns

Synchronization

• QUIC sync - Modern UDP-based protocol with TLS 1.3 (requires sync-quic feature)

Rhai Scripting

• Interactive REPL with tab completion and syntax highlighting
• Socket-based daemon for fast script execution
• Full API access from Rhai scripts

Installation

Add heroforge-core to your project:

cargo add heroforge-core

Or add to your Cargo.toml:

[dependencies]
heroforge-core = "0.2.2"

Optional Features

Enable QUIC sync for remote repository synchronization:

[dependencies]
heroforge-core = { version = "0.2.2", features = ["sync-quic"] }

CLI Usage

The heroforge-core binary provides both an interactive REPL and daemon mode for script execution.

Interactive REPL

# Start interactive shell (default)
heroforge-core

# Or explicitly with --ui flag
heroforge-core --ui

The REPL provides:

• Tab completion for all functions
• Syntax highlighting
• Command history
• Multi-line input support

Running Scripts

# Run a script file
heroforge-core myscript.rhai

# Execute inline script
heroforge-core -e 'let repo = repo_open("project.fossil"); print(branches_list(repo));'

# Read script from stdin
echo 'print(version())' | heroforge-core -i

# Run locally without daemon
heroforge-core --local -e 'print("Hello!")'

Daemon Mode

# Start daemon in background
heroforge-core start -bg

# Check daemon status
heroforge-core status

# Stop daemon
heroforge-core stop

REPL Commands

Quick Start

Reading from an Existing Repository

use heroforge_core::Repository;

fn main() -> heroforge_core::Result<()> {
    // Open a Fossil repository
    let repo = Repository::open("project.fossil")?;

    // Get the latest check-in on trunk
    let tip = repo.history().trunk_tip()?;
    println!("Latest: {} by {}", tip.hash, tip.user);
    println!("Comment: {}", tip.comment);

    // List all files on trunk
    let files = repo.files().on_trunk().list()?;
    for file in &files {
        println!("  {}", file.name);
    }

    // Read a specific file from trunk
    let content = repo.files().on_trunk().read("README.md")?;
    println!("README:\n{}", String::from_utf8_lossy(&content));

    Ok(())
}

Creating a New Repository

use heroforge_core::Repository;

fn main() -> heroforge_core::Result<()> {
    // Create a new repository
    let repo = Repository::init("new_project.fossil")?;

    // Create initial check-in
    let init_hash = repo.commit_builder()
        .message("initial empty check-in")
        .author("admin")
        .initial()
        .execute()?;

    // Add some files in a new commit
    let commit_hash = repo.commit_builder()
        .message("Initial project structure")
        .author("developer")
        .parent(&init_hash)
        .file("README.md", b"# My Project\n")
        .file("src/main.rs", b"fn main() { println!(\"Hello!\"); }\n")
        .execute()?;

    // Tag the release
    repo.tags()
        .create("v1.0.0")
        .at_commit(&commit_hash)
        .author("developer")
        .execute()?;

    // Create a feature branch
    repo.branches()
        .create("feature-x")
        .from_commit(&commit_hash)
        .author("developer")
        .execute()?;

    Ok(())
}

Using the Rhai API

// Open a repository
let repo = repo_open("project.fossil");

// List branches
let branches = branches_list(repo);
for b in branches {
    print("Branch: " + b);
}

// List files
let files = files_list(repo);
for f in files {
    print("  " + f.name() + " (" + f.size() + " bytes)");
}

// Read a file
let content = files_read(repo, "README.md");
print("Content:\n" + content);

// Use the filesystem interface for modifications
let repo_rw = repo_open_rw("project.fossil");
let fs = fs_new(repo_rw, "developer@example.com");
fs_write(fs, "newfile.txt", "Hello, World!");
fs_commit(fs);

Finding Files with Glob Patterns

use heroforge_core::Repository;

fn main() -> heroforge_core::Result<()> {
    let repo = Repository::open("project.fossil")?;

    // Find all Rust files on trunk
    let rust_files = repo.files().on_trunk().find("**/*.rs")?;
    for file in rust_files {
        println!("Found: {}", file.name);
    }

    // Find files in a specific directory on a branch
    let src_files = repo.files().on_branch("feature-x").find("src/**/*")?;
    
    // Find files at a specific tag
    let tagged_files = repo.files().at_tag("v1.0.0").find("*.md")?;
    
    Ok(())
}

Browsing History

use heroforge_core::Repository;

fn main() -> heroforge_core::Result<()> {
    let repo = Repository::open("project.fossil")?;

    // Get recent check-ins
    let history = repo.history().recent(10)?;
    for checkin in history {
        println!("{} | {} | {}", 
            &checkin.hash[..12], 
            checkin.user, 
            checkin.comment
        );
    }

    // List all branches
    let branches = repo.branches().list()?;
    for branch in branches {
        println!("Branch: {}", branch);
    }

    // List all tags
    let tags = repo.tags().list()?;
    for tag in tags {
        println!("Tag: {}", tag);
    }

    // Get the tip of a specific branch
    let feature_tip = repo.history().branch_tip("feature-x")?;
    println!("Feature branch tip: {}", feature_tip.hash);

    Ok(())
}

Filesystem Operations

use heroforge_core::Repository;

fn main() -> heroforge_core::Result<()> {
    let repo = Repository::open_rw("project.fossil")?;

    // Copy, move, delete files atomically
    let hash = repo.fs().modify()
        .message("Reorganize project")
        .author("developer")
        .copy_file("README.md", "docs/README.md")
        .move_dir("scripts", "tools")
        .delete_file("old_config.txt")
        .make_executable("tools/build.sh")
        .symlink("build", "tools/build.sh")
        .execute()?;

    // Advanced find with ignore patterns
    let files = repo.fs().find()
        .pattern("**/*.rs")
        .ignore("target/**")
        .ignore_hidden()
        .max_depth(3)
        .paths()?;
    
    // Utility functions
    println!("Exists: {}", repo.fs().exists("README.md")?);
    println!("Is dir: {}", repo.fs().is_dir("src")?);
    println!("Total size: {} bytes", repo.fs().du("**/*.rs")?);

    Ok(())
}

Builder API Overview

The library uses a fluent builder pattern for all operations:

Rhai API Functions

Repository

Filesystem

Files

Branches/Tags

Utilities

Feature Flags

Examples

The repository includes many examples demonstrating various features:

# Basic repository reading
cargo run --example read_repo

# Find files with glob patterns
cargo run --example find_and_read

# Branches and tags demonstration
cargo run --example branch_tag_test

# Comprehensive API demo
cargo run --example comprehensive_test

# Filesystem operations (copy, move, delete, chmod, symlinks)
cargo run --example fs_operations

# Advanced find with ignore patterns
cargo run --example fs_find

# QUIC sync (requires sync-quic feature)
cargo run --example quic_incremental_sync_test --features sync-quic

Documentation

Full API documentation is available at docs.rs/heroforge-core.

Generate local documentation with:

cargo doc --open --all-features

Architecture

A Fossil repository is a SQLite database containing:

• Blobs: Compressed file contents and manifests (zlib)
• Manifests: Check-in metadata with file lists, timestamps, comments
• Tags: Branch names, version tags, and other labels
• Events: Timeline of repository activity
• Delta encoding: Efficient storage of similar content

Many thanks to

• The Fossil SCM project for creating and maintaining Fossil on which this is based.