Rust Test Harness

A modern, feature-rich testing framework for Rust with real Docker container management, hooks, and parallel execution.

Requirements

Docker: This framework requires Docker to be installed and running on your system
Rust: Rust 1.70+ with Cargo

Features

Rust-Native Testing: Works exactly like Rust's built-in #[test] attribute
Docker Integration: Run tests in isolated containers
Container Hooks: NEW! before_each/after_each for per-test container lifecycle
Test Hooks: before_all, before_each, after_each, after_all
Parallel Execution: Run tests concurrently with configurable concurrency
Tag-based Filtering: Organize and filter tests by tags
Test Timeouts: Set maximum execution time for tests
HTML Reports: Generate beautiful, interactive test reports with automatic target folder organization

Quick Start

Basic Usage (Rust-Style)

Your test harness works exactly like Rust's built-in testing framework! You can use it in mod tests blocks and outside of main() functions. The framework is fully functional:

use rust_test_harness::test_case;

pub struct Calculator {
    value: i32,
}

impl Calculator {
    pub fn new() -> Self { Self { value: 0 } }
    pub fn add(&mut self, x: i32) { self.value += x; }
    pub fn get_value(&self) -> i32 { self.value }
}

#[cfg(test)]
mod tests {
    use super::*;
    
    // Test using test_case! macro - works exactly like #[test]
    test_case!(test_calculator_new, |_ctx| {
        let calc = Calculator::new();
        assert_eq!(calc.get_value(), 0);
        Ok(())
    });
    
    // Standard Rust test that also works
    #[test]
    fn test_subtraction() {
        let mut calc = Calculator::new();
        calc.add(10);
        // Your test logic here
        assert_eq!(calc.get_value(), 10);
    }
}

Running Tests

# Run all tests (discovered by cargo test)
cargo test

# Run specific test
cargo test test_calculator_new

# Run tests with filter
cargo test calculator

# Run specific example
cargo test --example rust_style_tests

# Run specific test in an example
cargo test --example rust_style_tests test_calculator_new

Advanced Features

Docker Integration

Docker integration is available through the container hooks system (see Container Integration section for details).

Test Hooks

Important Note: Hooks are built into the framework and work automatically. You don't need to manually call them.

use rust_test_harness::test_case;

#[cfg(test)]
mod tests {
    use super::*;
    
    // The framework automatically handles hooks for:
    // - before_all: Global setup before all tests
    // - before_each: Setup before each test (including container startup)
    // - after_each: Cleanup after each test (including container shutdown)
    // - after_all: Global cleanup after all tests
    
    test_case!(my_test, |_ctx| {
        // Your test logic here
        // Hooks run automatically
        Ok(())
    });
}

What Hooks Can Do:

✅ Setup/teardown test databases
✅ Container lifecycle management (NEW!)
✅ Initialize test data
✅ Clean up test files
✅ Manage test configuration
✅ Handle test environment setup

Container Management with Hooks:

use rust_test_harness::{before_each, after_each, ContainerConfig};

let container = ContainerConfig::new("postgres:13")
    .port(5432, 5432)
    .env("POSTGRES_PASSWORD", "test");

before_each(move |ctx| {
    let container_id = container.start()?;
    ctx.set_data("db_container_id", container_id);
    Ok(())
});

after_each(move |ctx| {
    let container_id = ctx.get_data::<String>("db_container_id").unwrap();
    container.stop(&container_id)?;
    Ok(())
});

What Hooks Are NOT For:

❌ Complex resource orchestration (use specialized macros)
❌ Cross-test data sharing (use before_all/after_all instead)

Tag-based Filtering

use rust_test_harness::test_with_tags;

test_with_tags("slow_integration_test", vec!["slow", "integration"], |_ctx| {
    // This test has tags: "slow" and "integration"
    std::thread::sleep(Duration::from_secs(1));
    Ok(())
});

// Skip slow tests: TEST_SKIP_TAGS=slow cargo test

HTML Reports

Generate beautiful, interactive HTML reports for your test results. All HTML reports are automatically stored in the target/test-reports/ directory for clean project organization and easy CI/CD integration:

use rust_test_harness::run_tests_with_config;

let config = TestConfig {
    html_report: Some("test-results.html".to_string()),
    ..Default::default()
};

let result = run_tests_with_config(config);
// Generates target/test-reports/test-results.html with comprehensive test results

HTML Report Features:

📊 Visual Summary: Pass/fail/skip statistics with color-coded cards
📋 Detailed Results: Individual test results with status and error details
🎨 Modern Design: Responsive, mobile-friendly interface with gradients
🔍 Test Information: Tags, timeouts, Docker configuration, and error messages
⏱️ Execution Time: Total test execution duration
📱 Responsive Layout: Works on desktop, tablet, and mobile devices
🔽 Expandable Details: Click any test to view detailed metadata and configuration
🔍 Search Functionality: Search tests by name, status, or tags in real-time
⌨️ Keyboard Shortcuts: Ctrl+F (search), Ctrl+A (expand all), Ctrl+Z (collapse all)
🚨 Auto-Expand Failed: Failed tests automatically expand for better visibility

Environment Variable:

export TEST_HTML_REPORT=test-results.html
cargo test
# Report will be saved to: target/test-reports/test-results.html

Path Resolution:

Relative paths (e.g., "report.html") → Automatically stored in target/test-reports/report.html
Absolute paths (e.g., "/tmp/report.html") → Stored at the exact specified location
Target directory: Uses CARGO_TARGET_DIR environment variable or defaults to "target"

Target Folder Benefits:

🗂️ Clean Project Structure: No HTML files cluttering your project root
🔄 CI/CD Friendly: Easy to exclude from version control and clean up
📁 Organized Output: All test reports in one dedicated location
🚀 Rust Conventions: Follows standard Rust project structure
🧹 Easy Cleanup: Simple to remove with cargo clean

Interactive Features

The HTML reports include several interactive features to enhance your testing experience:

🔽 Expandable Test Details

Click on any test header to expand/collapse detailed information
View test metadata including tags, timeouts, and Docker configuration
Error details are automatically displayed for failed tests

🔍 Real-Time Search

Search box filters tests by name, status, or tags
Results update instantly as you type
Case-insensitive search for better usability

⌨️ Keyboard Shortcuts

Ctrl+F (or Cmd+F on Mac): Focus search box
Ctrl+A (or Cmd+A on Mac): Expand all test details
Ctrl+Z (or Cmd+Z on Mac): Collapse all test details

🚨 Smart Defaults

Failed tests automatically expand for immediate visibility
Hover effects provide visual feedback
Responsive design adapts to all screen sizes

Framework vs. Standard Testing

Feature	Rust Standard	Rust Test Harness
Basic Tests	`#[test]`	`test_case!()` or `#[test]`
Test Discovery	✅	✅
Parallel Execution	✅	✅
Test Hooks	❌	✅ (Automatic)
Docker Integration	❌	✅
Tag-based Filtering	❌	✅
Test Timeouts	❌	✅
HTML Reports	❌	✅ (with target folder organization)

Note: The framework is fully functional and works correctly in all test environments. HTML reports are automatically stored in the organized target/test-reports/ directory for clean project structure.

Migration from Standard Tests

Converting from standard Rust tests is simple:

Before (Standard Rust):

#[test]
fn test_my_function() {
    assert_eq!(my_function(2, 2), 4);
}

After (Rust Test Harness):

test_case!(test_my_function, |_ctx| {
    assert_eq!(my_function(2, 2), 4);
    Ok(())
});

Or keep using #[test] unchanged:

#[cfg(test)]
mod tests {
    use super::*;
    
    #[test]
    fn test_my_function() {
        // Your existing test code works unchanged
        assert_eq!(my_function(2, 2), 4);
    }
}

ContainerConfig

The ContainerConfig struct provides a clean, builder-pattern approach to configuring containers for testing:

use rust_test_harness::ContainerConfig;
use std::time::Duration;

let container = ContainerConfig::new("redis:alpine")
    .port(6379, 6379)                    // Host port -> Container port
    .env("REDIS_PASSWORD", "secret")     // Environment variables
    .name("test_redis")                  // Container name
    .ready_timeout(Duration::from_secs(30)); // Wait for container readiness

Available Methods:

.port(host_port, container_port) - Map host ports to container ports
.auto_port(container_port) - Automatically assign available host port for container port
.env(key, value) - Set environment variables
.name(name) - Set container name
.ready_timeout(duration) - Set readiness timeout
.no_auto_cleanup() - Disable automatic cleanup (containers persist after tests)

Container Lifecycle Methods:

.start() - Start container and return ContainerInfo
.stop(container_id) - Stop container by ID

Automatic Cleanup: By default, all containers are automatically stopped and removed when tests complete. This ensures a clean environment for each test run.

Port Configuration Options:

Auto-Port Assignment (Recommended): The framework automatically finds available ports on your system, eliminating port conflicts:

let container = ContainerConfig::new("nginx:alpine")
    .auto_port(80)        // Automatically assign available host port for container port 80
    .auto_port(443);      // Automatically assign available host port for container port 443

let container_info = container.start()?;
println!("Container running on: {}", container_info.ports_summary());
println!("Web accessible at: {}", container_info.primary_url().unwrap());

User-Specified Ports: You can also specify exact port mappings when you need specific ports:

let container = ContainerConfig::new("postgres:13-alpine")
    .port(5432, 5432)     // Map host port 5432 to container port 5432
    .port(8080, 80);      // Map host port 8080 to container port 80

let container_info = container.start()?;
// PostgreSQL will be accessible on localhost:5432
// HTTP service will be accessible on localhost:8080

Mixed Configuration: Combine both approaches for maximum flexibility:

let container = ContainerConfig::new("httpd:alpine")
    .port(8080, 80)       // Fixed mapping for main service
    .auto_port(443)       // Auto-assign for HTTPS
    .auto_port(9090);     // Auto-assign for metrics

let container_info = container.start()?;
println!("HTTP: localhost:8080");
if let Some(https_port) = container_info.host_port_for(443) {
    println!("HTTPS: localhost:{}", https_port);
}

ContainerInfo Object: When you start a container, you get a ContainerInfo object with:

Port Information: Easy access to host ports and URLs
Container Details: ID, image, name, and status
Convenience Methods: Get URLs, port mappings, and summaries

let container_info = container.start()?;

// Get the host port for a specific container port
if let Some(host_port) = container_info.host_port_for(27017) {
    println!("MongoDB accessible at localhost:{}", host_port);
}

// Get all URLs
for url in &container_info.urls {
    println!("Service available at: {}", url);
}

// Get port summary
println!("Ports: {}", container_info.ports_summary());

🌐 Accessing Container Ports and URLs

The framework provides comprehensive access to container port information, making it easy to connect to your services:

Getting Host Ports

let container = ContainerConfig::new("postgres:13-alpine")
    .auto_port(5432)    // Auto-assign available host port
    .env("POSTGRES_PASSWORD", "testpass");

let container_info = container.start()?;

// Get the actual host port that was assigned
if let Some(host_port) = container_info.host_port_for(5432) {
    println!("PostgreSQL running on: localhost:{}", host_port);
    
    // Use in connection strings
    let conn_str = format!("postgresql://user:pass@localhost:{}/db", host_port);
}

Getting Service URLs

let web_container = ContainerConfig::new("nginx:alpine")
    .auto_port(80)
    .auto_port(443);

let web_info = web_container.start()?;

// Get ready-to-use URLs
if let Some(http_url) = web_info.url_for_port(80) {
    println!("HTTP service: {}", http_url);  // http://localhost:52341
    
    // Use directly in HTTP clients
    let response = reqwest::get(&http_url).await?;
}

// Get primary URL (first port)
if let Some(primary) = web_info.primary_url() {
    println!("Main service: {}", primary);
}

Real-World Usage Patterns

// Pattern 1: Database Testing
let db_info = postgres_container.start()?;
if let Some(port) = db_info.host_port_for(5432) {
    std::env::set_var("DATABASE_URL", format!("postgresql://localhost:{}/test", port));
    // Now your application can connect using the environment variable
}

// Pattern 2: API Testing
let api_info = api_container.start()?;
if let Some(api_url) = api_info.primary_url() {
    let client = reqwest::Client::new();
    let response = client.get(&format!("{}/health", api_url)).send().await?;
    assert_eq!(response.status(), 200);
}

// Pattern 3: Multiple Services
let web_info = web_container.start()?;
let db_info = db_container.start()?;

println!("Services started:");
println!("  Web: {}", web_info.ports_summary());
println!("  DB:  {}", db_info.ports_summary());

// All ports are different - no conflicts!

ContainerInfo Methods Reference

Method	Purpose	Example
`host_port_for(container_port)`	Get host port for specific container port	`container_info.host_port_for(80)`
`url_for_port(container_port)`	Get URL for specific container port	`container_info.url_for_port(80)`
`primary_url()`	Get URL for first port	`container_info.primary_url()`
`ports_summary()`	Human-readable port mappings	`container_info.ports_summary()`
`port_mappings`	All `(host_port, container_port)` pairs	`container_info.port_mappings`
`urls`	All service URLs	`container_info.urls`

Example with PostgreSQL:

let postgres = ContainerConfig::new("postgres:13")
    .port(5432, 5432)
    .env("POSTGRES_DB", "testdb")
    .env("POSTGRES_USER", "testuser")
    .env("POSTGRES_PASSWORD", "testpass")
    .name("test_postgres")
    .ready_timeout(Duration::from_secs(30));

// Use in hooks
before_each(move |ctx| {
    let container_id = postgres.start()?;
    ctx.set_data("postgres_id", container_id);
    Ok(())
});

Examples

Check out the examples/ directory for comprehensive examples:

Core Examples

minimal_rust_style.rs - Minimal example showing Rust-style testing
rust_style_tests.rs - Comprehensive Rust-style testing patterns
basic_usage.rs - Basic framework usage with test_case!

Real-World Examples

advanced_features.rs - Advanced features and patterns
real_world_calculator.rs - Real-world application testing
mongodb_integration.rs - NEW! Database testing with container hooks (recommended approach)
auto_port_demo.rs - NEW! Auto-port functionality and container management demo
container_port_access.rs - NEW! Detailed guide on accessing container ports and URLs
cargo_test_integration.rs - Integration with cargo test

Container Management Examples

mongodb_integration.rs - Container Hooks Pattern: Uses before_each/after_each for per-test container lifecycle

Container Hooks Pattern The container hooks approach provides excellent control and isolation:

Each test gets a fresh container
Automatic cleanup prevents resource leaks
Cleaner separation of concerns
More flexible configuration options

Running Examples

# Run a specific example
cargo run --example rust_style_tests

# Test a specific example
cargo test --example rust_style_tests

# Run specific test in an example
cargo test --example rust_style_tests test_calculator_new

# HTML reports are automatically generated in target/test-reports/
# when using examples with HTML reporting enabled

Port Access Examples:

# See auto-port functionality in action
cargo run --example auto_port_demo

# Learn how to access container ports and URLs
cargo run --example container_port_access

# Container port and URL access
cargo run --example container_port_access

# User-specified port mappings
cargo run --example user_specified_ports

Container Management Patterns

Container Lifecycle with Hooks (NEW!)

Use before_each and after_each for per-test container management:

use rust_test_harness::{test, before_each, after_each, ContainerConfig};

// Define container configuration
let mongo_container = ContainerConfig::new("mongo:5.0")
    .port(27017, 27017)
    .env("MONGO_INITDB_ROOT_USERNAME", "admin")
    .env("MONGO_INITDB_ROOT_PASSWORD", "password")
    .name("test_mongodb")
    .ready_timeout(Duration::from_secs(30));

// before_each starts a fresh container for each test
before_each(move |ctx| {
    let container_id = mongo_container.start()?;
    ctx.set_data("mongo_container_id", container_id);
    Ok(())
});

// after_each cleans up the container after each test
let mongo_container_clone = mongo_container.clone();
after_each(move |ctx| {
    let container_id = ctx.get_data::<String>("mongo_container_id").unwrap();
    mongo_container_clone.stop(&container_id)?;
    Ok(())
});

// Tests use the container
test("test_mongodb_operations", |ctx| {
    let container_id = ctx.get_data::<String>("mongo_container_id").unwrap();
    // Your test logic here
    Ok(())
});

Benefits of Container Hooks:

✅ Fresh Containers: Each test gets a completely isolated container
✅ Automatic Cleanup: No risk of resource leaks
✅ Easy Configuration: Builder pattern for container setup
✅ Clean Separation: Test logic separate from container management
✅ Real-world Ready: Can easily be extended to use actual Docker API

For Docker Containers

Use Container Hooks (Recommended approach): Refer to the Container Integration section above for the full example using before_each/after_each hooks.

Benefits of Container Hooks:

Fine-grained control: Full control over container lifecycle
Better isolation: Fresh container per test
Automatic cleanup: Prevents resource leaks
Flexible configuration: Custom setup/teardown logic

For Other Resources

Hooks work great for:

Database connections
File system operations
Configuration management
Test data setup/cleanup

Configuration

Set environment variables to configure test execution:

# Filter tests by name
export TEST_FILTER=integration

# Skip tests with specific tags
export TEST_SKIP_TAGS=slow,flaky

# Set maximum concurrency
export TEST_MAX_CONCURRENCY=8

# Generate HTML report
export TEST_HTML_REPORT=test-results.html

# Skip hooks (for debugging)
export TEST_SKIP_HOOKS=true

IDE Support

RustRover and Similar IDEs

test_case! macros:

❌ No play buttons (macros aren't expanded during compilation)
✅ Tests are discoverable by cargo test
✅ Tests run correctly

Standard #[test] functions:

✅ Play buttons work normally
✅ Full IDE integration

Recommended Approach:

#[cfg(test)]
mod tests {
    use super::*;
    
    // Use test_case! for framework features
    test_case!(test_with_framework, |_ctx| {
        // Framework-specific logic
        Ok(())
    });
    
    // Use #[test] for IDE play button support
    #[test]
    fn test_standard_rust() {
        // Standard Rust testing
        assert_eq!(2 + 2, 4);
    }
}

Framework Reliability: The framework is fully functional. All features work correctly in test environments, and HTML reports are automatically organized in the target/test-reports/ directory for clean project structure.

Why This Approach?

Familiar: Works exactly like Rust's built-in testing
Discoverable: Tests are found by cargo test automatically
Compatible: Existing Rust tests work unchanged
Enhanced: Adds powerful features without breaking existing patterns
Flexible: Use framework features when needed, standard patterns when not
Automatic: Hooks work automatically without manual setup
Container-Ready: NEW! Built-in container lifecycle management with hooks
Isolated: Each test gets fresh containers for true isolation
Clean: Builder pattern for container configuration
Docker Integration Ready: Can easily integrate with real Docker APIs
Organized: HTML reports automatically stored in target/test-reports/ for clean project structure
Reliable: Fully functional framework with comprehensive test coverage

Contributing

Contributions are welcome! This framework aims to enhance Rust's testing capabilities while maintaining full compatibility with existing testing patterns.

rust-test-harness 0.1.3

Rust Test Harness

Requirements

Features

Quick Start

Basic Usage (Rust-Style)

Running Tests

Advanced Features

Docker Integration

Test Hooks

Tag-based Filtering

HTML Reports

Interactive Features

Framework vs. Standard Testing

Migration from Standard Tests

ContainerConfig

🌐 Accessing Container Ports and URLs

Getting Host Ports

Getting Service URLs

Real-World Usage Patterns

ContainerInfo Methods Reference

Examples

Core Examples

Real-World Examples

Container Management Examples

Running Examples

Container Management Patterns

Container Lifecycle with Hooks (NEW!)

For Docker Containers

For Other Resources

Configuration

IDE Support

RustRover and Similar IDEs

Why This Approach?

Contributing