rust-test-harness 0.1.3

# Rust Test Harness

A modern, feature-rich testing framework for Rust with real Docker container management, hooks, and parallel execution.

## Requirements

- **Docker**: This framework requires Docker to be installed and running on your system
- **Rust**: Rust 1.70+ with Cargo

## Features

- **Rust-Native Testing**: Works exactly like Rust's built-in `#[test]` attribute
- **Docker Integration**: Run tests in isolated containers
- **Container Hooks**: **NEW!** `before_each`/`after_each` for per-test container lifecycle
- **Test Hooks**: `before_all`, `before_each`, `after_each`, `after_all`
- **Parallel Execution**: Run tests concurrently with configurable concurrency
- **Tag-based Filtering**: Organize and filter tests by tags
- **Test Timeouts**: Set maximum execution time for tests
- **HTML Reports**: Generate beautiful, interactive test reports with automatic target folder organization

## Quick Start

### Basic Usage (Rust-Style)

Your test harness works exactly like Rust's built-in testing framework! You can use it in `mod tests` blocks and outside of `main()` functions. The framework is fully functional:

```rust
use rust_test_harness::test_case;

pub struct Calculator {
    value: i32,
}

impl Calculator {
    pub fn new() -> Self { Self { value: 0 } }
    pub fn add(&mut self, x: i32) { self.value += x; }
    pub fn get_value(&self) -> i32 { self.value }
}

#[cfg(test)]
mod tests {
    use super::*;
    
    // Test using test_case! macro - works exactly like #[test]
    test_case!(test_calculator_new, |_ctx| {
        let calc = Calculator::new();
        assert_eq!(calc.get_value(), 0);
        Ok(())
    });
    
    // Standard Rust test that also works
    #[test]
    fn test_subtraction() {
        let mut calc = Calculator::new();
        calc.add(10);
        // Your test logic here
        assert_eq!(calc.get_value(), 10);
    }
}
```

### Running Tests

```bash
# Run all tests (discovered by cargo test)
cargo test

# Run specific test
cargo test test_calculator_new

# Run tests with filter
cargo test calculator

# Run specific example
cargo test --example rust_style_tests

# Run specific test in an example
cargo test --example rust_style_tests test_calculator_new
```

### Advanced Features

#### Docker Integration

Docker integration is available through the container hooks system (see Container Integration section for details).

#### Test Hooks

**Important Note**: Hooks are built into the framework and work automatically. You don't need to manually call them.

```rust
use rust_test_harness::test_case;

#[cfg(test)]
mod tests {
    use super::*;
    
    // The framework automatically handles hooks for:
    // - before_all: Global setup before all tests
    // - before_each: Setup before each test (including container startup)
    // - after_each: Cleanup after each test (including container shutdown)
    // - after_all: Global cleanup after all tests
    
    test_case!(my_test, |_ctx| {
        // Your test logic here
        // Hooks run automatically
        Ok(())
    });
}
```

**What Hooks Can Do:**
- ✅ Setup/teardown test databases
- ✅ **Container lifecycle management** (NEW!)
- ✅ Initialize test data
- ✅ Clean up test files
- ✅ Manage test configuration
- ✅ Handle test environment setup

**Container Management with Hooks:**
```rust
use rust_test_harness::{before_each, after_each, ContainerConfig};

let container = ContainerConfig::new("postgres:13")
    .port(5432, 5432)
    .env("POSTGRES_PASSWORD", "test");

before_each(move |ctx| {
    let container_id = container.start()?;
    ctx.set_data("db_container_id", container_id);
    Ok(())
});

after_each(move |ctx| {
    let container_id = ctx.get_data::<String>("db_container_id").unwrap();
    container.stop(&container_id)?;
    Ok(())
});
```

**What Hooks Are NOT For:**
- ❌ Complex resource orchestration (use specialized macros)
- ❌ Cross-test data sharing (use `before_all`/`after_all` instead)

#### Tag-based Filtering

```rust
use rust_test_harness::test_with_tags;

test_with_tags("slow_integration_test", vec!["slow", "integration"], |_ctx| {
    // This test has tags: "slow" and "integration"
    std::thread::sleep(Duration::from_secs(1));
    Ok(())
});

// Skip slow tests: TEST_SKIP_TAGS=slow cargo test
```

#### HTML Reports

Generate beautiful, interactive HTML reports for your test results. All HTML reports are automatically stored in the `target/test-reports/` directory for clean project organization and easy CI/CD integration:

```rust
use rust_test_harness::run_tests_with_config;

let config = TestConfig {
    html_report: Some("test-results.html".to_string()),
    ..Default::default()
};

let result = run_tests_with_config(config);
// Generates target/test-reports/test-results.html with comprehensive test results
```

**HTML Report Features:**
- 📊 **Visual Summary**: Pass/fail/skip statistics with color-coded cards
- 📋 **Detailed Results**: Individual test results with status and error details
- 🎨 **Modern Design**: Responsive, mobile-friendly interface with gradients
- 🔍 **Test Information**: Tags, timeouts, Docker configuration, and error messages
- ⏱️ **Execution Time**: Total test execution duration
- 📱 **Responsive Layout**: Works on desktop, tablet, and mobile devices
- 🔽 **Expandable Details**: Click any test to view detailed metadata and configuration
- 🔍 **Search Functionality**: Search tests by name, status, or tags in real-time
- ⌨️ **Keyboard Shortcuts**: Ctrl+F (search), Ctrl+A (expand all), Ctrl+Z (collapse all)
- 🚨 **Auto-Expand Failed**: Failed tests automatically expand for better visibility

**Environment Variable:**
```bash
export TEST_HTML_REPORT=test-results.html
cargo test
# Report will be saved to: target/test-reports/test-results.html
```

**Path Resolution:**
- **Relative paths** (e.g., `"report.html"`) → Automatically stored in `target/test-reports/report.html`
- **Absolute paths** (e.g., `"/tmp/report.html"`) → Stored at the exact specified location
- **Target directory**: Uses `CARGO_TARGET_DIR` environment variable or defaults to `"target"`

**Target Folder Benefits:**
- 🗂️ **Clean Project Structure**: No HTML files cluttering your project root
- 🔄 **CI/CD Friendly**: Easy to exclude from version control and clean up
- 📁 **Organized Output**: All test reports in one dedicated location
- 🚀 **Rust Conventions**: Follows standard Rust project structure
- 🧹 **Easy Cleanup**: Simple to remove with `cargo clean`

#### **Interactive Features**

The HTML reports include several interactive features to enhance your testing experience:

**🔽 Expandable Test Details**
- Click on any test header to expand/collapse detailed information
- View test metadata including tags, timeouts, and Docker configuration
- Error details are automatically displayed for failed tests

**🔍 Real-Time Search**
- Search box filters tests by name, status, or tags
- Results update instantly as you type
- Case-insensitive search for better usability

**⌨️ Keyboard Shortcuts**
- `Ctrl+F` (or `Cmd+F` on Mac): Focus search box
- `Ctrl+A` (or `Cmd+A` on Mac): Expand all test details
- `Ctrl+Z` (or `Cmd+Z` on Mac): Collapse all test details

**🚨 Smart Defaults**
- Failed tests automatically expand for immediate visibility
- Hover effects provide visual feedback
- Responsive design adapts to all screen sizes

## Framework vs. Standard Testing

| Feature | Rust Standard | Rust Test Harness |
|---------|---------------|-------------------|
| Basic Tests | `#[test]` | `test_case!()` or `#[test]` |
| Test Discovery | ✅ | ✅ |
| Parallel Execution | ✅ | ✅ |
| Test Hooks | ❌ | ✅ (Automatic) |
| Docker Integration | ❌ | ✅ |
| Tag-based Filtering | ❌ | ✅ |
| Test Timeouts | ❌ | ✅ |
| HTML Reports | ❌ | ✅ (with target folder organization) |

**Note:** The framework is fully functional and works correctly in all test environments. HTML reports are automatically stored in the organized `target/test-reports/` directory for clean project structure.

## Migration from Standard Tests

Converting from standard Rust tests is simple:

**Before (Standard Rust):**
```rust
#[test]
fn test_my_function() {
    assert_eq!(my_function(2, 2), 4);
}
```

**After (Rust Test Harness):**
```rust
test_case!(test_my_function, |_ctx| {
    assert_eq!(my_function(2, 2), 4);
    Ok(())
});
```

**Or keep using `#[test]` unchanged:**
```rust
#[cfg(test)]
mod tests {
    use super::*;
    
    #[test]
    fn test_my_function() {
        // Your existing test code works unchanged
        assert_eq!(my_function(2, 2), 4);
    }
}
```

## ContainerConfig

The `ContainerConfig` struct provides a clean, builder-pattern approach to configuring containers for testing:

```rust
use rust_test_harness::ContainerConfig;
use std::time::Duration;

let container = ContainerConfig::new("redis:alpine")
    .port(6379, 6379)                    // Host port -> Container port
    .env("REDIS_PASSWORD", "secret")     // Environment variables
    .name("test_redis")                  // Container name
    .ready_timeout(Duration::from_secs(30)); // Wait for container readiness
```

**Available Methods:**
- `.port(host_port, container_port)` - Map host ports to container ports
- `.auto_port(container_port)` - Automatically assign available host port for container port
- `.env(key, value)` - Set environment variables
- `.name(name)` - Set container name
- `.ready_timeout(duration)` - Set readiness timeout
- `.no_auto_cleanup()` - Disable automatic cleanup (containers persist after tests)

**Container Lifecycle Methods:**
- `.start()` - Start container and return `ContainerInfo`
- `.stop(container_id)` - Stop container by ID

**Automatic Cleanup:**
By default, all containers are automatically stopped and removed when tests complete. This ensures a clean environment for each test run.

**Port Configuration Options:**

1. **Auto-Port Assignment** (Recommended):
   The framework automatically finds available ports on your system, eliminating port conflicts:

   ```rust
   let container = ContainerConfig::new("nginx:alpine")
       .auto_port(80)        // Automatically assign available host port for container port 80
       .auto_port(443);      // Automatically assign available host port for container port 443

   let container_info = container.start()?;
   println!("Container running on: {}", container_info.ports_summary());
   println!("Web accessible at: {}", container_info.primary_url().unwrap());
   ```

2. **User-Specified Ports**:
   You can also specify exact port mappings when you need specific ports:

   ```rust
   let container = ContainerConfig::new("postgres:13-alpine")
       .port(5432, 5432)     // Map host port 5432 to container port 5432
       .port(8080, 80);      // Map host port 8080 to container port 80

   let container_info = container.start()?;
   // PostgreSQL will be accessible on localhost:5432
   // HTTP service will be accessible on localhost:8080
   ```

3. **Mixed Configuration**:
   Combine both approaches for maximum flexibility:

   ```rust
   let container = ContainerConfig::new("httpd:alpine")
       .port(8080, 80)       // Fixed mapping for main service
       .auto_port(443)       // Auto-assign for HTTPS
       .auto_port(9090);     // Auto-assign for metrics

   let container_info = container.start()?;
   println!("HTTP: localhost:8080");
   if let Some(https_port) = container_info.host_port_for(443) {
       println!("HTTPS: localhost:{}", https_port);
   }
   ```

**ContainerInfo Object:**
When you start a container, you get a `ContainerInfo` object with:

- **Port Information**: Easy access to host ports and URLs
- **Container Details**: ID, image, name, and status
- **Convenience Methods**: Get URLs, port mappings, and summaries

```rust
let container_info = container.start()?;

// Get the host port for a specific container port
if let Some(host_port) = container_info.host_port_for(27017) {
    println!("MongoDB accessible at localhost:{}", host_port);
}

// Get all URLs
for url in &container_info.urls {
    println!("Service available at: {}", url);
}

// Get port summary
println!("Ports: {}", container_info.ports_summary());
```

## 🌐 Accessing Container Ports and URLs

The framework provides comprehensive access to container port information, making it easy to connect to your services:

### **Getting Host Ports**

```rust
let container = ContainerConfig::new("postgres:13-alpine")
    .auto_port(5432)    // Auto-assign available host port
    .env("POSTGRES_PASSWORD", "testpass");

let container_info = container.start()?;

// Get the actual host port that was assigned
if let Some(host_port) = container_info.host_port_for(5432) {
    println!("PostgreSQL running on: localhost:{}", host_port);
    
    // Use in connection strings
    let conn_str = format!("postgresql://user:pass@localhost:{}/db", host_port);
}
```

### **Getting Service URLs**

```rust
let web_container = ContainerConfig::new("nginx:alpine")
    .auto_port(80)
    .auto_port(443);

let web_info = web_container.start()?;

// Get ready-to-use URLs
if let Some(http_url) = web_info.url_for_port(80) {
    println!("HTTP service: {}", http_url);  // http://localhost:52341
    
    // Use directly in HTTP clients
    let response = reqwest::get(&http_url).await?;
}

// Get primary URL (first port)
if let Some(primary) = web_info.primary_url() {
    println!("Main service: {}", primary);
}
```

### **Real-World Usage Patterns**

```rust
// Pattern 1: Database Testing
let db_info = postgres_container.start()?;
if let Some(port) = db_info.host_port_for(5432) {
    std::env::set_var("DATABASE_URL", format!("postgresql://localhost:{}/test", port));
    // Now your application can connect using the environment variable
}

// Pattern 2: API Testing
let api_info = api_container.start()?;
if let Some(api_url) = api_info.primary_url() {
    let client = reqwest::Client::new();
    let response = client.get(&format!("{}/health", api_url)).send().await?;
    assert_eq!(response.status(), 200);
}

// Pattern 3: Multiple Services
let web_info = web_container.start()?;
let db_info = db_container.start()?;

println!("Services started:");
println!("  Web: {}", web_info.ports_summary());
println!("  DB:  {}", db_info.ports_summary());

// All ports are different - no conflicts!
```

### **ContainerInfo Methods Reference**

| Method | Purpose | Example |
|--------|---------|---------|
| `host_port_for(container_port)` | Get host port for specific container port | `container_info.host_port_for(80)` |
| `url_for_port(container_port)` | Get URL for specific container port | `container_info.url_for_port(80)` |
| `primary_url()` | Get URL for first port | `container_info.primary_url()` |
| `ports_summary()` | Human-readable port mappings | `container_info.ports_summary()` |
| `port_mappings` | All `(host_port, container_port)` pairs | `container_info.port_mappings` |
| `urls` | All service URLs | `container_info.urls` |

**Example with PostgreSQL:**
```rust
let postgres = ContainerConfig::new("postgres:13")
    .port(5432, 5432)
    .env("POSTGRES_DB", "testdb")
    .env("POSTGRES_USER", "testuser")
    .env("POSTGRES_PASSWORD", "testpass")
    .name("test_postgres")
    .ready_timeout(Duration::from_secs(30));

// Use in hooks
before_each(move |ctx| {
    let container_id = postgres.start()?;
    ctx.set_data("postgres_id", container_id);
    Ok(())
});
```

## Examples

Check out the `examples/` directory for comprehensive examples:

### Core Examples
- `minimal_rust_style.rs` - Minimal example showing Rust-style testing
- `rust_style_tests.rs` - Comprehensive Rust-style testing patterns
- `basic_usage.rs` - Basic framework usage with `test_case!`

### Real-World Examples
- `advanced_features.rs` - Advanced features and patterns
- `real_world_calculator.rs` - Real-world application testing
- `mongodb_integration.rs` - **NEW!** Database testing with container hooks (recommended approach)
- `auto_port_demo.rs` - **NEW!** Auto-port functionality and container management demo
- `container_port_access.rs` - **NEW!** Detailed guide on accessing container ports and URLs
- `cargo_test_integration.rs` - Integration with cargo test

### Container Management Examples
- `mongodb_integration.rs` - **Container Hooks Pattern**: Uses `before_each`/`after_each` for per-test container lifecycle

**Container Hooks Pattern**
The container hooks approach provides excellent control and isolation:
- Each test gets a fresh container
- Automatic cleanup prevents resource leaks
- Cleaner separation of concerns
- More flexible configuration options

### Running Examples

```bash
# Run a specific example
cargo run --example rust_style_tests

# Test a specific example
cargo test --example rust_style_tests

# Run specific test in an example
cargo test --example rust_style_tests test_calculator_new

# HTML reports are automatically generated in target/test-reports/
# when using examples with HTML reporting enabled
```

**Port Access Examples:**
```bash
# See auto-port functionality in action
cargo run --example auto_port_demo

# Learn how to access container ports and URLs
cargo run --example container_port_access

# Container port and URL access
cargo run --example container_port_access

# User-specified port mappings
cargo run --example user_specified_ports
```

## Container Management Patterns

### Container Lifecycle with Hooks (NEW!)

**Use `before_each` and `after_each` for per-test container management:**

```rust
use rust_test_harness::{test, before_each, after_each, ContainerConfig};

// Define container configuration
let mongo_container = ContainerConfig::new("mongo:5.0")
    .port(27017, 27017)
    .env("MONGO_INITDB_ROOT_USERNAME", "admin")
    .env("MONGO_INITDB_ROOT_PASSWORD", "password")
    .name("test_mongodb")
    .ready_timeout(Duration::from_secs(30));

// before_each starts a fresh container for each test
before_each(move |ctx| {
    let container_id = mongo_container.start()?;
    ctx.set_data("mongo_container_id", container_id);
    Ok(())
});

// after_each cleans up the container after each test
let mongo_container_clone = mongo_container.clone();
after_each(move |ctx| {
    let container_id = ctx.get_data::<String>("mongo_container_id").unwrap();
    mongo_container_clone.stop(&container_id)?;
    Ok(())
});

// Tests use the container
test("test_mongodb_operations", |ctx| {
    let container_id = ctx.get_data::<String>("mongo_container_id").unwrap();
    // Your test logic here
    Ok(())
});
```

**Benefits of Container Hooks:**
- ✅ **Fresh Containers**: Each test gets a completely isolated container
- ✅ **Automatic Cleanup**: No risk of resource leaks
- ✅ **Easy Configuration**: Builder pattern for container setup
- ✅ **Clean Separation**: Test logic separate from container management
- ✅ **Real-world Ready**: Can easily be extended to use actual Docker API

### For Docker Containers

**Use Container Hooks (Recommended approach):**
Refer to the Container Integration section above for the full example using `before_each`/`after_each` hooks.

**Benefits of Container Hooks:**
- **Fine-grained control**: Full control over container lifecycle
- **Better isolation**: Fresh container per test  
- **Automatic cleanup**: Prevents resource leaks
- **Flexible configuration**: Custom setup/teardown logic

### For Other Resources

**Hooks work great for:**
- Database connections
- File system operations
- Configuration management
- Test data setup/cleanup

## Configuration

Set environment variables to configure test execution:

```bash
# Filter tests by name
export TEST_FILTER=integration

# Skip tests with specific tags
export TEST_SKIP_TAGS=slow,flaky

# Set maximum concurrency
export TEST_MAX_CONCURRENCY=8

# Generate HTML report
export TEST_HTML_REPORT=test-results.html

# Skip hooks (for debugging)
export TEST_SKIP_HOOKS=true
```

## IDE Support

### RustRover and Similar IDEs

**`test_case!` macros:**
- ❌ No play buttons (macros aren't expanded during compilation)
- ✅ Tests are discoverable by `cargo test`
- ✅ Tests run correctly

**Standard `#[test]` functions:**
- ✅ Play buttons work normally
- ✅ Full IDE integration

**Recommended Approach:**
```rust
#[cfg(test)]
mod tests {
    use super::*;
    
    // Use test_case! for framework features
    test_case!(test_with_framework, |_ctx| {
        // Framework-specific logic
        Ok(())
    });
    
    // Use #[test] for IDE play button support
    #[test]
    fn test_standard_rust() {
        // Standard Rust testing
        assert_eq!(2 + 2, 4);
    }
}
```

**Framework Reliability:**
The framework is fully functional. All features work correctly in test environments, and HTML reports are automatically organized in the `target/test-reports/` directory for clean project structure.

## Why This Approach?

1. **Familiar**: Works exactly like Rust's built-in testing
2. **Discoverable**: Tests are found by `cargo test` automatically
3. **Compatible**: Existing Rust tests work unchanged
4. **Enhanced**: Adds powerful features without breaking existing patterns
5. **Flexible**: Use framework features when needed, standard patterns when not
6. **Automatic**: Hooks work automatically without manual setup
7. **Container-Ready**: **NEW!** Built-in container lifecycle management with hooks
8. **Isolated**: Each test gets fresh containers for true isolation
9. **Clean**: Builder pattern for container configuration
10. **Docker Integration Ready**: Can easily integrate with real Docker APIs
11. **Organized**: HTML reports automatically stored in `target/test-reports/` for clean project structure
12. **Reliable**: Fully functional framework with comprehensive test coverage


## Contributing

Contributions are welcome! This framework aims to enhance Rust's testing capabilities while maintaining full compatibility with existing testing patterns.