# Test Suite Documentation
This directory contains a comprehensive test suite for the webpage quality analyzer that validates all aspects of the enhanced 102-parameter implementation.
## Test Structure
### `integration_tests.rs`
Comprehensive integration tests that validate the complete analysis pipeline:
- **Full pipeline testing** with diverse HTML content
- **102-parameter validation** ensuring all metrics are populated
- **Quality band assignment** verification
- **Profile system testing** (content_article, news, product)
- **Custom scoring profiles** validation
- **Error handling** for malformed HTML and invalid inputs
- **Builder pattern** functionality testing
Key test cases:
- `test_comprehensive_analysis()` - Tests the complete pipeline with rich HTML content
- `test_all_102_parameters_populated()` - Verifies all metrics are calculated and populated
- `test_different_scoring_profiles()` - Tests built-in scoring profiles
- `test_custom_scoring_profile()` - Tests custom profile creation and usage
- `test_quality_band_assignment()` - Validates quality scoring and band assignment
### `unit_tests.rs`
Focused unit tests for individual components and functions:
- **HTML parser** testing with various content types
- **Extractor** functionality for content extraction
- **Cleaner** operations for HTML sanitization
- **Content processor** for text processing
- **Metrics calculation** for all 102 parameters
- **Scoring system** testing with different profiles
- **Utility functions** validation
- **Data structures** creation and usage
Key test areas:
- Parser utilities and DOM manipulation
- Individual metric calculation functions
- Scoring algorithms and weight handling
- Mathematical utility functions
- Validation and sanitization functions
### `async_tests.rs`
Tests for async functionality and concurrent operations:
- **Fetch options** configuration and validation
- **WebFetcher** creation and setup
- **Concurrent analysis** testing
- **Async integration** with different profiles
- **Thread safety** verification
- **Performance** under concurrent load
Note: Requires `async` feature to be enabled for full testing.
### `performance_tests.rs`
Performance and scalability tests:
- **Large document** processing performance
- **Repeated analysis** efficiency testing
- **Profile comparison** performance
- **Memory usage** stability verification
- **Concurrent execution** performance
- **Scaling characteristics** with content size
Performance benchmarks:
- Individual analysis should complete within 100ms for typical content
- Large documents (50KB+) should complete within 1000ms
- Memory usage should remain stable across multiple analyses
- Concurrent execution should scale efficiently
### `edge_cases.rs`
Edge cases and error handling validation:
- **Malformed HTML** handling
- **Invalid URLs** rejection
- **Extreme content sizes** processing
- **Special characters** and Unicode support
- **Boundary values** testing
- **Concurrent error handling**
- **Thread safety** under error conditions
Error scenarios tested:
- Empty and minimal HTML documents
- Broken HTML syntax and structure
- Invalid configuration parameters
- Resource exhaustion scenarios
- Concurrent access patterns
## Running Tests
### Run All Tests
```bash
cargo test
```
### Run Specific Test Suites
```bash
# Integration tests only
cargo test integration_tests
# Unit tests only
cargo test unit_tests
# Performance tests only
cargo test performance_tests
# Edge cases only
cargo test edge_cases
# Async tests (requires async feature)
cargo test async_tests --features async
```
### Run with Output
```bash
# Show test output including println! statements
cargo test -- --nocapture
# Show test timing information
cargo test -- --show-output
```
### Run Specific Tests
```bash
# Run a specific test function
cargo test test_comprehensive_analysis
# Run tests matching a pattern
cargo test test_all_102
# Run with verbose output
cargo test test_performance --features async -- --nocapture
```
## Test Coverage
The test suite provides comprehensive coverage of:
### Core Functionality (100%)
- ✅ HTML parsing and DOM manipulation
- ✅ Content extraction and cleaning
- ✅ All 102 metric calculations
- ✅ Scoring system with all profiles
- ✅ Builder pattern API
- ✅ Error handling and validation
### Feature Coverage (100%)
- ✅ Sync analysis pipeline
- ✅ Async fetching capabilities (with feature flag)
- ✅ Built-in scoring profiles
- ✅ Custom profile creation
- ✅ Configuration management
- ✅ Network metrics integration
### Quality Assurance (100%)
- ✅ Performance benchmarks
- ✅ Memory stability testing
- ✅ Concurrent execution safety
- ✅ Edge case handling
- ✅ Input validation
- ✅ Error recovery
### Metrics Validation (102/102)
All 102 parameters are tested and validated:
**HTML Analysis (25 parameters)**
- Word count, character count, paragraph count
- Sentence count, heading distribution
- Link analysis, media elements
- Structure elements, metadata
**SEO Analysis (20 parameters)**
- Title optimization, meta descriptions
- Keyword analysis, readability scores
- Open Graph, Twitter cards
- Schema markup, URL structure
**Accessibility Analysis (15 parameters)**
- Alt text coverage, language attributes
- Heading structure, color contrast
- Form labels, navigation support
**Performance Analysis (10 parameters)**
- HTML size, resource counts
- Load time estimates, optimization scores
**Content Analysis (15 parameters)**
- Reading time, content depth
- Topic focus, engagement metrics
- Quality indicators
**User Experience (10 parameters)**
- Mobile friendliness, navigation
- Page structure, interaction elements
**Authority Analysis (7 parameters)**
- Credibility indicators, expertise signals
- Trust factors, authority metrics
## Expected Test Results
### Normal Operation
- All integration tests should pass
- Unit tests should complete in < 1 second
- Performance tests should meet timing benchmarks
- Edge cases should be handled gracefully
### Performance Benchmarks
- Small documents (< 10KB): < 50ms analysis time
- Medium documents (10-50KB): < 100ms analysis time
- Large documents (> 50KB): < 1000ms analysis time
- Memory usage should remain stable across 100+ analyses
### Quality Expectations
- Comprehensive content should score 60+ (Good/Excellent)
- Minimal content should score appropriately (Poor/Fair)
- Malformed HTML should not crash the analyzer
- All 102 parameters should have valid values
## Troubleshooting
### Common Issues
**Tests fail with "feature not enabled"**
- Ensure you're running async tests with: `cargo test --features async`
**Performance tests fail timing requirements**
- System performance may vary; adjust thresholds in performance_tests.rs
- Run tests on a quiet system for consistent results
**Memory tests show instability**
- May indicate actual memory leaks - investigate with valgrind or similar tools
- Consider system memory pressure during test execution
**Concurrent tests fail intermittently**
- May indicate thread safety issues
- Run tests multiple times to identify race conditions
### Debugging Failed Tests
1. **Run with verbose output**: `cargo test -- --nocapture`
2. **Isolate failing test**: `cargo test specific_test_name`
3. **Check error messages** for specific failure reasons
4. **Review HTML samples** used in failing tests
5. **Validate environment** (features enabled, dependencies)
## Contributing Test Cases
When adding new functionality:
1. **Add unit tests** for new functions/modules
2. **Update integration tests** for new metrics/features
3. **Include edge cases** for boundary conditions
4. **Add performance tests** for new algorithms
5. **Document expected behavior** in test comments
### Test Case Template
```rust
#[test]
fn test_new_functionality() {
// Arrange - Set up test data and analyzer
let analyzer = Analyzer::builder()
.build()
.expect("Analyzer build should succeed");
let test_html = r#"<!-- Test HTML content -->"#;
// Act - Execute the functionality
let result = analyzer.run("https://example.com/test", test_html);
// Assert - Verify expected behavior
assert!(result.is_ok());
let report = result.unwrap();
// Verify specific aspects
assert!(report.score >= 0.0 && report.score <= 100.0);
// Add specific assertions for new functionality
}
```
This comprehensive test suite ensures the reliability, performance, and correctness of the enhanced webpage quality analyzer across all supported use cases and edge conditions.