aprender 0.31.2

<!-- PCU: methodology-what-is-extreme-tdd | contract: contracts/apr-page-methodology-what-is-extreme-tdd-v1.yaml -->
<!-- Example: cargo run -p aprender-core --example none -->
<!-- Status: enforced -->

# What is EXTREME TDD?

## Prerequisites

This chapter is suitable for:
- Developers familiar with basic testing concepts
- Anyone interested in improving code quality
- No prior TDD experience required (we'll start from scratch)

**Recommended reading order:**
1. [Introduction](../introduction.md) ← Start here
2. This chapter (What is EXTREME TDD?)
3. [The RED-GREEN-REFACTOR Cycle](./red-green-refactor.md)

---

**EXTREME TDD** is a rigorous, zero-defect approach to test-driven development that combines traditional TDD with advanced testing techniques, automated quality gates, and Toyota Way principles.

## The Core Definition

EXTREME TDD extends classical Test-Driven Development by adding:

1. **Absolute test-first discipline** - No exceptions, no shortcuts
2. **Multiple testing layers** - Unit, integration, property-based, and mutation tests
3. **Automated quality enforcement** - Pre-commit hooks and CI/CD gates
4. **Mutation testing** - Verify tests actually catch bugs
5. **Zero-tolerance standards** - All tests pass, zero warnings, always
6. **Continuous improvement** - Kaizen mindset applied to code quality

## The Six Pillars

### 1. Tests Written First (NO Exceptions)

**Rule:** All production code must be preceded by a failing test.

```rust,ignore
// ❌ WRONG: Writing implementation first
pub fn train_test_split(x: &Matrix<f32>, y: &Vector<f32>, test_size: f32) {
    // ... implementation ...
}

// ✅ CORRECT: Write test first
#[test]
fn test_train_test_split_basic() {
    let x = Matrix::from_vec(10, 2, vec![/* ... */]).unwrap();
    let y = Vector::from_vec(vec![/* ... */]);

    let (x_train, x_test, y_train, y_test) =
        train_test_split(&x, &y, 0.2, None).unwrap();

    assert_eq!(x_train.shape().0, 8);  // 80% train
    assert_eq!(x_test.shape().0, 2);   // 20% test
}

// NOW implement train_test_split() to make this test pass
```

### 2. Minimal Implementation (Just Enough to Pass)

**Rule:** Write only the code needed to make tests pass.

Avoid:
- Premature optimization
- Speculative features
- "What if" scenarios
- Over-engineering

Example from aprender's Random Forest:

```rust,ignore
// CYCLE 1: Minimal bootstrap sampling
fn _bootstrap_sample(n_samples: usize, _seed: Option<u64>) -> Vec<usize> {
    // First implementation: just return indices
    (0..n_samples).collect()  // Fails test - not random!
}

// CYCLE 2: Add randomness (minimal to pass)
fn _bootstrap_sample(n_samples: usize, seed: Option<u64>) -> Vec<usize> {
    use rand::distributions::{Distribution, Uniform};
    use rand::SeedableRng;

    let dist = Uniform::from(0..n_samples);
    let mut indices = Vec::with_capacity(n_samples);

    if let Some(seed) = seed {
        let mut rng = rand::rngs::StdRng::seed_from_u64(seed);
        for _ in 0..n_samples {
            indices.push(dist.sample(&mut rng));
        }
    } else {
        let mut rng = rand::thread_rng();
        for _ in 0..n_samples {
            indices.push(dist.sample(&mut rng));
        }
    }

    indices
}
```

### 3. Comprehensive Refactoring (With Safety Net)

**Rule:** After tests pass, improve code quality while maintaining test coverage.

Refactor phase includes:
- Adding unit tests for edge cases
- Running clippy and fixing warnings
- Checking cyclomatic complexity
- Adding documentation
- Performance optimization
- Running mutation tests

### 4. Property-Based Testing (Cover Edge Cases)

**Rule:** Use property-based testing to automatically generate test cases.

Example from aprender:

```rust,ignore
use proptest::prelude::*;

proptest! {
    #[test]
    fn test_kfold_split_never_panics(
        n_samples in 2usize..1000,
        n_splits in 2usize..20
    ) {
        // Property: KFold.split() should never panic for valid inputs
        let kfold = KFold::new(n_splits);
        let _ = kfold.split(n_samples);  // Should not panic
    }

    #[test]
    fn test_kfold_uses_all_samples(
        n_samples in 10usize..100,
        n_splits in 2usize..10
    ) {
        // Property: All samples should appear exactly once as test data
        let kfold = KFold::new(n_splits);
        let splits = kfold.split(n_samples);

        let mut all_test_indices = Vec::new();
        for (_train, test) in splits {
            all_test_indices.extend(test);
        }

        all_test_indices.sort();
        let expected: Vec<usize> = (0..n_samples).collect();

        // Every sample should appear exactly once across all folds
        prop_assert_eq!(all_test_indices, expected);
    }
}
```

### 5. Mutation Testing (Verify Tests Work)

**Rule:** Use mutation testing to verify tests actually catch bugs.

```bash
# Run mutation tests
cargo mutants --in-place

# Example output:
# src/model_selection/mod.rs:148: CAUGHT (replaced >= with <=)
# src/model_selection/mod.rs:156: CAUGHT (replaced + with -)
# src/tree/mod.rs:234: MISSED (removed return statement)
```

**Target:** 80%+ mutation score (caught mutations / total mutations)

### 6. Zero Tolerance (All Gates Must Pass)

**Rule:** Every commit must pass ALL quality gates.

Quality gates (enforced via pre-commit hook):

```bash
#!/bin/bash
# .git/hooks/pre-commit

echo "Running quality gates..."

# 1. Format check
cargo fmt --check || {
    echo "❌ Format check failed. Run: cargo fmt"
    exit 1
}

# 2. Clippy (zero warnings)
cargo clippy -- -D warnings || {
    echo "❌ Clippy found warnings"
    exit 1
}

# 3. All tests pass
cargo test || {
    echo "❌ Tests failed"
    exit 1
}

# 4. Fast tests (quick feedback loop)
cargo test --lib || {
    echo "❌ Fast tests failed"
    exit 1
}

echo "✅ All quality gates passed"
```

## How EXTREME TDD Differs

| Aspect | Traditional TDD | EXTREME TDD |
|--------|----------------|-------------|
| **Test-First** | Encouraged | **Mandatory** (no exceptions) |
| **Test Types** | Mostly unit tests | Unit + Integration + **Property + Mutation** |
| **Quality Gates** | Optional CI checks | **Enforced pre-commit hooks** |
| **Coverage Target** | ~70-80% | **>90% + mutation score >80%** |
| **Warnings** | Fix eventually | **Zero tolerance** (must fix immediately) |
| **Refactoring** | As needed | **Comprehensive phase** in every cycle |
| **Documentation** | Write later | **Part of REFACTOR phase** |
| **Complexity** | Monitor occasionally | **Measured and enforced** (≤10 target) |
| **Philosophy** | Good practice | **Toyota Way principles** (Kaizen, Jidoka) |

## Benefits of EXTREME TDD

### 1. Zero Defects from Day One
By catching bugs through comprehensive testing and mutation testing, production code is defect-free.

### 2. Fearless Refactoring
With comprehensive test coverage, you can refactor with confidence, knowing tests will catch regressions.

### 3. Living Documentation
Tests serve as executable documentation that never gets outdated.

### 4. Faster Development
Paradoxically, writing tests first speeds up development by:
- Catching bugs earlier (cheaper to fix)
- Reducing debugging time
- Enabling confident refactoring
- Preventing regression bugs

### 5. Better API Design
Writing tests first forces you to think about API usability before implementation.

Example from aprender:

```rust,ignore
// Test-first API design led to clean builder pattern
let mut rf = RandomForestClassifier::new(20)
    .with_max_depth(5)
    .with_random_state(42);  // Fluent, readable API
```

### 6. Objective Quality Metrics
TDG (Technical Debt Gradient) provides measurable quality:

```bash
$ pmat analyze tdg src/
TDG Score: 93.3/100 (A)

Breakdown:
- Test Coverage:  97.2% (weight: 30%) ✅
- Complexity:     8.1 avg (weight: 25%) ✅
- Documentation:  94% (weight: 20%) ✅
- Modularity:     A (weight: 15%) ✅
- Error Handling: 96% (weight: 10%) ✅
```

## Real-World Impact

**Aprender Results** (using EXTREME TDD):
- 184 passing tests (+19 in latest session)
- ~97% coverage
- 93.3/100 TDG score (A grade)
- Zero production defects
- <0.01s fast test time

**Traditional Approach** (typical results):
- ~60-70% coverage
- ~80/100 TDG score (C grade)
- Multiple production defects
- Regression bugs
- Fear of refactoring

## When to Use EXTREME TDD

**✅ Ideal for:**
- Production libraries and frameworks
- Safety-critical systems
- Financial and medical software
- Open-source projects (quality signal)
- ML/AI systems (complex logic)
- Long-term maintainability

**⚠️ Consider tradeoffs for:**
- Prototypes and spikes (use regular TDD)
- UI/UX exploration (harder to test-first)
- Throwaway code
- Very tight deadlines (though EXTREME TDD often saves time)

## Summary

EXTREME TDD is:
- **Disciplined**: Tests FIRST, no exceptions
- **Comprehensive**: Multiple testing layers
- **Automated**: Quality gates enforced
- **Measured**: Objective metrics (TDG, mutation score)
- **Continuous**: Kaizen mindset
- **Zero-tolerance**: All tests pass, zero warnings

## Next Steps

Now that you understand what EXTREME TDD is, continue your learning:

1. **[The RED-GREEN-REFACTOR Cycle](./red-green-refactor.md)** ← Next
   Learn the fundamental cycle of EXTREME TDD

2. **[Test-First Philosophy](./test-first-philosophy.md)**
   Understand why tests must come first

3. **[Zero Tolerance Quality](./zero-tolerance.md)**
   Learn about enforcing quality gates

4. **[Property-Based Testing](../advanced-testing/property-based-testing.md)**
   Advanced testing techniques for edge cases

5. **[Mutation Testing](../advanced-testing/mutation-testing.md)**
   Verify your tests actually catch bugs