# Test Vectors Guide
**Version**: 1.0.0
**Generated**: 2025-01-14
**Algorithm**: bayesian_reputation_v1
## Overview
Test vectors provide a transparent, verifiable reference implementation of the KnowThat Reputation Engine's calculations. These JSON test cases enable third-party verification and ensure algorithm consistency across implementations.
## Test Vector Format
```json
{
"version": "1.0.0",
"algorithm": "bayesian_reputation_v1",
"generated": "2025-01-14T00:00:00Z",
"calculator_config": {
"confidence_k": 15.0,
"prior_base": 50.0,
"prior_max": 80.0
},
"test_cases": [...]
}
```
## Categories
### 1. New Agents (TC001-TC003)
Tests for agents with no interaction history:
- Brand new agent
- Week-old agent with no activity
- Month-old agent with no activity
### 2. Verified Agents (TC004-TC008, TC030-TC035)
Tests for agents with various credentials:
- MCP Levels 1, 2, and 3
- Identity verification
- Security audit passed
- Open source contribution
- Combined credentials
### 3. Review Patterns (TC009-TC014)
Tests for different rating distributions:
- 1-5 star ratings (50 reviews each)
- Mixed reviews (3.5 stars)
### 4. Confidence Levels (TC015-TC019)
Tests demonstrating confidence progression:
- Low confidence (1 interaction) - 6.25%
- Low-Medium (10 interactions) - 40%
- Medium (50 interactions) - 76.9%
- High (200 interactions) - 93%
- Very High (1000 interactions) - 98.5%
### 5. Edge Cases (TC020-TC024, TC036-TC041)
Boundary conditions and special cases:
- No reviews despite interactions
- Maximum allowed values
- Single perfect review
- Prior score cap
- Age progression (0-365 days)
### 6. High Interaction (TC025-TC029)
Performance at scale:
- 100 to 10,000 interactions
- Consistent 4.2 star rating
## Key Calculations
### Confidence Formula
```
confidence = interactions / (interactions + k)
where k = 15.0
```
### Score Formula
```
score = (1 - confidence) × prior_score + confidence × empirical_score
```
### Prior Score Components
- Base: 50.0
- MCP Level 1: +5.0
- MCP Level 2: +10.0
- MCP Level 3: +15.0
- Identity Verified: +5.0
- Security Audit: +5.0
- Open Source: +3.0
- Age Bonus: 0-5.0 (based on account age)
- Maximum: 80.0 (capped)
### Empirical Score
```
empirical_score = (average_rating - 1.0) / 4.0 × 100
```
## Using Test Vectors
### For Implementation Verification
```rust
// Load test vectors
let vectors = load_test_vectors("test_vectors.json");
// Validate your implementation
for test_case in vectors.test_cases {
let score = your_calculator.calculate(test_case.input);
assert_eq!(score, test_case.expected);
}
```
### For Third-Party Validation
1. Load the JSON test vectors
2. For each test case:
- Convert input to your agent data format
- Calculate the reputation score
- Compare with expected values
3. All values should match within 0.0001 tolerance
## Notable Test Cases
### TC001: New Agent Baseline
- Score: 50.0 (base prior)
- Confidence: 0.0
- Provisional: true
### TC008: Fully Verified Agent
- All credentials maxed
- Prior score: 80.0 (capped)
- Shows maximum achievable prior
### TC021: Maximum Scale
- 1M interactions, 500K reviews
- Confidence: 99.985%
- Tests algorithm at scale
### TC023: Prior Cap Example
- All credentials + 1 year age
- Prior components sum > 80
- Demonstrates cap enforcement
## Validation
Run the validation test:
```bash
cargo test validate_test_vectors
```
This ensures:
- All test cases produce expected results
- Categories have sufficient coverage
- Algorithm version matches
## Regenerating Test Vectors
To regenerate test vectors with current algorithm:
```bash
cargo run --example generate_test_vectors
```
This creates:
- `test_vectors.json` in project root
- `docs/test_vectors.json` for documentation
## Version History
- **1.0.0** (2025-01-14): Initial test vectors for Phase 3
- 41 comprehensive test cases
- 6 categories covering all scenarios
- Full algorithm transparency