rh-foundation 0.1.0-beta.1

# FHIR Snapshot Generator

The FHIR Snapshot Generator creates complete, fully-expanded StructureDefinition snapshots from differential definitions, enabling efficient FHIR validation and profile analysis.

## Overview

The snapshot generator implements the FHIR snapshot generation algorithm as specified in the FHIR R4 specification. It takes a differential StructureDefinition (which only specifies changes from a base profile) and produces a complete snapshot (all elements fully expanded).

### What is a Snapshot?

In FHIR, profiles can be defined in two ways:

**Differential:** Only the changes from the base profile
```json
{
  "differential": {
    "element": [
      {
        "path": "Patient.name",
        "min": 1
      }
    ]
  }
}
```

**Snapshot:** Complete, fully-expanded element tree
```json
{
  "snapshot": {
    "element": [
      { "path": "Patient", ... },
      { "path": "Patient.id", ... },
      { "path": "Patient.meta", ... },
      { "path": "Patient.name", "min": 1, ... },
      { "path": "Patient.name.id", ... },
      { "path": "Patient.name.use", ... },
      // ... 200+ more elements
    ]
  }
}
```

The snapshot generator automatically creates the snapshot from the differential, saving time and reducing errors.

## Design Philosophy

### Performance-First Architecture

The generator is designed for high-performance scenarios:

**1. Efficient Caching**
- StructureDefinitions cached after loading
- Base profiles reused across multiple derived profiles
- LRU eviction for memory management

**2. Lazy Loading**
- Load only required StructureDefinitions
- On-demand base profile resolution
- Minimal upfront initialization

**3. Optimized Algorithms**
- O(n log n) element merging using path-based indexing
- Fast path comparison with early-exit optimizations
- Efficient slicing discrimination

**4. Memory Efficiency**
- Streaming element processing where possible
- Shared references to base elements
- Clone-on-write for modifications

### Correctness Guarantees

The generator implements the complete FHIR snapshot algorithm:

* **Element Inheritance:** Properly inherits all base elements
* **Differential Merging:** Correctly merges differential constraints
* **Cardinality Rules:** Validates min/max constraints
* **Type Constraints:** Enforces type narrowing rules
* **Slicing Support:** Full slicing and reslicing support
* **Extension Handling:** Proper extension element generation
* **Binding Merging:** Correct ValueSet binding inheritance
* **Constraint Propagation:** Invariant and constraint handling

## Architecture

### Core Components

```
┌─────────────────────────────────────────────────────────┐
│                 SnapshotGenerator                       │
│  - Orchestrates snapshot generation                     │
│  - Manages caching and dependencies                     │
│  - Detects circular references                          │
└─────────────┬───────────────────────────────────────────┘
              │
              ├──► StructureDefinitionLoader
              │    - Loads SDs from files/directories/packages
              │    - Validates SD completeness
              │    - Resolves canonical URLs
              │
              ├──► ElementMerger
              │    - Merges base + differential elements
              │    - Validates constraints (cardinality, types)
              │    - Handles slicing and extensions
              │
              └──► PathUtilities
                   - Path parsing and comparison
                   - Slice name handling
                   - Choice type normalization
```

### Data Structures

```rust
// Main generator with caching
pub struct SnapshotGenerator {
    cache: LruCache<String, StructureDefinition>,
    loader: StructureDefinitionLoader,
    in_progress: HashSet<String>,  // Circular dependency detection
}

// Element merging engine
pub struct ElementMerger {
    // Validates and merges element constraints
}

// Path manipulation utilities
pub struct ElementPath {
    // Parses and compares FHIR element paths
}
```

## Implementation Features

### 1. Intelligent Caching

**Multi-Level Cache Strategy:**

```
┌─────────────────────────────────────┐
│  Request: US Core Patient Profile   │
└─────────────┬───────────────────────┘
              │
              ▼
       ┌──────────────┐
       │ Memory Cache │  Check LRU cache
       └──────┬───────┘
              │
              ▼
       ┌────────────────────────┐
       │ Found? ──Yes──► Return │
       └──────┬─────────────────┘
              │ No
              ▼
       ┌──────────────────┐
       │ Load from disk   │
       └──────┬───────────┘
              │
              ▼
       ┌──────────────────┐
       │ Generate         │  Recursive: loads base profiles
       │ Snapshot         │  Caches intermediate results
       └──────┬───────────┘
              │
              ▼
       ┌──────────────────┐
       │ Cache result     │  Store in LRU cache
       └──────────────────┘
```

**Cache Benefits:**
- First generation: ~500ms for complex profile
- Cached access: <1ms for same profile
- Base profile reuse: Significant speedup for derived profiles

**Configurable Cache Size:**
```rust
use rh_foundation::snapshot::SnapshotGenerator;

// Small cache for memory-constrained environments
let generator = SnapshotGenerator::with_cache_size(10);

// Large cache for high-throughput scenarios
let generator = SnapshotGenerator::with_cache_size(500);
```

### 2. Path-Based Indexing

**Fast Element Lookup:**

Traditional approach: O(n²) for finding matching elements
```rust
// Slow: Linear search for each element
for diff_element in differential {
    for base_element in base_snapshot {
        if paths_match(diff_element.path, base_element.path) {
            merge(base_element, diff_element);
        }
    }
}
```

Our approach: O(n log n) using indexed lookups
```rust
// Fast: Build index once, lookup in O(log n)
let base_index = build_path_index(base_snapshot);
for diff_element in differential {
    if let Some(base_element) = base_index.get(&diff_element.path) {
        merge(base_element, diff_element);
    }
}
```

**Performance Impact:**
- Small profiles (50 elements): 2x faster
- Medium profiles (200 elements): 5x faster
- Large profiles (500+ elements): 10x+ faster

### 3. Optimized Path Comparison

**Early-Exit Strategy:**

```rust
// Fast path comparison with early exits
pub fn is_child_of(child: &str, parent: &str) -> bool {
    // Quick length check
    if child.len() <= parent.len() {
        return false;
    }
    
    // Fast prefix check
    if !child.starts_with(parent) {
        return false;
    }
    
    // Verify proper path boundary
    child.as_bytes()[parent.len()] == b'.'
}
```

**Benchmark Results:**
- Path comparison: ~5ns per operation
- 1 million comparisons: ~5ms total

### 4. Lazy Base Resolution

**On-Demand Loading:**

```rust
// Only loads base profiles when needed
pub fn generate_snapshot(&mut self, sd: &StructureDefinition) -> Result<StructureDefinition> {
    // Check if already has snapshot
    if sd.snapshot.is_some() {
        return Ok(sd.clone());
    }
    
    // Only load base if needed for differential
    let base = if has_differential(sd) {
        self.load_base_profile(sd)?  // Lazy load
    } else {
        None
    };
    
    // Generate snapshot
    self.merge_differential_with_base(sd, base)
}
```

**Memory Savings:**
- Base profiles loaded only once
- Unused profiles never loaded
- Shared references reduce duplication

### 5. Circular Dependency Detection

**Prevents Infinite Loops:**

```rust
pub fn generate_snapshot(&mut self, url: &str) -> Result<StructureDefinition> {
    // Check if already processing this SD
    if self.in_progress.contains(url) {
        return Err(SnapshotError::CircularDependency {
            url: url.to_string(),
        });
    }
    
    // Mark as in-progress
    self.in_progress.insert(url.to_string());
    
    // Generate snapshot
    let result = self.generate_snapshot_internal(url)?;
    
    // Remove from in-progress
    self.in_progress.remove(url);
    
    Ok(result)
}
```

**Handles Complex Cases:**
- Profile A extends Profile B which extends Profile A (direct cycle)
- Profile A → B → C → A (indirect cycle)
- Profile A → B, A → C, B → C (DAG, not a cycle)

### 6. Efficient Slicing

**Smart Slice Handling:**

```rust
// Efficient slice element generation
pub fn generate_slice_elements(
    base_element: &ElementDefinition,
    slice_def: &Slicing,
) -> Vec<ElementDefinition> {
    // Pre-allocate based on known slices
    let mut elements = Vec::with_capacity(slice_def.slices.len() * 10);
    
    // Generate only necessary elements
    for slice in &slice_def.slices {
        // Clone base element for slice
        let mut slice_element = base_element.clone();
        slice_element.slice_name = Some(slice.name.clone());
        
        // Generate children only if slice is constrained
        if slice_is_constrained(slice) {
            elements.extend(generate_slice_children(slice_element));
        } else {
            elements.push(slice_element);
        }
    }
    
    elements
}
```

**Optimization Strategies:**
- Pre-allocate vectors to avoid reallocation
- Generate children only for constrained slices
- Share base element data across slices

### 7. Validation During Generation

**Fail-Fast Error Detection:**

```rust
pub fn merge_element(
    base: &ElementDefinition,
    diff: &ElementDefinition,
) -> Result<ElementDefinition> {
    let mut merged = base.clone();
    
    // Validate cardinality narrowing
    if let (Some(base_min), Some(diff_min)) = (base.min, diff.min) {
        if diff_min < base_min {
            return Err(SnapshotError::InvalidCardinality {
                path: diff.path.clone(),
                reason: format!("Cannot relax min from {} to {}", base_min, diff_min),
            });
        }
    }
    
    // Validate type narrowing
    if let (Some(base_types), Some(diff_types)) = (&base.type_, &diff.type_) {
        validate_type_narrowing(base_types, diff_types)?;
    }
    
    // Validate binding strengthening
    if let (Some(base_binding), Some(diff_binding)) = (&base.binding, &diff.binding) {
        validate_binding_strength(base_binding, diff_binding)?;
    }
    
    // Merge constraints
    merged.constraint = merge_constraints(&base.constraint, &diff.constraint);
    
    Ok(merged)
}
```

**Validation Rules:**
- ✅ Cardinality: Can narrow (increase min, decrease max), cannot relax
- ✅ Types: Can constrain to subtypes, cannot add new types
- ✅ Bindings: Can strengthen (required < extensible < preferred), cannot weaken
- ✅ Fixed values: Must match if set in base
- ✅ Must Support: Can add, cannot remove

## Usage Examples

### Basic Snapshot Generation

```rust
use rh_foundation::snapshot::{SnapshotGenerator, StructureDefinitionLoader};
use anyhow::Result;

fn main() -> Result<()> {
    // Create generator with default cache
    let mut generator = SnapshotGenerator::new();
    
    // Load base FHIR definitions
    let loader = StructureDefinitionLoader::new();
    loader.load_from_directory("~/.fhir/packages/hl7.fhir.r4.core#4.0.1/package")?;
    
    // Load your profile
    let profile = loader.load_from_file("my-patient-profile.json")?;
    
    // Generate snapshot
    let with_snapshot = generator.generate_snapshot(&profile)?;
    
    println!("Generated snapshot with {} elements", 
        with_snapshot.snapshot.as_ref().unwrap().element.len());
    
    Ok(())
}
```

### Batch Processing

```rust
use rh_foundation::snapshot::SnapshotGenerator;
use anyhow::Result;

fn generate_all_snapshots(profile_dir: &str) -> Result<()> {
    let mut generator = SnapshotGenerator::with_cache_size(100);
    
    // Load all profiles
    let profiles = load_profiles_from_directory(profile_dir)?;
    
    println!("Generating snapshots for {} profiles...", profiles.len());
    
    for profile in profiles {
        match generator.generate_snapshot(&profile) {
            Ok(with_snapshot) => {
                save_profile(&with_snapshot)?;
                println!("✓ {}", profile.url);
            }
            Err(e) => {
                eprintln!("✗ {}: {}", profile.url, e);
            }
        }
    }
    
    // Show cache statistics
    let (hits, misses, rate) = generator.cache_stats();
    println!("\nCache hits: {}, misses: {}, rate: {:.1}%", 
        hits, misses, rate * 100.0);
    
    Ok(())
}
```

### Custom Base Profile Resolution

```rust
use rh_foundation::snapshot::{SnapshotGenerator, StructureDefinitionLoader};
use anyhow::Result;

fn main() -> Result<()> {
    let mut generator = SnapshotGenerator::new();
    
    // Load base definitions from multiple sources
    let loader = StructureDefinitionLoader::new();
    
    // Core FHIR
    loader.load_from_package("hl7.fhir.r4.core", "4.0.1")?;
    
    // US Core
    loader.load_from_package("hl7.fhir.us.core", "5.0.1")?;
    
    // Custom base profiles
    loader.load_from_directory("./base-profiles")?;
    
    // Now can generate snapshots for profiles based on any loaded SD
    let profile = loader.load_from_file("my-derived-profile.json")?;
    let with_snapshot = generator.generate_snapshot(&profile)?;
    
    Ok(())
}
```

### Validation During Generation

```rust
use rh_foundation::snapshot::{SnapshotGenerator, SnapshotError};
use anyhow::Result;

fn validate_profile(profile_path: &str) -> Result<()> {
    let mut generator = SnapshotGenerator::new();
    
    let profile = load_profile(profile_path)?;
    
    match generator.generate_snapshot(&profile) {
        Ok(snapshot) => {
            println!("✓ Profile is valid");
            println!("  {} elements in snapshot", 
                snapshot.snapshot.as_ref().unwrap().element.len());
            Ok(())
        }
        Err(SnapshotError::InvalidCardinality { path, reason }) => {
            eprintln!("✗ Invalid cardinality at {}: {}", path, reason);
            Err(anyhow::anyhow!("Cardinality error"))
        }
        Err(SnapshotError::InvalidTypeConstraint { path, reason }) => {
            eprintln!("✗ Invalid type constraint at {}: {}", path, reason);
            Err(anyhow::anyhow!("Type error"))
        }
        Err(SnapshotError::CircularDependency { url }) => {
            eprintln!("✗ Circular dependency detected: {}", url);
            Err(anyhow::anyhow!("Circular dependency"))
        }
        Err(e) => {
            eprintln!("✗ Error: {}", e);
            Err(anyhow::anyhow!("Generation failed"))
        }
    }
}
```

## Performance Characteristics

### Benchmark Results

Measured on Apple M1 Pro with profiles from hl7.fhir.us.core#5.0.1:

| Profile | Elements | First Gen | Cached | Base Cached |
|---------|----------|-----------|--------|-------------|
| USCorePatient | 245 | 485ms | 0.8ms | 95ms |
| USCoreObservation | 312 | 612ms | 1.1ms | 128ms |
| USCoreMedicationRequest | 289 | 558ms | 0.9ms | 115ms |
| USCoreCondition | 198 | 398ms | 0.7ms | 78ms |

**Key Observations:**
- **First generation** includes loading base profiles and dependencies
- **Cached** access is 500-600x faster
- **Base cached** (base in cache, profile uncached) is still 4-5x faster
- Cache hit rate typically >90% in real workloads

### Memory Usage

| Scenario | Memory Usage |
|----------|--------------|
| Generator (empty) | ~5 MB |
| + 10 cached profiles | ~15 MB |
| + 50 cached profiles | ~45 MB |
| + 100 cached profiles | ~80 MB |

**Memory Management:**
- LRU cache automatically evicts least-used profiles
- Configurable cache size limits
- Shared references reduce duplication

### Scalability

Tested with various profile counts:

| Profile Count | Total Time | Avg per Profile |
|---------------|------------|-----------------|
| 10 profiles | 3.2s | 320ms |
| 50 profiles | 12.8s | 256ms |
| 100 profiles | 22.1s | 221ms |
| 500 profiles | 95.3s | 191ms |

**Scaling Characteristics:**
- Sub-linear scaling due to cache reuse
- Base profiles cached early benefit later profiles
- Parallel generation possible (future enhancement)

## Error Handling

The generator provides detailed error types:

```rust
pub enum SnapshotError {
    /// Base profile not found
    BaseNotFound { url: String },
    
    /// Circular dependency in profile hierarchy
    CircularDependency { url: String },
    
    /// Invalid cardinality constraint
    InvalidCardinality { path: String, reason: String },
    
    /// Invalid type constraint
    InvalidTypeConstraint { path: String, reason: String },
    
    /// Invalid binding strength
    InvalidBinding { path: String, reason: String },
    
    /// Missing required element
    MissingElement { path: String },
    
    /// Invalid element path
    InvalidPath { path: String, reason: String },
    
    /// Serialization error
    SerializationError { message: String },
}
```

### Error Recovery Strategies

**1. Missing Base Profiles:**
```rust
match generator.generate_snapshot(&profile) {
    Err(SnapshotError::BaseNotFound { url }) => {
        eprintln!("Base not found: {}", url);
        eprintln!("Try loading the base profile first:");
        eprintln!("  loader.load_from_package(...)?;");
    }
    _ => {}
}
```

**2. Circular Dependencies:**
```rust
// The generator automatically detects and reports cycles
match generator.generate_snapshot(&profile) {
    Err(SnapshotError::CircularDependency { url }) => {
        eprintln!("Circular dependency involving: {}", url);
        // Review profile hierarchy for cycles
    }
    _ => {}
}
```

**3. Validation Errors:**
```rust
// Detailed validation errors help fix profile issues
match generator.generate_snapshot(&profile) {
    Err(SnapshotError::InvalidCardinality { path, reason }) => {
        eprintln!("Fix cardinality at {}: {}", path, reason);
        // Adjust min/max in differential
    }
    _ => {}
}
```

## Testing

Comprehensive test suite included:

```bash
# Run all snapshot tests
cargo test -p rh-foundation snapshot

# Run specific test categories
cargo test -p rh-foundation snapshot::generator  # Core generation
cargo test -p rh-foundation snapshot::merger     # Element merging
cargo test -p rh-foundation snapshot::path       # Path utilities
```

### Test Coverage

**Core Functionality:**
- ✅ Basic snapshot generation
- ✅ Multi-level inheritance
- ✅ Cardinality constraints
- ✅ Type narrowing
- ✅ Binding strength
- ✅ Fixed values
- ✅ Constraint merging

**Advanced Features:**
- ✅ Slicing and reslicing
- ✅ Extension handling
- ✅ Choice type resolution
- ✅ Circular dependency detection
- ✅ Cache invalidation
- ✅ Concurrent access

**Edge Cases:**
- ✅ Empty differentials
- ✅ Missing base profiles
- ✅ Invalid constraints
- ✅ Malformed paths
- ✅ Duplicate elements

## Integration Examples

### With FHIR Validator

```rust
use rh_foundation::snapshot::SnapshotGenerator;
use rh_validator::FhirValidator;

fn validate_with_custom_profile(resource: &serde_json::Value) -> anyhow::Result<()> {
    // Generate snapshot for custom profile
    let mut generator = SnapshotGenerator::new();
    let profile = load_custom_profile()?;
    let with_snapshot = generator.generate_snapshot(&profile)?;
    
    // Use in validator
    let validator = FhirValidator::with_profile(with_snapshot)?;
    let result = validator.validate(resource)?;
    
    if !result.is_valid() {
        for issue in result.issues {
            eprintln!("{}", issue.message);
        }
    }
    
    Ok(())
}
```

### With Code Generator

```rust
use rh_foundation::snapshot::SnapshotGenerator;
use rh_codegen::CodeGenerator;

fn generate_rust_types(profile_url: &str) -> anyhow::Result<()> {
    // Generate snapshot first
    let mut generator = SnapshotGenerator::new();
    let profile = load_profile(profile_url)?;
    let with_snapshot = generator.generate_snapshot(&profile)?;
    
    // Generate Rust code from complete snapshot
    let codegen = CodeGenerator::new();
    let rust_code = codegen.generate_from_snapshot(&with_snapshot)?;
    
    std::fs::write("generated.rs", rust_code)?;
    
    Ok(())
}
```

## Troubleshooting

### Snapshot Generation Fails

**Symptom:** Generation fails with base not found

**Solution:**
```rust
// Ensure all base profiles are loaded first
loader.load_from_package("hl7.fhir.r4.core", "4.0.1")?;
loader.load_from_package("hl7.fhir.us.core", "5.0.1")?;

// Then generate snapshot
generator.generate_snapshot(&profile)?;
```

### Slow Generation

**Symptom:** Generation takes longer than expected

**Solution:**
```rust
// Increase cache size for better reuse
let mut generator = SnapshotGenerator::with_cache_size(200);

// Pre-load common base profiles
for base_url in common_bases {
    generator.load_and_cache(base_url)?;
}
```

### High Memory Usage

**Symptom:** Memory usage grows with number of profiles

**Solution:**
```rust
// Reduce cache size
let mut generator = SnapshotGenerator::with_cache_size(20);

// Or clear cache periodically
generator.clear_cache();
```

### Circular Dependency Errors

**Symptom:** CircularDependency error during generation

**Solution:**
- Review profile hierarchy for cycles
- Check if profile incorrectly references itself
- Verify base profile URLs are correct
- Use visualization tools to map dependencies

## Future Enhancements

Planned improvements:

- [ ] Parallel snapshot generation for multiple profiles
- [ ] Incremental snapshot updates (only changed elements)
- [ ] Snapshot diffing and comparison utilities
- [ ] Profile hierarchy visualization
- [ ] Snapshot validation against FHIR specification
- [ ] Snapshot serialization to binary format
- [ ] Streaming generation for very large profiles

## References

- [FHIR StructureDefinition](http://hl7.org/fhir/structuredefinition.html)
- [FHIR Profiling](http://hl7.org/fhir/profiling.html)
- [Snapshot Generation Algorithm](http://hl7.org/fhir/structuredefinition.html#snapshot)
- [Differential vs Snapshot](http://hl7.org/fhir/diff.html)