ggen 4.0.0

# Production-Grade Packs Domain Implementation

## Overview

This document describes the complete, production-ready implementation of the packs domain layer in `ggen-domain`. The packs system enables curated package collections for specific workflows with real project installation capabilities.

## Architecture

### Module Structure

```
crates/ggen-domain/src/packs/
├── mod.rs                  # Public API and exports
├── types.rs                # Core data types (Pack, PackTemplate, PackDependency)
├── repository.rs           # Storage abstraction (PackRepository trait + FileSystem impl)
├── installer.rs            # Production-grade pack installer with marketplace integration
├── install.rs              # High-level convenience API
├── composer.rs             # Multi-pack composition engine
├── compose.rs              # High-level composition API
├── dependency_graph.rs     # Dependency resolution & topological sorting
├── generator.rs            # Template generation from packs
├── validator.rs            # Pack validation
├── score.rs                # Pack quality scoring
└── metadata.rs             # Pack metadata loading
```

### Design Patterns

1. **Repository Pattern**: Abstract storage layer supporting multiple backends
2. **Strategy Pattern**: Composition strategies (Merge, Layer, Custom)
3. **Builder Pattern**: Fluent options for installation and composition
4. **Facade Pattern**: Simple high-level API wrapping complex subsystems

## Core Components

### 1. Repository Layer (`repository.rs`)

**Purpose**: Abstraction over pack storage (filesystem, remote, database, etc.)

**Key Features**:
- `PackRepository` trait for pluggable backends
- `FileSystemRepository` implementation with automatic discovery
- Security: Path traversal prevention, ID validation
- Async operations using tokio
- Safe file operations with error handling

**Example**:
```rust
use ggen_domain::packs::{PackRepository, FileSystemRepository};

// Auto-discover packs directory
let repo = FileSystemRepository::discover()?;

// Load a pack
let pack = repo.load("web-api-stack").await?;

// List packs by category
let web_packs = repo.list(Some("web")).await?;
```

### 2. Dependency Graph (`dependency_graph.rs`)

**Purpose**: Resolve pack dependencies and determine installation order

**Key Features**:
- Circular dependency detection using DFS
- Topological sorting for correct install order
- Transitive dependency resolution
- Graph analysis (node count, edge count, etc.)

**Example**:
```rust
use ggen_domain::packs::DependencyGraph;

let graph = DependencyGraph::from_packs(&packs)?;

// Detect cycles (throws error if found)
graph.detect_cycles()?;

// Get install order
let install_order = graph.topological_sort()?;
// Returns: ["base-pack", "web-framework", "api-pack"]
```

**Algorithm**: Khan's algorithm for topological sorting with cycle detection

### 3. Pack Installer (`installer.rs`)

**Purpose**: Production-grade pack installation with marketplace integration

**Key Features**:
- **Real Package Installation**: Calls `marketplace::execute_install` for each package
- **Dependency Resolution**: Recursively resolves pack dependencies
- **Conflict Detection**: Identifies packages provided by multiple packs
- **Topological Ordering**: Installs packs in correct dependency order
- **Rollback Support**: Fails atomically unless `force` is set
- **Performance Tracking**: Reports duration and detailed metrics
- **Dry Run Mode**: Simulates installation without changes

**Example**:
```rust
use ggen_domain::packs::installer::{PackInstaller, InstallOptions};

let installer = PackInstaller::with_default_repo()?;

let options = InstallOptions {
    target_dir: Some("./my-project".into()),
    force: false,
    dry_run: false,
    skip_dependencies: false,
};

let report = installer.install("web-api-stack", &options).await?;

println!("{}", report.detailed_report());
// Shows:
// - Packages installed
// - Templates available
// - Dependencies resolved
// - Installation order
// - Conflicts (if any)
// - Duration
```

**Integration**:
```rust
// Calls marketplace domain for each package
marketplace::execute_install(&marketplace::InstallOptions {
    package_name: "express-api".to_string(),
    version: Some("1.0.0".to_string()),
    target_path: Some(install_dir),
    force: false,
    with_dependencies: true,
    dry_run: false,
}).await?;
```

### 4. Pack Composer (`composer.rs`)

**Purpose**: Compose multiple packs into cohesive projects

**Key Features**:
- **Multi-Pack Composition**: Merge 10+ packs efficiently
- **Conflict Resolution**: Detects and reports conflicts
- **Composition Strategies**:
  - **Merge**: Combine all packs (deduplicate packages/templates)
  - **Layer**: Apply packs in sequence (later overrides earlier)
  - **Custom**: User-defined composition rules
- **Composition Plan**: Shows what will be installed and in what order
- **Safe Operations**: Validates before executing

**Example**:
```rust
use ggen_domain::packs::composer::{PackComposer, CompositionOptions};
use ggen_domain::packs::types::CompositionStrategy;

let composer = PackComposer::with_default_repo()?;

let options = CompositionOptions {
    strategy: CompositionStrategy::Merge,
    output_dir: Some("./fullstack-app".into()),
    force_composition: false,
    dry_run: false,
};

let result = composer.compose(
    &["web-api-pack", "database-pack", "auth-pack"],
    "fullstack-app",
    &options
).await?;

println!("Composed {} packs", result.packs_composed.len());
println!("Total packages: {}", result.composed_pack.packages.len());
println!("Composition order: {:?}", result.composition_order);
```

**Conflict Detection**:
```
Package 'express' provided by: web-api-pack, microservices-pack
Template 'main.rs' provided by: web-api-pack, cli-pack
```

### 5. Data Types (`types.rs`)

**Core Types**:

```rust
/// Pack definition
pub struct Pack {
    pub id: String,
    pub name: String,
    pub version: String,
    pub description: String,
    pub category: String,

    // Content
    pub packages: Vec<String>,           // Packages to install
    pub templates: Vec<PackTemplate>,    // Templates for generation
    pub sparql_queries: HashMap<String, String>,  // Semantic queries

    // Dependencies
    pub dependencies: Vec<PackDependency>,

    // Metadata
    pub tags: Vec<String>,
    pub keywords: Vec<String>,
    pub production_ready: bool,
    pub metadata: PackMetadata,
}

/// Pack template
pub struct PackTemplate {
    pub name: String,
    pub path: String,
    pub description: String,
    pub variables: Vec<String>,
}

/// Pack dependency
pub struct PackDependency {
    pub pack_id: String,
    pub version: String,
    pub optional: bool,
}

/// Composition strategies
pub enum CompositionStrategy {
    Merge,
    Layer,
    Custom(HashMap<String, serde_json::Value>),
}
```

## Integration Points

### Marketplace Integration

The packs domain integrates directly with the marketplace domain for real package installation:

```rust
// In installer.rs
use crate::marketplace;

async fn install_package(&self, package_name: &str, target_dir: &PathBuf) -> Result<()> {
    let install_options = marketplace::InstallOptions {
        package_name: name,
        version,
        target_path: Some(target_dir.clone()),
        force: false,
        with_dependencies: true,
        dry_run: false,
    };

    marketplace::execute_install(&install_options).await?;
    Ok(())
}
```

### Template Engine Integration

```rust
// Future integration with ggen-core template engine
use ggen_core::templates::{FileTreeGenerator, TemplateContext};

let generator = FileTreeGenerator::new();
let ctx = TemplateContext::new()
    .with_var("project_name", project_name)
    .with_var("author", author);

generator.generate(template_path, output_dir, &ctx)?;
```

### RDF/SPARQL Integration

```rust
// Future integration with ggen-core graph
use ggen_core::Graph;

let graph = Graph::new()?;
graph.insert_turtle(&pack.rdf_metadata)?;

// Execute SPARQL query
let results = graph.query(&pack.sparql_queries["list_dependencies"])?;
```

## Performance Characteristics

### Time Complexity

| Operation | Complexity | Notes |
|-----------|------------|-------|
| Load pack | O(1) | File I/O |
| List packs | O(n) | n = number of pack files |
| Dependency resolution | O(V + E) | V = packs, E = dependencies |
| Topological sort | O(V + E) | Khan's algorithm |
| Cycle detection | O(V + E) | DFS-based |
| Conflict detection | O(P × K) | P = packs, K = avg packages per pack |
| Install pack | O(V × I) | V = packs, I = install time per package |

### Performance Targets

- **Pack loading**: <50ms per pack
- **Dependency resolution**: <100ms for 20 packs
- **Topological sort**: <50ms for 50 packs
- **Conflict detection**: <100ms for 10 packs with 100 packages each
- **Pack installation**: <500ms overhead (actual install time depends on packages)
- **Multi-pack composition**: <500ms for 10 packs

### Memory Usage

- **Pack metadata**: ~5KB per pack in memory
- **Dependency graph**: O(V + E) space
- **Installation report**: ~2KB per report

## Error Handling

### Error Types

```rust
// Dependency errors
"Circular dependency detected: pack-a -> pack-b -> pack-c -> pack-a"

// Conflict errors
"Package 'express' provided by multiple packs: web-api-pack, microservices-pack"

// Validation errors
"Pack name cannot be empty"
"Invalid pack ID: must not contain path separators"

// Installation errors
"Failed to install package 'express@4.18.0': network error"

// Repository errors
"Pack 'nonexistent-pack' not found at /path/to/marketplace/packs/"
```

### Error Recovery

- **Atomic operations**: Installations fail atomically unless `force=true`
- **Rollback support**: Failed installations don't leave partial state
- **Graceful degradation**: Optional dependencies are skipped on failure
- **Detailed error context**: Every error includes actionable information

## Security Features

### Input Validation

```rust
// Path traversal prevention
fn validate_pack_id(&self, pack_id: &str) -> Result<()> {
    if pack_id.contains("..") || pack_id.contains('/') || pack_id.contains('\\') {
        return Err(Error::new("Invalid pack ID: path traversal attempt"));
    }
    Ok(())
}
```

### Safe File Operations

- No path traversal sequences allowed
- All pack IDs validated before file access
- Proper error handling for file I/O
- Secure temp directory usage

## Testing

### Unit Tests

Each module includes comprehensive unit tests:

```rust
#[cfg(test)]
mod tests {
    #[test]
    fn test_dependency_graph_detects_cycles() {
        // A -> B -> C -> A
        let result = DependencyGraph::from_packs(&packs_with_cycle);
        assert!(result.is_err());
    }

    #[test]
    fn test_topological_sort_orders_correctly() {
        // Verify dependencies come before dependents
        let sorted = graph.topological_sort()?;
        assert!(sorted.index_of("base") < sorted.index_of("framework"));
    }

    #[tokio::test]
    async fn test_installer_resolves_dependencies() {
        let report = installer.install("top-level-pack", &options).await?;
        assert!(report.dependencies_resolved.contains("dependency-pack"));
    }
}
```

### Integration Tests

```rust
#[tokio::test]
async fn test_full_pack_installation_workflow() {
    // 1. Create temporary repository
    let temp_repo = create_temp_packs();

    // 2. Create installer
    let installer = PackInstaller::new(Box::new(temp_repo));

    // 3. Install pack with dependencies
    let report = installer.install("test-pack", &options).await?;

    // 4. Verify all packages installed
    assert_eq!(report.packages_installed.len(), 5);

    // 5. Verify install order is correct
    assert_eq!(report.install_order, vec!["base", "framework", "test-pack"]);
}
```

## Usage Patterns

### Pattern 1: Simple Pack Installation

```rust
use ggen_domain::packs::{install_pack, InstallInput};

let input = InstallInput {
    pack_id: "web-api-stack".to_string(),
    target_dir: Some("./my-api".into()),
    force: false,
    dry_run: false,
};

let output = install_pack(&input).await?;
println!("Installed {} packages", output.packages_installed.len());
```

### Pattern 2: Multi-Pack Composition

```rust
use ggen_domain::packs::{compose_packs, ComposePacksInput};

let input = ComposePacksInput {
    pack_ids: vec![
        "backend-api".to_string(),
        "frontend-react".to_string(),
        "database-postgres".to_string(),
    ],
    project_name: "fullstack-app".to_string(),
    output_dir: None,
    strategy: Default::default(),
};

let output = compose_packs(&input).await?;
```

### Pattern 3: Custom Repository

```rust
use ggen_domain::packs::repository::{PackRepository, FileSystemRepository};
use ggen_domain::packs::installer::PackInstaller;

// Custom repository location
let repo = FileSystemRepository::new("/custom/packs/path");
let installer = PackInstaller::new(Box::new(repo));

let report = installer.install("my-pack", &options).await?;
```

### Pattern 4: Dry Run Analysis

```rust
let options = InstallOptions {
    dry_run: true,
    ..Default::default()
};

let report = installer.install("web-api-stack", &options).await?;

println!("Would install:");
println!("  Packs: {:?}", report.dependencies_resolved);
println!("  Order: {:?}", report.install_order);
println!("  Conflicts: {:?}", report.conflicts);
```

## Pack Manifest Format

### TOML Format

```toml
[pack]
name = "Web API Stack"
id = "web-api-stack"
version = "1.0.0"
description = "Complete stack for building REST APIs"
category = "web"
author = "Your Name"
license = "MIT"
production_ready = true

# Packages to install
packages = [
    "express@4.18.0",
    "typescript@5.0.0",
    "jest@29.0.0"
]

# Templates for generation
[[templates]]
name = "api-server"
path = "templates/api-server.ts.tmpl"
description = "Express API server with TypeScript"
variables = ["project_name", "port"]

[[templates]]
name = "config"
path = "templates/config.toml.tmpl"
description = "Application configuration"
variables = ["database_url", "api_key"]

# Dependencies on other packs
[[dependencies]]
pack_id = "base-nodejs"
version = "1.0.0"
optional = false

[[dependencies]]
pack_id = "testing-utils"
version = "1.0.0"
optional = true

# SPARQL queries for semantic operations
[sparql_queries]
list_dependencies = """
SELECT ?dep ?version WHERE {
    ?pack ggen:hasDependency ?dep .
    ?dep ggen:version ?version .
}
"""

# Metadata
tags = ["web", "api", "typescript"]
keywords = ["rest", "express", "nodejs"]

[metadata]
test_coverage = "90%"
rdf_ontology_size = "5KB"
sparql_templates = 3
code_examples = 5
documentation_files = 10
```

## Future Enhancements

### Phase 2 Features

1. **Remote Pack Repositories**
   - HTTP/HTTPS remote pack fetching
   - Pack versioning and updates
   - Pack registry integration

2. **Advanced Composition**
   - Custom composition rules engine
   - Template merging with conflict resolution
   - Variable inheritance between packs

3. **Template Generation**
   - Full integration with ggen-core template engine
   - SPARQL variable substitution
   - Multi-template project generation

4. **Caching**
   - Pack metadata caching
   - Dependency resolution caching
   - Template compilation caching

5. **Observability**
   - Installation progress tracking
   - Telemetry integration
   - Health checks

## Conclusion

The packs domain implementation provides a production-grade foundation for curated package collections. It includes:

✅ **Complete functionality**: Real package installation via marketplace
✅ **Robust dependency resolution**: Topological sort with cycle detection
✅ **Conflict detection**: Identifies problems before installation
✅ **Multi-pack composition**: Merge complex project stacks
✅ **Repository abstraction**: Pluggable storage backends
✅ **Performance**: Sub-second operations for typical workloads
✅ **Security**: Input validation and safe file operations
✅ **Error handling**: Comprehensive error reporting
✅ **Testing**: Unit and integration test coverage
✅ **Documentation**: Clear API docs and examples

The implementation seamlessly integrates with:
- `ggen-domain/marketplace` for package installation
- `ggen-core` templates for project generation
- `ggen-core` graph for SPARQL queries

All operations target <500ms performance with support for 10+ pack composition.