opencrates 3.0.1

# OpenCrates

**An intelligent Rust crate generator and registry management system powered by AI**

[![Crates.io](https://img.shields.io/crates/v/opencrates.svg)](https://crates.io/crates/opencrates)
[![Documentation](https://docs.rs/opencrates/badge.svg)](https://docs.rs/opencrates)
[![License](https://img.shields.io/badge/license-MIT%20OR%20Apache--2.0-blue.svg)](LICENSE)
[![CI](https://github.com/your-username/opencrates/workflows/CI/badge.svg)](https://github.com/your-username/opencrates/actions)

## Key Features

- **AI-Powered Code Generation**: Advanced LLM integration for intelligent crate creation
- **Multi-Provider Support**: OpenAI, Codex, and extensible provider architecture
- **Enterprise-Grade**: Production-ready with comprehensive monitoring and security
- **Template System**: Sophisticated templating with reasoning patterns
- **Real-time Analytics**: Built-in metrics, health checks, and observability
- **Docker Ready**: Complete containerization with multi-stage builds

### FastAPI Server & Endpoints

- **RESTful API**: Complete FastAPI-style server with automatic OpenAPI documentation
- **WebSocket Support**: Real-time communication for streaming operations
- **Authentication**: JWT-based auth with role-based access control
- **Rate Limiting**: Configurable rate limiting and request throttling
- **CORS Support**: Cross-origin resource sharing for web applications
- **Health Checks**: Comprehensive health monitoring endpoints

### Enterprise Architecture

- **Microservices Ready**: Modular design for distributed deployments
- **Database Integration**: SQLite, PostgreSQL, and Redis support
- **Caching Layer**: Multi-tier caching with Redis backend
- **Message Queues**: Async processing with queue management
- **Monitoring Stack**: Prometheus metrics and structured logging
- **Security**: Input validation, sanitization, and security headers

### Advanced Project Analysis

- **Dependency Analysis**: Smart dependency resolution and optimization
- **Code Quality Metrics**: Automated code quality assessment
- **Performance Profiling**: Built-in benchmarking and performance analysis
- **Security Scanning**: Vulnerability detection and security auditing

### Development Tools

- **CLI Interface**: Rich command-line tools for developers
- **IDE Integration**: Language server protocol support
- **Testing Framework**: Comprehensive testing utilities
- **Documentation Generation**: Automated docs with examples

## Quick Start

### Installation

```bash
# Install from crates.io
cargo install opencrates

# Or add as dependency
cargo add opencrates
```

### Docker Setup

```bash
# Pull and run
docker pull opencrates/opencrates:latest
docker run -p 8080:8080 opencrates/opencrates

# Or build locally
docker build -t opencrates .
docker-compose up
```

### Basic Usage

```rust
use opencrates::{OpenCrates, providers::OpenAIProvider};
use anyhow::Result;

#[tokio::main]
async fn main() -> Result<()> {
    // Initialize OpenCrates
    let opencrates = OpenCrates::new().await?;
    
    // Generate a new crate
    opencrates.generate_crate(
        "my-awesome-crate",
        "A fantastic Rust library",
        vec!["async".to_string(), "serde".to_string()],
        "./output".into()
    ).await?;
    
    // Analyze existing project
    let analysis = opencrates.analyze_crate("./my-project").await?;
    println!("Project analysis: {:#?}", analysis);
    
    Ok(())
}
```

### CLI Usage

```bash
# Generate a new crate
opencrates generate --name "my-crate" --description "A sample crate" --output ./output

# Analyze existing crate
opencrates analyze ./existing-project

# Start the server
opencrates server --host 0.0.0.0 --port 8080

# Health check
opencrates health

# Interactive mode
opencrates interactive
```

## Configuration

OpenCrates supports comprehensive configuration through multiple sources:

### Environment Variables

```bash
export OPENCRATES_ENV=production
export OPENAI_API_KEY=your_api_key_here
export OPENCRATES_DATABASE_URL=sqlite:./opencrates.db
export OPENCRATES_REDIS_URL=redis://localhost:6379
export OPENCRATES_LOG_LEVEL=info
```

### Configuration File (`opencrates.toml`)

```toml
[server]
host = "127.0.0.1"
port = 8080
workers = 4

[database]
url = "sqlite:./opencrates.db"
max_connections = 10

[ai]
provider = "openai"
model = "gpt-4"
temperature = 0.7
max_tokens = 4000

[cache]
backend = "redis"
ttl = 3600
max_size = 1000

[monitoring]
enabled = true
health_check_interval = 30
enable_profiling = false

[security]
enable_auth = true
enable_rate_limiting = true
jwt_secret = "your-secret-key"

[features]
enable_web_ui = true
enable_api = true
enable_cli = true
```

### Runtime Configuration

```rust
use opencrates::utils::config::OpenCratesConfig;

let config = OpenCratesConfig {
    server: ServerConfig {
        host: "0.0.0.0".to_string(),
        port: 3000,
        workers: 8,
        ..Default::default()
    },
    ai: AiConfig {
        provider: "openai".to_string(),
        model: "gpt-4-turbo".to_string(),
        temperature: 0.8,
        ..Default::default()
    },
    ..Default::default()
};

let opencrates = OpenCrates::new_with_config(config).await?;
```

## Development & Testing

### Prerequisites

```bash
# Install Rust (latest stable)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Install required tools
cargo install cargo-audit cargo-tarpaulin

# For full development
docker-compose up -d redis postgres
```

### Building from Source

```bash
# Clone the repository
git clone https://github.com/your-username/opencrates.git
cd opencrates

# Build in development mode
cargo build

# Build optimized release
cargo build --release

# Run with all features
cargo run --all-features -- --help
```

### Running Tests

```bash
# Run all tests
cargo test --all-features

# Run specific test suites
cargo test --test basic_functionality
cargo test --test integration_tests
cargo test --test codex_integration

# Run with coverage
cargo tarpaulin --all-features --out html

# Benchmark tests
cargo bench

# Security audit
cargo audit
```

### Development Scripts

```bash
# Comprehensive build and test
./build_and_test.sh

# Quick test run
./test_all.sh

# Format and lint
cargo fmt
cargo clippy --all-features -- -D warnings

# Generate documentation
cargo doc --no-deps --all-features --open
```

## Architecture

### Core Components

```
OpenCrates/
├── Core Engine           # Central orchestration
│   ├── Template Manager  # Code generation templates
│   ├── Provider Registry # AI provider management
│   └── Configuration     # System configuration
├── Providers/            # AI service integrations
│   ├── OpenAI           # GPT-4, GPT-3.5-turbo
│   ├── Codex            # GitHub Copilot integration
│   └── Custom          # Extensible provider system
├── Server/              # FastAPI-style HTTP server
│   ├── REST API         # RESTful endpoints
│   ├── WebSocket        # Real-time communication
│   └── GraphQL          # Advanced querying
├── CLI/                 # Command-line interface
│   ├── Interactive      # REPL-style interaction
│   ├── Batch           # Script automation
│   └── TUI             # Terminal user interface
├── Storage/            # Data persistence
│   ├── SQLite          # Embedded database
│   ├── PostgreSQL      # Production database
│   └── Redis           # Caching and sessions
└── Monitoring/         # Observability stack
    ├── Metrics         # Prometheus integration
    ├── Logging         # Structured logging
    └── Tracing         # Distributed tracing
```

### Request Flow

```
[Client Request] 
    ↓
[Rate Limiting & Auth]
    ↓
[Request Validation]
    ↓
[Provider Selection]
    ↓
[AI Processing]
    ↓
[Template Application]
    ↓
[Code Generation]
    ↓
[Quality Validation]
    ↓
[Response Formation]
    ↓
[Client Response]
```

### Data Models

```rust
// Core entities
pub struct CrateSpec {
    pub name: String,
    pub description: String,
    pub version: String,
    pub dependencies: HashMap<String, String>,
    pub features: Vec<String>,
    pub metadata: CrateMetadata,
}

pub struct GenerationRequest {
    pub prompt: String,
    pub context: String,
    pub provider: String,
    pub model: String,
    pub parameters: GenerationParameters,
}

pub struct AnalysisResult {
    pub metrics: CodeMetrics,
    pub dependencies: DependencyGraph,
    pub suggestions: Vec<Improvement>,
    pub security: SecurityReport,
}
```

## Monitoring & Observability

### Metrics Collection

```bash
# Prometheus metrics endpoint
curl http://localhost:8080/metrics

# Health check endpoint
curl http://localhost:8080/health

# System status
curl http://localhost:8080/status
```

### Monitoring Stack

- **Prometheus**: Metrics collection and alerting
- **Grafana**: Dashboards and visualization
- **Jaeger**: Distributed tracing
- **ELK Stack**: Log aggregation and analysis

### Key Metrics

- Request latency and throughput
- AI provider response times
- Code generation success rates
- Resource utilization
- Error rates and types
- Cache hit ratios

### Logging

```rust
use tracing::{info, warn, error};

// Structured logging with context
info!(
    user_id = %user.id,
    crate_name = %spec.name,
    generation_time_ms = generation_time.as_millis(),
    "Crate generation completed successfully"
);
```

## Security

### Authentication & Authorization

```rust
// JWT-based authentication
#[derive(Serialize, Deserialize)]
struct Claims {
    sub: String,
    role: UserRole,
    exp: usize,
}

// Role-based access control
#[derive(Serialize, Deserialize)]
enum UserRole {
    Admin,
    Developer,
    ReadOnly,
}
```

### Security Features

- **Input Validation**: Comprehensive input sanitization
- **Rate Limiting**: Configurable request throttling
- **API Key Management**: Secure API key handling
- **Audit Logging**: Complete audit trail
- **Dependency Scanning**: Automated vulnerability detection
- **Secure Defaults**: Security-first configuration

### Best Practices

1. **Environment Variables**: Store secrets in environment variables
2. **HTTPS Only**: Force HTTPS in production
3. **Regular Updates**: Keep dependencies updated
4. **Monitoring**: Monitor for suspicious activity
5. **Backup**: Regular database backups
6. **Access Control**: Principle of least privilege

## API Reference

### Core Endpoints

```http
# Crate generation
POST /api/v1/generate
Content-Type: application/json

{
  "name": "my-crate",
  "description": "Description here",
  "features": ["async", "serde"],
  "template": "library"
}

# Project analysis
POST /api/v1/analyze
Content-Type: multipart/form-data

# Health check
GET /health

# Metrics
GET /metrics

# OpenAPI specification
GET /docs
```

### WebSocket Endpoints

```javascript
// Real-time generation updates
const ws = new WebSocket('ws://localhost:8080/ws/generate');

ws.onmessage = (event) => {
    const update = JSON.parse(event.data);
    console.log('Generation progress:', update.progress);
};
```

## Contributing

We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.

### Development Setup

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Ensure all tests pass
6. Submit a pull request

### Code Style

- Follow Rust standard formatting (`cargo fmt`)
- Pass all lints (`cargo clippy`)
- Add documentation for public APIs
- Include tests for new functionality

## License

Licensed under either of:

- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE))
- MIT License ([LICENSE-MIT](LICENSE-MIT))

at your option.

## Changelog

### Version 3.0.0 (Released!)

**Major Release - Complete Rewrite**

- **New Architecture**: Completely redesigned modular architecture
- **AI Integration**: Advanced LLM provider system with OpenAI and Codex
- **FastAPI Server**: Production-ready HTTP server with OpenAPI docs
- **Enterprise Features**: Monitoring, security, and scalability improvements
- **Docker Support**: Complete containerization with multi-stage builds
- **Database Integration**: SQLite, PostgreSQL, and Redis support
- **Comprehensive Testing**: 67+ tests covering all functionality
- **Documentation**: Complete API documentation and examples

### Version 2.0.0
- Template system improvements
- Basic AI integration
- CLI enhancements

### Version 1.0.0
- Initial release
- Basic crate generation
- Simple template system

---

**OpenCrates** - Intelligent Rust Development, Powered by AI