codex-memory 3.0.15

# Build System Architecture - Feature-Complete Pipeline

## Overview

The Codex build system supports conditional compilation through feature flags, enabling both minimal deployments and full-featured installations. This document describes the build pipeline architecture, feature flag usage, and deployment strategies.

## Feature Flag Architecture

### Available Features

#### `codex-dreams` (Primary Feature)
- **Purpose**: Enables advanced cognitive memory features and LLM integration
- **Dependencies**: 
  - `ollama-rs v0.3+`: LLM communication and model management
  - `tokenizers v0.20+`: Text tokenization for embeddings
  - `candle-core v0.9+`: ML tensor operations and computations
  - `candle-transformers v0.9+`: Transformer model implementations
- **Build Impact**: ~1MB additional binary size, 70+ additional dependencies
- **Use Cases**: Full cognitive memory system, insights generation, research applications

#### Default Configuration
- **Features**: None (minimal build)
- **Dependencies**: Core storage, database, MCP server only
- **Build Impact**: 7.3MB binary, minimal dependency footprint
- **Use Cases**: Simple text storage, resource-constrained environments

## Build Pipeline Commands

### Basic Commands

```bash
# Minimal build (no features)
cargo build --release

# Full-featured build (recommended for local development)
cargo build --release --features codex-dreams

# Development build with features
cargo build --features codex-dreams

# Installation with all features (REQUIRED for documented functionality)
cargo install --path . --features codex-dreams --force
```

### Advanced Build Operations

```bash
# Check build compatibility
cargo check --features codex-dreams

# Build both configurations for CI/CD
cargo build --release                     # Basic: 7.3MB
cargo build --release --features codex-dreams  # Full: 8.3MB

# Test feature dependency resolution
cargo tree --features codex-dreams | grep -E "(ollama-rs|tokenizers|candle)"

# Verify no feature conflicts
cargo check --no-default-features
```

## Build Scripts

### Local Development: `scripts/build.sh`

Comprehensive build script supporting various configurations:

```bash
# Basic build
./scripts/build.sh

# Full-featured build with verification
ENABLE_CODEX_DREAMS=true VERIFY_BUILD=true ./scripts/build.sh

# Debug build with local installation
BUILD_MODE=debug ENABLE_CODEX_DREAMS=true INSTALL_LOCAL=true ./scripts/build.sh

# Clean build with custom features
CLEAN_FIRST=true FEATURES="codex-dreams" ./scripts/build.sh
```

**Script Features**:
- Automatic prerequisite checking
- Build verification tests
- Feature dependency validation
- Local installation support
- Build artifact analysis
- Configurable build modes

### CI/CD Integration: `scripts/ci-build.sh`

Production-ready CI/CD pipeline supporting multiple build matrices:

```bash
# Full CI pipeline (builds both configurations)
./scripts/ci-build.sh

# Feature-only CI (faster for feature branch testing)
CI_MODE=features ./scripts/ci-build.sh

# Basic-only CI (minimal for simple deployments)
CI_MODE=basic ./scripts/ci-build.sh
```

**CI/CD Features**:
- Multi-configuration build matrix
- Code quality enforcement (fmt, clippy)
- Security audit integration
- Artifact packaging with metadata
- Test matrix execution
- Environment setup automation

## Build Verification

### Automated Testing

The build system includes comprehensive verification tests in `tests/build_system_test.rs`:

1. **Build Success Tests**
   - Basic build compilation
   - Feature-enabled build compilation
   - No-default-features build

2. **Feature Verification Tests**
   - Cargo.toml feature section validation
   - Optional dependency marking verification
   - Feature-to-dependency mapping validation

3. **Artifact Verification Tests**
   - Binary size difference validation
   - Dependency tree inclusion verification
   - Installation process testing

4. **Consistency Tests**
   - Binary naming consistency
   - Feature dependency version compatibility

### Build Targets and Sizes

| Configuration | Binary Size | Dependencies | Use Case |
|--------------|-------------|--------------|----------|
| Basic | 7.3MB | 47 core | Simple storage, lightweight deployment |
| codex-dreams | 8.3MB | 120+ total | Full cognitive features, research applications |

### Performance Benchmarks

- **Build Time (Basic)**: ~10s on modern hardware
- **Build Time (Features)**: ~40s on modern hardware (due to ML dependencies)
- **Dependency Resolution**: <2s for cached builds
- **Test Execution**: ~100s for full build verification suite

## Deployment Strategies

### Production Deployment

1. **Minimal Deployment** (Basic):
   ```bash
   cargo build --release
   # Deploy 7.3MB binary with core functionality only
   ```

2. **Full Deployment** (Recommended):
   ```bash
   cargo build --release --features codex-dreams
   # Deploy 8.3MB binary with complete cognitive features
   ```

### Development Environment

Always use full features for development consistency:

```bash
# Development setup
cargo build --features codex-dreams
cargo install --path . --features codex-dreams --force

# Verify installation
codex --version
echo '{"jsonrpc":"2.0","method":"tools/list","id":1}' | codex mcp-stdio
```

### Docker Deployment

```dockerfile
# Multi-stage build supporting feature flags
FROM rust:1.70 as builder
WORKDIR /app
COPY . .

# Build with or without features based on build argument
ARG ENABLE_FEATURES=false
RUN if [ "$ENABLE_FEATURES" = "true" ]; then \
        cargo build --release --features codex-dreams; \
    else \
        cargo build --release; \
    fi

FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y ca-certificates
COPY --from=builder /app/target/release/codex /usr/local/bin/
CMD ["codex"]
```

## Troubleshooting

### Common Build Issues

#### Dependency Version Conflicts
```bash
# Update dependency versions
cargo update

# Check for version conflicts
cargo tree --duplicates

# Clean and rebuild
cargo clean && cargo build --features codex-dreams
```

#### Feature Dependencies Not Found
```bash
# Verify feature definition
grep -A 5 "\[features\]" Cargo.toml

# Check optional dependencies
grep "optional = true" Cargo.toml

# Test dependency resolution
cargo tree --features codex-dreams
```

#### Build Performance Issues
```bash
# Use incremental compilation
export CARGO_INCREMENTAL=1

# Parallel builds
cargo build -j 4

# Target specific features only
cargo build --features codex-dreams --no-default-features
```

### Build Environment Validation

```bash
# Check Rust version compatibility
rustc --version

# Verify toolchain components
rustup component list --installed

# Check available features
cargo metadata --format-version 1 | jq '.packages[0].features'
```

## Integration with MCP System

### Feature-Aware MCP Tools

The build system ensures that MCP tools are available based on enabled features:

- **Basic Build**: 5 core MCP tools (store_memory, get_memory, delete_memory, get_statistics, store_file)
- **Feature Build**: 7+ MCP tools (adds generate_insights, show_insights, search_insights, export_insights)

### Runtime Feature Detection

```rust
// Feature-gated code compilation
#[cfg(feature = "codex-dreams")]
mod insights {
    // Insights generation code only compiled with feature
}

#[cfg(not(feature = "codex-dreams"))]
mod insights {
    // Stub implementations for basic build
}
```

## Performance Impact Analysis

### Build Performance
- **Basic Build**: Fast compilation, minimal dependencies
- **Feature Build**: Longer compilation due to ML dependencies, but comprehensive functionality

### Runtime Performance  
- **Basic Build**: Minimal memory footprint, fast startup
- **Feature Build**: Higher memory usage due to ML models, full cognitive capabilities

### Deployment Size
- **Basic Build**: Optimal for containerized deployments
- **Feature Build**: Recommended for development and research environments

## Maintenance and Updates

### Dependency Management
- Use `cargo outdated` to check for dependency updates
- Update feature dependencies together to maintain compatibility
- Test both build configurations after dependency updates

### Version Compatibility
- Maintain semantic versioning for feature API stability
- Document breaking changes in feature interfaces
- Provide migration guides for major version updates

### Security Considerations
- Run `cargo audit` on both build configurations
- Monitor feature dependencies for security advisories
- Update dependencies promptly for security patches

## Future Enhancements

### Planned Features
- Additional ML model backends (OpenAI, Anthropic)
- GPU acceleration support
- Custom embedding model support
- Advanced cognitive pattern recognition

### Build System Improvements
- Cross-compilation support for multiple architectures
- Optimized Docker build caching
- Automated dependency update workflows
- Performance regression testing

---

**Created by**: BUILD-SYSTEM-EXPERT (CODEX-BUILD-001)  
**Last Updated**: 2025-08-31  
**Version**: 2.1.0  
**Related Stories**: CODEX-FEAT-001 (Feature Flag Implementation)