Module chunk_processor_adapters 

Chunk processor adapters for service integration

Chunk Processor Adapters

This module provides adapter implementations that bridge domain services with the chunk processing interface. These adapters enable domain services to be used as chunk processors in the file processing pipeline.

Overview

The chunk processor adapters provide:

• Service Integration: Bridge domain services with chunk processing
• Type Safety: Generic adapters with compile-time type checking
• Configuration: Flexible configuration for different service types
• Reusability: Reusable adapters for common service patterns
• Performance: Efficient adaptation with minimal overhead

Architecture

The adapters follow the Adapter pattern:

• Generic Design: Generic adapters that work with any service type
• Service Wrapping: Wrap domain services to implement ChunkProcessor
• Configuration-Driven: Behavior controlled through configuration
• Async Operations: Full async support for non-blocking operations

Adapter Types

Service Chunk Adapter

Generic adapter that can wrap any service:

• Compression Services: Adapt compression services for chunk processing
• Encryption Services: Adapt encryption services for chunk processing
• Custom Services: Adapt any domain service with appropriate interface

Specialized Adapters

• Compression Adapter: Specialized adapter for compression services
• Encryption Adapter: Specialized adapter for encryption services
• Validation Adapter: Specialized adapter for validation services

Usage Examples

Basic Service Adaptation

Compression Service Adaptation

Encryption Service Adaptation

Pipeline Integration

Adapter Configuration

Configuration Options

• modifies_data: Whether the adapter modifies chunk data

Performance Tuning

• Batch Processing: Process multiple chunks in batches
• Memory Management: Efficient memory usage and cleanup
• Async Operations: Non-blocking operations for better throughput

Performance Considerations

Adaptation Overhead

• Minimal Overhead: Adapters add minimal performance overhead
• Zero-Cost Abstractions: Generic design enables compiler optimizations
• Efficient Wrapping: Direct service calls without unnecessary indirection

Memory Usage

• Shared Services: Services are shared via Arc for memory efficiency
• Chunk Copying: Minimal chunk copying during adaptation
• Resource Cleanup: Automatic cleanup of adapter resources

Concurrency

• Thread Safety: All adapters are thread-safe
• Concurrent Processing: Support for concurrent chunk processing
• Lock-Free Operations: Lock-free operations where possible

Error Handling

Adapter Errors

• Service Errors: Proper propagation of service errors
• Configuration Errors: Validation of adapter configuration
• Processing Errors: Comprehensive error context and recovery

Error Recovery

• Graceful Degradation: Graceful handling of service failures
• Fallback Processing: Alternative processing strategies

Integration

The adapters integrate with:

• Domain Services: Bridge domain services with chunk processing
• File Processor: Used by file processor service for chunk processing
• Pipeline System: Integrate with pipeline processing workflow
• Configuration System: Support for runtime configuration

Thread Safety

All adapters are fully thread-safe:

• Shared Services: Services are shared safely via Arc
• Concurrent Access: Safe concurrent access to adapter methods
• Immutable Configuration: Configuration is immutable after creation

Future Enhancements

Planned enhancements include:

• Retry Logic: Configurable retry logic for transient failures
• Security Context Enforcement: Permission-based operation validation
• Dynamic Adaptation: Runtime adaptation of service behavior
• Metrics Integration: Built-in metrics collection and reporting
• Caching: Intelligent caching of processing results
• Load Balancing: Load balancing across multiple service instances

Structs

AdapterConfig

Configuration for service adapters (generic)

CompressionAdapterConfig

Configuration for compression adapters with typed configuration

CompressionChunkAdapter

Compression service adapter with typed configuration

EncryptionAdapterConfig

Configuration for encryption adapters with required key material

EncryptionChunkAdapter

Encryption service adapter with required configuration

ServiceAdapterFactory

Generic factory for creating any service adapter

ServiceChunkAdapter

Generic adapter that wraps any service as a ChunkProcessor