Expand description
Offline batch embedding generation with incremental updates
This module provides comprehensive batch processing capabilities for generating embeddings offline, with support for incremental updates, resumable jobs, and efficient resource utilization with SciRS2 integration.
Structs§
- Batch
Input - Batch input specification
- Batch
Job - Batch job definition
- Batch
JobConfig - Job-specific configuration
- Batch
Output - Batch output specification
- Batch
Processing Config - Configuration for batch processing
- Batch
Processing Manager - Batch processing manager for offline embedding generation
- Batch
Processing Result - Batch processing result
- Batch
Processing Stats - Batch processing statistics
- Incremental
Config - Incremental processing configuration
- JobCheckpoint
- Job checkpoint for resumability
- JobProgress
- Job progress information
- Memory
Optimized Batch Iterator - Memory-optimized batch iterator for large datasets
- Output
Config - Output configuration
- Output
Info - Output information
- Parallel
Batch Config - Configuration for parallel batch processing
- Parallel
Batch Processor - Advanced parallel batch processor using SciRS2 and Rayon
- Parallel
Processing Stats - Statistics for parallel batch processing
- Quality
Metrics - Quality metrics for batch processing
- Retry
Config - Retry configuration
Enums§
- Input
Type - Input types
- JobStatus
- Job status
- Output
Format - Output formats
- Partitioning
Strategy - Partitioning strategy