Expand description
Batch processing support for efficient inference
Structs§
- Batch
Config - Batch processing configuration
- Batch
Processor - Batch processor for efficient inference
- Batch
Request - Batch request with metadata
- Batch
Response - Batch response with results
- Batch
Stats - Batch statistics for monitoring
Enums§
- Batching
Strategy - Batching strategies for different use cases
- Priority
- Priority levels for batch requests