ai_lib_rust/batch/
mod.rs

1//! 请求批处理模块：提供高效的批量请求收集和执行功能。
2//!
3//! # Request Batching Module
4//!
5//! This module provides efficient request batching capabilities for optimizing
6//! throughput when dealing with multiple AI requests that can be processed together.
7//!
8//! ## Overview
9//!
10//! Batching is essential for:
11//! - Reducing API call overhead by grouping multiple requests
12//! - Optimizing network utilization with concurrent execution
13//! - Managing rate limits through controlled batch sizes
14//! - Improving overall throughput for bulk operations
15//!
16//! ## Key Components
17//!
18//! | Component | Description |
19//! |-----------|-------------|
20//! | [`BatchCollector`] | Collects requests until batch criteria are met |
21//! | [`BatchConfig`] | Configuration for batch size, timing, and auto-flush |
22//! | [`BatchItem`] | Wrapper for individual batch items with metadata |
23//! | [`BatchExecutor`] | Executes batches with configurable strategies |
24//! | [`BatchStrategy`] | Execution strategy (Sequential, Parallel, Concurrent) |
25//!
26//! ## Example
27//!
28//! ```rust
29//! use ai_lib_rust::batch::{BatchCollector, BatchConfig, BatchItem};
30//!
31//! // Create a collector with max batch size of 10
32//! let config = BatchConfig::new().with_max_batch_size(10);
33//! let collector: BatchCollector<String> = BatchCollector::new(config);
34//!
35//! // Add items to the batch
36//! collector.add_data("request_1".to_string());
37//! collector.add_data("request_2".to_string());
38//!
39//! // Check if batch should be flushed
40//! if collector.should_flush() {
41//!     let items = collector.drain();
42//!     // Process items...
43//! }
44//! ```
45//!
46//! ## Strategies
47//!
48//! - **Sequential**: Process items one at a time, preserving order
49//! - **Parallel**: Process all items concurrently with no limit
50//! - **Concurrent**: Process up to N items concurrently (recommended for rate-limited APIs)
51
52mod collector;
53mod executor;
54
55pub use collector::{BatchAddResult, BatchCollector, BatchConfig, BatchItem};
56pub use executor::{BatchError, BatchExecutor, BatchExecutorConfig, BatchResult, BatchStrategy};
ai_lib_rust/batch/mod.rs

ai_lib_rust/batch/
mod.rs
