process_batch_smart

ai_lib::api::chat::batch_utils

Function process_batch_smart 

Source 
pub async fn process_batch_smart<T: ChatApi + ?Sized>(
    api: &T,
    requests: Vec<ChatCompletionRequest>,
    concurrency_limit: Option<usize>,
) -> Result<Vec<Result<ChatCompletionResponse, AiLibError>>, AiLibError>
Expand description

Smart batch processing: automatically choose processing strategy based on request type and size