Skip to main content

execute_buffered_with_timeouts

Function execute_buffered_with_timeouts 

Source
pub async fn execute_buffered_with_timeouts(
    catalog: &Catalog,
    route_name: &str,
    legs: &[ChainLeg],
    req: &ChatRequest,
    t: StreamTimeouts,
) -> Result<Completion, GatewayError>
Expand description

Buffered executor with explicit first-chunk and idle timeouts. Walk legs, fully buffering each. Any failure (start, timeout, or mid-stream) advances to the next leg. Nothing is ever flushed to the client, so full-chain fallback is preserved.