Skip to main content

execute_buffered

Function execute_buffered 

Source
pub async fn execute_buffered(
    catalog: &Catalog,
    route_name: &str,
    legs: &[ChainLeg],
    req: &ChatRequest,
) -> Result<Completion, GatewayError>
Expand description

Buffered (non-streaming) executor: walk legs, fully buffering each. Any failure (start or mid-stream) advances to the next leg. Nothing is ever flushed to the client, so full-chain fallback is preserved.

Unlike execute_chain, the connection phase here is NOT yet wrapped in the per-leg circuit-breaker/retry — a tracked follow-up. The streaming path trades that resilience plumbing for guaranteed whole-response fallback.

Delegates to execute_buffered_with_timeouts with StreamTimeouts::default.