pub fn proxy(
upstream: &str,
req: &Request,
client: &mut TcpStream,
connect_timeout: Duration,
) -> Result<(), String>Expand description
Forward a request to a model’s endpoint and relay the response back.
Relayed in chunks rather than buffered, so a streamed completion reaches the caller token by token. Buffering here would hold every token until the model finished — turning a live stream into a long silence followed by a wall of text, which is the single most noticeable way a gateway ruins a chat UI.